From owner-svn-src-all@freebsd.org Fri Jul 19 00:15:26 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D5A0B530C; Fri, 19 Jul 2019 00:15:26 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F92C80307; Fri, 19 Jul 2019 00:15:26 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5B32D18424; Fri, 19 Jul 2019 00:15:26 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x6J0FQrJ023962; Fri, 19 Jul 2019 00:15:26 GMT (envelope-from bdrewery@FreeBSD.org) Received: (from bdrewery@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x6J0FPCM023960; Fri, 19 Jul 2019 00:15:25 GMT (envelope-from bdrewery@FreeBSD.org) Message-Id: <201907190015.x6J0FPCM023960@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bdrewery set sender to bdrewery@FreeBSD.org using -f From: Bryan Drewery Date: Fri, 19 Jul 2019 00:15:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r350119 - head/share/mk X-SVN-Group: head X-SVN-Commit-Author: bdrewery X-SVN-Commit-Paths: head/share/mk X-SVN-Commit-Revision: 350119 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 7F92C80307 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.956,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 00:15:26 -0000 Author: bdrewery Date: Fri Jul 19 00:15:25 2019 New Revision: 350119 URL: https://svnweb.freebsd.org/changeset/base/350119 Log: Rework some multi-output target dependency handling. This reworks my last commit in r301285 to more closely match what was in r241298 (but reverted in r294878). This is addressing "missing .meta file" rebuilds but also ensuring that files are always generated when needed in each case. Note that this is not a complete rework of the problem areas identified in r301285 as most are "good enough" right now as the new pattern is too verbose. It's only worth making this current change where headers may be generated in the INCS list; where missing .meta file rebuilds are spotted. --- Technical details follow --- Several attempts to deal with this problem of multi-output targets, with and without META MODE, were explained in r241298, r294878, and r301285. The general problem is with multi-output targets such as: foo.c foo.h: touch foo.c foo.h foo.c foo.h: touch foo.c touch foo.h foo.c foo.h: foo.in ./generator ${.ALLSRC} This pattern is problematic in jobs mode as both files end up being built concurrently and leads to races. With META MODE it is worse as both targets end up rebuilding if they lack a .meta file. So the generator is force built twice even though it is only needed once. There are also problems in that 'make foo.h' may be ran before 'make foo.c'; The order of make generating the targets is not guaranteed. An older attempted workaround to this (discussed in r294878) was: foo.h: foo.c foo.c: foo.in ./generator ${.ALLSRC} This appears fine except that if foo.h is missing and foo.c exists then foo.h will never be regenerated. This pattern is close to the solution in this commit though: foo.h: foo.c .NOMETA .if !exists(foo.h) foo.c: .PHONY .META .endif foo.c: foo.in ./generator ${.ALLSRC} There's 2 differences here: 1. foo.h will never expect to have a .meta file since the foo.c target will generate both and own the .meta file. 2. If foo.h does not exist then it needs to force foo.c to be rebuilt with .PHONY. That normally disables META MODE though so .META is given to tell bmake we do really expect a .meta file. This pattern cannot work with implicit suffix rules since the .c and .h files may be generated at different times (buildincludes vs depend/all). Sponsored by: Dell EMC MFC after: 2 weeks Modified: head/share/mk/bsd.dep.mk head/share/mk/bsd.snmpmod.mk Modified: head/share/mk/bsd.dep.mk ============================================================================== --- head/share/mk/bsd.dep.mk Thu Jul 18 21:58:51 2019 (r350118) +++ head/share/mk/bsd.dep.mk Fri Jul 19 00:15:25 2019 (r350119) @@ -121,17 +121,27 @@ CLEANFILES+= ${_LC} SRCS:= ${SRCS:S/${_YSRC}/${_YC}/} CLEANFILES+= ${_YC} .if !empty(YFLAGS:M-d) && !empty(SRCS:My.tab.h) -.ORDER: ${_YC} y.tab.h -y.tab.h: .NOMETA -${_YC} y.tab.h: ${_YSRC} +# Multi-output targets both expect a .meta file and will fight over it. Only +# allow it on the .c file instead. +y.tab.h: ${_YC} .NOMETA +# Force rebuild the .c file if any of its other outputs are missing. +.if !exists(y.tab.h) +${_YC}: .PHONY .META +.endif +${_YC}: ${_YSRC} ${YACC} ${YFLAGS} ${.ALLSRC} cp y.tab.c ${_YC} CLEANFILES+= y.tab.c y.tab.h .elif !empty(YFLAGS:M-d) .for _YH in ${_YC:R}.h -.ORDER: ${_YC} ${_YH} -${_YH}: .NOMETA -${_YC} ${_YH}: ${_YSRC} +# Multi-output targets both expect a .meta file and will fight over it. Only +# allow it on the .c file instead. +${_YH}: ${_YC} .NOMETA +# Force rebuild the .c file if any of its other outputs are missing. +.if !exists(${_YH}) +${_YC}: .PHONY .META +.endif +${_YC}: ${_YSRC} ${YACC} ${YFLAGS} -o ${_YC} ${.ALLSRC} SRCS+= ${_YH} CLEANFILES+= ${_YH} Modified: head/share/mk/bsd.snmpmod.mk ============================================================================== --- head/share/mk/bsd.snmpmod.mk Thu Jul 18 21:58:51 2019 (r350118) +++ head/share/mk/bsd.snmpmod.mk Fri Jul 19 00:15:25 2019 (r350119) @@ -12,9 +12,14 @@ GENSNMPTREEFLAGS+= -I${SHAREDIR}/snmpdefs ${MOD}_oid.h: ${MOD}_tree.def ${EXTRAMIBDEFS} ${EXTRAMIBSYMS} cat ${.ALLSRC} | gensnmptree ${GENSNMPTREEFLAGS} -e ${XSYM} > ${.TARGET} -.ORDER: ${MOD}_tree.c ${MOD}_tree.h -${MOD}_tree.h: .NOMETA -${MOD}_tree.c ${MOD}_tree.h: ${MOD}_tree.def ${EXTRAMIBDEFS} +# Multi-output targets both expect a .meta file and will fight over it. Only +# allow it on the .c file instead. +${MOD}_tree.h: ${MOD}_tree.c .NOMETA +# Force rebuild the .c file if any of its other outputs are missing. +.if !exists(${MOD}_tree.h) +${MOD}_tree.c: .PHONY .META +.endif +${MOD}_tree.c: ${MOD}_tree.def ${EXTRAMIBDEFS} cat ${.ALLSRC} | gensnmptree -f ${GENSNMPTREEFLAGS} -p ${MOD}_ .if defined(DEFS)