Date: Fri, 9 Sep 2016 11:24:27 -0700 From: Bryan Drewery <bdrewery@FreeBSD.org> To: "Simon J. Gerraty" <sjg@FreeBSD.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r305634 - head/share/mk Message-ID: <c564348e-59dd-dec8-180e-7a29a4700194@FreeBSD.org> In-Reply-To: <201609090121.u891LZOO023156@repo.freebsd.org> References: <201609090121.u891LZOO023156@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --PcFSMTIux9EKEPS6KoGo3qwL1cV4LMvcV Content-Type: multipart/mixed; boundary="bgE9gdGpkrBXFgD5wOIBaia4GB6mFp6SX"; protected-headers="v1" From: Bryan Drewery <bdrewery@FreeBSD.org> To: "Simon J. Gerraty" <sjg@FreeBSD.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Message-ID: <c564348e-59dd-dec8-180e-7a29a4700194@FreeBSD.org> Subject: Re: svn commit: r305634 - head/share/mk References: <201609090121.u891LZOO023156@repo.freebsd.org> In-Reply-To: <201609090121.u891LZOO023156@repo.freebsd.org> --bgE9gdGpkrBXFgD5wOIBaia4GB6mFp6SX Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 9/8/16 6:21 PM, Simon J. Gerraty wrote: > Author: sjg > Date: Fri Sep 9 01:21:35 2016 > New Revision: 305634 > URL: https://svnweb.freebsd.org/changeset/base/305634 >=20 > Log: > Update to latest dirdeps.mk > =20 > Take advantage of new bmake feature to only consider Makefile.depend > as invalidating DIRDEPS_CACHE. > =20 > When bootstrapping allow more filtering via .MAKE.DEPENDFILE_BOOTSTRA= P_SED > =20 > Move some comments back to where they make sense. > =20 > meta.sys.mk: add META_COOKIE_TOUCH and META_NOPHONY to better handle = some > targets in meta mode vs non-meta mode. > Also use .MAKE.META.IGNORE_PATHS to ignore mtime of makefiles - which= do > not matter in meta mode. >=20 > Modified: > head/share/mk/dirdeps.mk > head/share/mk/meta.sys.mk >=20 > Modified: head/share/mk/dirdeps.mk > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/share/mk/dirdeps.mk Fri Sep 9 01:09:39 2016 (r305633) > +++ head/share/mk/dirdeps.mk Fri Sep 9 01:21:35 2016 (r305634) > @@ -1,5 +1,5 @@ > # $FreeBSD$ > -# $Id: dirdeps.mk,v 1.62 2016/03/16 00:11:53 sjg Exp $ > +# $Id: dirdeps.mk,v 1.73 2016/08/15 19:28:13 sjg Exp $ > =20 > # Copyright (c) 2010-2013, Juniper Networks, Inc. > # All rights reserved. > @@ -117,12 +117,17 @@ _DIRDEP_USE_LEVEL?=3D 0 > .if ${.MAKE.LEVEL} =3D=3D ${_DIRDEP_USE_LEVEL} > # only the first instance is interested in all this > =20 > -# First off, we want to know what ${MACHINE} to build for. > -# This can be complicated if we are using a mixture of ${MACHINE} spec= ific > -# and non-specific Makefile.depend* > - > .if !target(_DIRDEP_USE) > =20 > +# do some setup we only need once > +_CURDIR ?=3D ${.CURDIR} > +_OBJDIR ?=3D ${.OBJDIR} > + > +now_utc =3D ${%s:L:gmtime} > +.if !defined(start_utc) > +start_utc :=3D ${now_utc} > +.endif > + > .if ${MAKEFILE:T} =3D=3D ${.PARSEFILE} && empty(DIRDEPS) && ${.TARGETS= :Uall:M*/*} !=3D "" > # This little trick let's us do > # > @@ -145,15 +150,6 @@ MK_DIRDEPS_CACHE =3D no > # make sure we get the behavior we expect > .MAKE.SAVE_DOLLARS =3D no > =20 > -# do some setup we only need once > -_CURDIR ?=3D ${.CURDIR} > -_OBJDIR ?=3D ${.OBJDIR} > - > -now_utc =3D ${%s:L:gmtime} > -.if !defined(start_utc) > -start_utc :=3D ${now_utc} > -.endif > - > # make sure these are empty to start with > _DEP_TARGET_SPEC =3D > =20 > @@ -224,6 +220,10 @@ N_notmachine :=3D ${.MAKE.DEPENDFILE_PREFE > =20 > .endif # !target(_DIRDEP_USE) > =20 > +# First off, we want to know what ${MACHINE} to build for. > +# This can be complicated if we are using a mixture of ${MACHINE} spec= ific > +# and non-specific Makefile.depend* > + > # if we were included recursively _DEP_TARGET_SPEC should be valid. > .if empty(_DEP_TARGET_SPEC) > # we may or may not have included a dependfile yet > @@ -386,7 +386,7 @@ BUILD_DIRDEPS ?=3D yes > .if !defined(NO_DIRDEPS) && !defined(NO_DIRDEPS_BELOW) > .if ${MK_DIRDEPS_CACHE} =3D=3D "yes" > # this is where we will cache all our work > -DIRDEPS_CACHE?=3D ${_OBJDIR}/dirdeps.cache${.TARGETS:Nall:O:u:ts-:S,/,= _,g:S,^,.,:N.} > +DIRDEPS_CACHE?=3D ${_OBJDIR:tA}/dirdeps.cache${.TARGETS:Nall:O:u:ts-:S= ,/,_,g:S,^,.,:N.} > =20 > # just ensure this exists > build-dirdeps: > @@ -398,6 +398,9 @@ M_oneperline =3D @x@\\${.newline} $$x@ > # we do this via sub-make > BUILD_DIRDEPS =3D no > =20 > +# ignore anything but these > +.MAKE.META.IGNORE_FILTER =3D M*/${.MAKE.DEPENDFILE_PREFIX}* > + > dirdeps: dirdeps-cached > dirdeps-cached: ${DIRDEPS_CACHE} .MAKE > @echo "${TRACER}Using ${DIRDEPS_CACHE}" > @@ -670,7 +673,10 @@ _DEP_RELDIR :=3D ${RELDIR} > make(bootstrap-recurse) || \ > make(bootstrap-empty)) > =20 > -.if exists(${.CURDIR}/${.MAKE.DEPENDFILE:T}) > +# if we are bootstrapping create the default > +_want =3D ${.CURDIR}/${.MAKE.DEPENDFILE_DEFAULT:T} > + > +.if exists(${_want}) > # stop here > ${.TARGETS:Mboot*}: > .elif !make(bootstrap-empty) > @@ -680,12 +686,19 @@ _src !=3D cd ${.CURDIR} && for m in ${.MAK > .error cannot find any of ${.MAKE.DEPENDFILE_PREFERENCE:T}${.newline}U= se: bootstrap-empty > .endif > =20 > -_src?=3D ${.MAKE.DEPENDFILE:T} > +_src?=3D ${.MAKE.DEPENDFILE} > + > +.MAKE.DEPENDFILE_BOOTSTRAP_SED+=3D -e 's,${_src:E},${MACHINE},g' > =20 > # just create Makefile.depend* for this dir > bootstrap-this: .NOTMAIN > - @echo Bootstrapping ${RELDIR}/${.MAKE.DEPENDFILE:T} from ${_src:T} > - (cd ${.CURDIR} && sed 's,${_src:E},${MACHINE},g' ${_src} > ${.MAKE.DE= PENDFILE:T}) > + @echo Bootstrapping ${RELDIR}/${_want:T} from ${_src:T}; \ > + echo You need to build ${RELDIR} to correctly populate it. > +.if ${_src:T} !=3D ${.MAKE.DEPENDFILE_PREFIX:T} > + (cd ${.CURDIR} && sed ${.MAKE.DEPENDFILE_BOOTSTRAP_SED} ${_src} > ${_= want}) > +.else > + cp ${.CURDIR}/${_src:T} ${_want} > +.endif > =20 > # create Makefile.depend* for this dir and its dependencies > bootstrap: bootstrap-recurse > @@ -705,8 +718,8 @@ bootstrap-recurse: .NOTMAIN .MAKE > =20 > # create an empty Makefile.depend* to get the ball rolling. > bootstrap-empty: .NOTMAIN .NOMETA > - @echo Creating empty ${RELDIR}/${.MAKE.DEPENDFILE:T}; \ > + @echo Creating empty ${RELDIR}/${_want:T}; \ > echo You need to build ${RELDIR} to correctly populate it. > - @{ echo DIRDEPS=3D; echo ".include <dirdeps.mk>"; } > ${.CURDIR}/${.M= AKE.DEPENDFILE:T} > + @{ echo DIRDEPS=3D; echo ".include <dirdeps.mk>"; } > ${_want} > =20 > .endif >=20 > Modified: head/share/mk/meta.sys.mk > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/share/mk/meta.sys.mk Fri Sep 9 01:09:39 2016 (r305633) > +++ head/share/mk/meta.sys.mk Fri Sep 9 01:21:35 2016 (r305634) > @@ -26,6 +26,10 @@ > # absoulte path to what we are reading. > _PARSEDIR =3D ${.PARSEDIR:tA} > =20 > +.if !defined(SYS_MK_DIR) > +SYS_MK_DIR :=3D ${_PARSEDIR} > +.endif > + > META_MODE +=3D meta verbose > .MAKE.MODE ?=3D ${META_MODE} > =20 > @@ -110,10 +114,26 @@ _metaError: .NOMETA .NOTMAIN > =20 > .endif > =20 > +META_COOKIE_TOUCH=3D > +# some targets need to be .PHONY in non-meta mode > +META_NOPHONY=3D .PHONY > # Are we, after all, in meta mode? > .if ${.MAKE.MODE:Uno:Mmeta*} !=3D "" > MKDEP_MK =3D meta.autodep.mk > =20 > +# we can afford to use cookies to prevent some targets > +# re-running needlessly > +META_COOKIE_TOUCH=3D touch ${COOKIE.${.TARGET}:U${.OBJDIR}/${.TARGET}}= Could you use ?=3D here please? I have a META_COOKIE_TOUCH in local.sys.m= k that is slightly different for handling absolute pathed cookies (with =2EOBJDIR in them). Also note that you need to remove this cookie immediately in any target that uses it. I covered this a bit in my BSDCan presentation. The problem is if meta mode determines that a target is outdated, then partially rebuilds the target, then fails before touching the cookie. The next build may not consider the target out-of-date anymore and fail to retry. Consider something like: foo: foo.c target_install: cp foo ${DESTDIR}/usr/bin/ touch target_install It may consider this target out-of-date due to foo rebuilding (and detected in target_install.meta), but the cp may fail. The next build, target_install.meta was overwritten with the failed attempt, and is now newer than foo and still has its target_install cookie. Nothing will cause it to retry copying, and it proceeds on with the build without properly failing again. I hit this case in at least include/ (which went through a lot of further changes, finally of which was the removal of cookies in r300346) and some other cases which I don't recall the specifics for. > +META_NOPHONY=3D > + > +# some targets involve old pre-built targets > +# ignore mtime of shell > +# and mtime of makefiles does not matter in meta mode > +.MAKE.META.IGNORE_PATHS +=3D \ > + ${MAKEFILE} \ > + ${SHELL} \ > + ${SYS_MK_DIR} > + > # if we think we are updating dependencies,=20 > # then filemon had better be present > .if ${UPDATE_DEPENDFILE:Uyes:tl} !=3D "no" && !exists(/dev/filemon) >=20 --=20 Regards, Bryan Drewery --bgE9gdGpkrBXFgD5wOIBaia4GB6mFp6SX-- --PcFSMTIux9EKEPS6KoGo3qwL1cV4LMvcV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX0v5cAAoJEDXXcbtuRpfP+zIIAIabEawHgnsctmnHUdfsctKx lUJl7kz2Teil11cq/wNiD6vohkmEXNB+3zAh6iWkva3JpgCNdXcrqC1i1pcZM1GV bC5oE/b9imbp58EeaQlOrcev5qU5IOaGEMDnU1lUkoJnnqeQYw0VhLWdaTgErEWK /XRq2kFGxc7QOYoRzexf4SC8RjthbZY/AquR6ShQeIGXmefDzBZE2Sc9YcIaVjmx JuOOnw3ywoP0hdb5vLAHxw0dZ7E1ZK743S2XrYe4r2BHFU9nqpoOCwl0R987Gycx YBgnp9PqjBPLd7fN5UxPqz77wz+B99PXhKM1FlMWpBpBhNbmr+WKd0VFO+o714c= =t09I -----END PGP SIGNATURE----- --PcFSMTIux9EKEPS6KoGo3qwL1cV4LMvcV--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c564348e-59dd-dec8-180e-7a29a4700194>