Date: Wed, 22 Feb 2023 21:09:54 -0800 From: Mark Millard <marklmi@yahoo.com> To: "Simon J. Gerraty" <sjg@juniper.net> Cc: Bryan Drewery <bdrewery@FreeBSD.org>, Current FreeBSD <freebsd-current@FreeBSD.org>, Peter <pmc@citylink.dinoex.sub.org> Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) Message-ID: <7FB6F619-6E71-4075-8A6C-573564371DD5@yahoo.com> In-Reply-To: <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> References: <B74790D9-FBC2-4818-BEAF-34E5B705C460@yahoo.com> <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <DB0C7B41-2101-4C5C-BFC8-3C95CC0B9F6F@yahoo.com> <73088.1611797582@kaos.jnpr.net> <CB7040D0-3BF4-496F-A54F-87E5378016E0@yahoo.com> <F6BF110D-7855-4A10-A53F-52B34282234F@yahoo.com> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Feb 22, 2023, at 19:47, Mark Millard <marklmi@yahoo.com> wrote:
> On Feb 22, 2023, at 17:18, Simon J. Gerraty <sjg@juniper.net> wrote:
>=20
>> Mark Millard <marklmi@yahoo.com> wrote:
>>=20
>>> Thanks for the information.
>>>=20
>>>> strings `which bmake` | grep META.IGNORE
>>>> .MAKE.META.IGNORE_PATHS
>>>> .MAKE.META.IGNORE_PATTERNS
>>>> ${.MAKE.META.IGNORE_PATHS:O:u:tA}
>>>=20
>>> The -dM output's "is newer than the target" lines
>>> show the path from before the above transformation.
>>> (The :tA results possibly could use another
>>> sort/uniq sequence for the realpath results?)
>>=20
>> That indicates the above IGNOREs are not working.
>>=20
>>> I've been pondering things because, so far, my
>>> attempts to experiment with this has failed to make
>>> the -dM output lines for the paths go away and it
>>> still does the related build activity. I've been
>>> trying the likes of:
>>>=20
>>> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide =
dd egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp =
mv patch realpath rm sed sh touch truncate uudecode uuencode
>>> xargs
>>=20
>> Is there anything under ${OBJTOP}/tmp that you don't want to ignore?
>=20
> More than just _bootstrap_tools_links entries end up in
> ${WORLDTMP}/legacy/bin/ (so in ${WORLDTMP}/legacy/sbin/
> via the symbolic link pointing to ${WORLDTMP}/legacy/bin/ ).
> So: yes.
>=20
> Also, OBJTOP is not constant over all the parts of
> buildworld buildkernel . Having the late-substitution
> form of notation ${OBJTOP} might not be appropriate
> for the content of .MAKE.META.IGNORE_PATHS .
>=20
> I'm trying to figure out if there is a stable way of
> getting a path that would not suffer variability
> via late substitution.=20
>=20
>> Otherwise you could simply use
>>=20
>> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/
>=20
> (Ignoring the variability of OBJTOP issue . . .)
>=20
> I do not expect that would work: ignoring things
> it likely should not.
>=20
> Also, I'd rather grow a smaller set of ignores
> gradually to make it easier to detect if an
> addition starts causing a problem and can be
> backed out. Starting with everything ignored
> would make things much harder to figure out
> when ignoring creates a problem.
>=20
>> You might need ${OBJTOP:tA}/tmp/
>> or both.
>>=20
>>> .MAKE.META.IGNORE_PATHS+=3D =
${OBJTOP}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
>>> .endfor
>>> .for ignore_other_tool in ctfconvert objcopy nm
>>> .MAKE.META.IGNORE_PATHS+=3D =
${OBJTOP}/tmp/usr/bin/${ignore_other_tool}
>>> .endfor
>>>=20
>>> in what I use for make.conf via:
>>>=20
>>> __MAKE_CONF=3D/usr/home/root/src.configs/make.conf
>>>=20
>>> It is using paths that match the -dM output lines ( sbin
>>> use despite sbin -> ../bin being a symbolic link).
>>>=20
>>> Note: WORLDTMP is not defined that early, thus the ${OBJTOP}/tmp
>>> use.
>>>=20
>>> -V.MAKE.META.IGNORE_PATHS is showing the paths I would
>>> expect, matching the -dM lines.
>>=20
>> Do you have example?
>> I really need to add some unit-tests for these...
>=20
> You may want to wait while I see if I can come up with
> a better example context to show things. I only just
> noticed the late-substitution potential issue and
> started looking for a way to avoid it, for example.
> (Either a value that does not vary or a form of
> causing up-front substitutions in my make.conf .)
>=20
I got some useful information about one type of context
that is odd and creates a far mount of "is newer than
the target" notices.
This is an example of something that always "rebuilds"
via a . . ./sbin/realpath "is newer":
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/aac/machine.meta: 23: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/aac/machine
It turns out that the .meta file is for a symbolic link:
# ls -Tld =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/aac/machine
lrwxr-xr-x 1 root wheel 31 Feb 22 20:19:27 2023 =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/aac/machine -> =
/usr/main-src/sys/amd64/include
. . ./sys/modules/*/machine being a symbolic link to a directory is =
normal.
It turns out that . . ./sbin/ln "is newer" examples are
tied to a similar issue:
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw=
k/awkgram.tab.h.meta: 12: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/ln' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw=
k/awkgram.tab.h
But . . .
# ls -Tld =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw=
k/awkgram.tab.h
lrwxr-xr-x 1 root wheel 9 Feb 22 20:18:24 2023 =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw=
k/awkgram.tab.h -> awkgram.h
The .meta file is for a symbolic link, to a file for this
type of context, not to a directory. (All the examples
happen for what I'm looking at happen to be symbolic
links to header files.)
It appears that for symbolic links being the target, META_MODE does
not respect .MAKE.META.IGNORE_PATHS .
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7FB6F619-6E71-4075-8A6C-573564371DD5>