Date: Thu, 23 Feb 2023 00:07:54 -0800 From: Mark Millard <marklmi@yahoo.com> To: "Simon J. Gerraty" <sjg@juniper.net> Cc: Bryan Drewery <bdrewery@FreeBSD.org>, Current FreeBSD <freebsd-current@FreeBSD.org>, Peter <pmc@citylink.dinoex.sub.org> Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) Message-ID: <B11DA944-90E7-42D9-81A4-145686767305@yahoo.com> In-Reply-To: <72419.1677133429@kaos.jnpr.net> References: <B74790D9-FBC2-4818-BEAF-34E5B705C460@yahoo.com> <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <DB0C7B41-2101-4C5C-BFC8-3C95CC0B9F6F@yahoo.com> <73088.1611797582@kaos.jnpr.net> <CB7040D0-3BF4-496F-A54F-87E5378016E0@yahoo.com> <F6BF110D-7855-4A10-A53F-52B34282234F@yahoo.com> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Feb 22, 2023, at 22:23, Simon J. Gerraty <sjg@juniper.net> wrote:
> Mark Millard <marklmi@yahoo.com> wrote:
>>>=20
>>> Is there anything under ${OBJTOP}/tmp that you don't want to ignore?
>>=20
>> More than just _bootstrap_tools_links entries end up in
>> ${WORLDTMP}/legacy/bin/ (so in ${WORLDTMP}/legacy/sbin/
>> via the symbolic link pointing to ${WORLDTMP}/legacy/bin/ ).
>> So: yes.
>>=20
>> Also, OBJTOP is not constant over all the parts of
>> buildworld buildkernel . Having the late-substitution
>> form of notation ${OBJTOP} might not be appropriate
>> for the content of .MAKE.META.IGNORE_PATHS .
>=20
> Fwiw .MAKE.META.IGNORE_PATHS is evaluated when meta_init() is
> called after all the makefiles have been read - and had a
> chance to influence .MAKE.MODE, so I'm not sure how varaiablity of
> OBJTOP would matter?
To my knowledge, there is no place to find documentation
about when/how-often the .MAKE.META.IGNORE_PATHS original
text is reevaluated other than the above statement or
source code inspection. (Others have a similar status.)
-dV -V.MAKE.META.IGNORE_PATHS use does list ${__MAKE_SHELL}
but lists /bin/sh without the -dV . (So -V use does not
give a direct clue at what you report.)
I got past the issue using :=3D before reading the above.
(I'm also using MAKEOBJDIR instead of OBJTOP currently.)
>>> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/
>>=20
>> (Ignoring the variability of OBJTOP issue . . .)
>>=20
>> I do not expect that would work: ignoring things
>> it likely should not.
>=20
> Sure, but it may be useful as an experiment to ensure things are
> behaving as expected.
As a test:
.if ${.MAKE.LEVEL} =3D=3D 0
.MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR:tA}/tmp/
.MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS}
.endif
leads to:
# =
~/sys-build-scripts.amd64-host/make-main-amd64-nodbg-clang.amd64-host.sh =
-dV -V.MAKE.META.IGNORE_PATHS buildworld buildkernel
Script started, output file is =
/usr/obj/BUILDs/main-amd64-nodbg-clang/sys-typescripts/typescript-make-amd=
64-nodbg-clang-amd64-host-2023-02-22:23:29:01
${__MAKE_SHELL} /bin /lib /rescue /sbin /usr/bin /usr/lib =
/usr/sbin /usr/share /usr/include /usr/local/etc/libmap.d =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/
Script done, output file is =
/usr/obj/BUILDs/main-amd64-nodbg-clang/sys-typescripts/typescript-make-amd=
64-nodbg-clang-amd64-host-2023-02-22:23:29:01
Note: I'm currently avoiding -jN for 1<N for tests.
I still get things like:
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/zlib/x86.meta: 23: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/zlib/x86
and:
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h.meta: 12: =
file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/ln' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h
for both of a pair of back-to-back runs of buildworld buildkernel.
FYI: The file system is zfs with mounts that look
like:
zoptb /zoptb
zoptb/BUILDs /usr/obj/BUILDs
. . .
zoptb/BUILDs/main-amd64-nodbg-clang =
/usr/obj/BUILDs/main-amd64-nodbg-clang
. . .
zoptb/ROOT/main-amd64 /
. . .
zoptb/tmp /tmp
. . .
# bectl list
BE Active Mountpoint Space Created
13S-amd64 - - 4.97G 2021-08-20 16:57
13_0R-amd64 - - 4.30G 2021-08-20 16:56
13_1R-amd64 - - 4.12G 2022-03-10 12:38
main-amd64 NR / 7.42G 2023-02-19 15:37
old-main-amd64 - - 2.25G 2023-02-09 19:07
(I use zfs in order to use bectl on a couple of
systems, not for redundancy.)
>> Also, I'd rather grow a smaller set of ignores
>> gradually to make it easier to detect if an
>> addition starts causing a problem and can be
>> backed out. Starting with everything ignored
>> would make things much harder to figure out
>> when ignoring creates a problem.
>=20
> Yes.
>=20
>>=20
>>> You might need ${OBJTOP:tA}/tmp/
>>> or both.
>=20
> I found it necessary in the unit tests to add :tA to both TMPDIR
> and .OBJDIR to get sane result on one test platform.
>=20
>>>> It is using paths that match the -dM output lines ( sbin
>>>> use despite sbin -> ../bin being a symbolic link).
>=20
> use :tA if you want to ensure consistent results.
So, for each:
.MAKE.META.IGNORE_PATHS+=3D =
${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
I need to form an overall :tA on the path? Something
like:
.if ${.MAKE.LEVEL} =3D=3D 0
.for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd =
egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv =
patch realpath rm sed sh touch truncate uudecode uuencode
xargs
IGNORELEGACY_${ignore_legacy_tool}=3D =
${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
.MAKE.META.IGNORE_PATHS+=3D ${IGNORELEGACY_${ignore_legacy_tool}:tA}
.endfor
.for ignore_other_tool in ctfconvert objcopy nm
IGNOREOTHER_${ignore_other_tool}=3D =
${MAKEOBJDIR}/tmp/usr/bin/${ignore_other_tool}
.MAKE.META.IGNORE_PATHS+=3D ${IGNOREOTHER_${ignore_other_tool}:tA}
.endfor
.MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS}
.endif
Such seems to make no difference to the text reported via
-dV -V.MAKE.META.IGNORE_PATHS in my context.
>>> I really need to add some unit-tests for these...
>=20
> Done - not yet imported to FreeBSD though
>=20
>> You may want to wait while I see if I can come up with
>> a better example context to show things. I only just
>> noticed the late-substitution potential issue and
>> started looking for a way to avoid it, for example.
>> (Either a value that does not vary or a form of
>> causing up-front substitutions in my make.conf .)
>=20
> Ok
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B11DA944-90E7-42D9-81A4-145686767305>