Date: Thu, 23 Feb 2023 13:44:57 -0800 From: Mark Millard <marklmi@yahoo.com> To: "Simon J. Gerraty" <sjg@juniper.net> Cc: Bryan Drewery <bdrewery@FreeBSD.org>, Current FreeBSD <freebsd-current@FreeBSD.org>, Peter <pmc@citylink.dinoex.sub.org> Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) Message-ID: <CEABBA9E-7D06-49C0-B593-BBA30AD1C8C4@yahoo.com> In-Reply-To: <30.1677183903@kaos.jnpr.net> References: <B74790D9-FBC2-4818-BEAF-34E5B705C460@yahoo.com> <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <DB0C7B41-2101-4C5C-BFC8-3C95CC0B9F6F@yahoo.com> <73088.1611797582@kaos.jnpr.net> <CB7040D0-3BF4-496F-A54F-87E5378016E0@yahoo.com> <F6BF110D-7855-4A10-A53F-52B34282234F@yahoo.com> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net> <B11DA944-90E7-42D9-81A4-145686767305@yahoo.com> <30.1677183903@kaos.jnpr.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[Note for "how many separate bmake instances are in that log?":
I do not know how to tell how many submakes are run total. It
was with -j32 on the threadripper 1950X, if that is was you
were after.]
On Feb 23, 2023, at 12:25, Simon J. Gerraty <sjg@juniper.net> wrote:
> Mark Millard <marklmi@yahoo.com> wrote:
>>>> . . .
>=20
>> I got past the issue using :=3D before reading the above.
>> (I'm also using MAKEOBJDIR instead of OBJTOP currently.)
>=20
> Per my last response, I'd be pretty sure MAKEOBJDIR is incorrect.
Which still leaves me experimenting to find a correct
reference. Do you know notation will always lead to the
same absolute path with the proper /usr/obj/BUILDs/*/usr/main-src/*.*/
prefix ? (I've been showing just the main-amd64-nodbg-clang
and amd64.amd64 combination but there are many more.)
>>>>> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/
>>>>=20
>>>> (Ignoring the variability of OBJTOP issue . . .)
>>>>=20
>>>> I do not expect that would work: ignoring things
>>>> it likely should not.
>>>=20
>>> Sure, but it may be useful as an experiment to ensure things are
>>> behaving as expected.
>>=20
>> As a test:
>>=20
>> .if ${.MAKE.LEVEL} =3D=3D 0
>> .MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR:tA}/tmp/
>> .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS}
>> .endif
>=20
> Lose the .if ${.MAKE.LEVEL} =3D=3D 0
> it is almost certainly keeping things from working as expected.
What do you want tested instead of MAKEOBJDIR ?
I'm taking a guess (no .MAKE.LEVEL use):
.MAKE.META.IGNORE_PATHS+=3D ${OBJTOP:tA}/tmp/
.MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS:tA}
>> I still get things like:
>>=20
>> =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/zlib/x86.meta: 23: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
>> Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/zlib/x86
>=20
> Because that will not be level 0 and so .MAKE.META.IGNORE_PATHS is not
> set.
I tried the above and I still get (picking to look for
tmp/legacy/usr/sbin/ln examples):
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h.meta: 22: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/ln' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h
--
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/asmc/opt_acpi.h.meta: 22: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/ln' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/asmc/opt_acpi.h
--
. . .
and (looking for realpath examples):
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/accf_dns/machine.meta: 23: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/accf_dns/machine
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/accf_data/machine.meta: 23: =
file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/accf_http/machine.meta: 23: =
file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/accf_http/machine
--
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/ae/machine.meta: 23: file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/ae/machine
=
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/acl_posix1e/machine.meta: 23: =
file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/realpath' is newer than the target...
Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/acl_posix1e/machine
--
. . .
>>=20
>> and:
>>=20
>> =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h.meta: 12: =
file =
'/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac=
y/usr/sbin/ln' is newer than the target...
>> Building =
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI=
C-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h
>>=20
>> for both of a pair of back-to-back runs of buildworld buildkernel.
>>=20
>> FYI: The file system is zfs with mounts that look
>> like:
>>=20
>> zoptb /zoptb
>> zoptb/BUILDs /usr/obj/BUILDs
>> . . .
>> zoptb/BUILDs/main-amd64-nodbg-clang =
/usr/obj/BUILDs/main-amd64-nodbg-clang
>> . . .
>> zoptb/ROOT/main-amd64 /
>> . . .
>> zoptb/tmp /tmp
>> . . .
>>=20
>> # bectl list
>> BE Active Mountpoint Space Created
>> 13S-amd64 - - 4.97G 2021-08-20 16:57
>> 13_0R-amd64 - - 4.30G 2021-08-20 16:56
>> 13_1R-amd64 - - 4.12G 2022-03-10 12:38
>> main-amd64 NR / 7.42G 2023-02-19 15:37
>> old-main-amd64 - - 2.25G 2023-02-09 19:07
>>=20
>> (I use zfs in order to use bectl on a couple of
>> systems, not for redundancy.)
>>=20
>>=20
>>>> Also, I'd rather grow a smaller set of ignores
>>>> gradually to make it easier to detect if an
>>>> addition starts causing a problem and can be
>>>> backed out. Starting with everything ignored
>>>> would make things much harder to figure out
>>>> when ignoring creates a problem.
>>>=20
>>> Yes.
>>>=20
>>>>=20
>>>>> You might need ${OBJTOP:tA}/tmp/
>>>>> or both.
>>>=20
>>> I found it necessary in the unit tests to add :tA to both TMPDIR
>>> and .OBJDIR to get sane result on one test platform.
>>>=20
>>>>>> It is using paths that match the -dM output lines ( sbin
>>>>>> use despite sbin -> ../bin being a symbolic link).
>>>=20
>>> use :tA if you want to ensure consistent results.
>>=20
>> So, for each:
>>=20
>> .MAKE.META.IGNORE_PATHS+=3D =
${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
>>=20
>> I need to form an overall :tA on the path? Something
>> like:
>>=20
>> .if ${.MAKE.LEVEL} =3D=3D 0
>=20
> I think you need to first get rid of that level 0 check before
> worrying about anything else.
Done. But it appears to make no difference.
>> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd =
egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv =
patch realpath rm sed sh touch truncate uudecode uuencode
>> xargs
>> IGNORELEGACY_${ignore_legacy_tool}=3D =
${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
>> .MAKE.META.IGNORE_PATHS+=3D ${IGNORELEGACY_${ignore_legacy_tool}:tA}
>> .endfor
>> .for ignore_other_tool in ctfconvert objcopy nm
>> IGNOREOTHER_${ignore_other_tool}=3D =
${MAKEOBJDIR}/tmp/usr/bin/${ignore_other_tool}
>> .MAKE.META.IGNORE_PATHS+=3D ${IGNOREOTHER_${ignore_other_tool}:tA}
>> .endfor
>> .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS}
>> .endif
>>=20
>> Such seems to make no difference to the text reported via
>> -dV -V.MAKE.META.IGNORE_PATHS in my context.
>=20
> Yes, right now I think your main problem is only setting
> .MAKE.META.IGNORE_PATHS at level 0
Not as far as I can see in the results.
(Note: I did not start with explicit use of .MAKE.LEVEL .
It was just something I'd added into my experiments
recently. Nothing that I've tried has worked. I keep
exploring, looking for evidence of some way being able
to control the behavior.)
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CEABBA9E-7D06-49C0-B593-BBA30AD1C8C4>