Date: Thu, 23 Feb 2023 12:25:03 -0800 From: "Simon J. Gerraty" <sjg@juniper.net> To: Mark Millard <marklmi@yahoo.com> Cc: Bryan Drewery <bdrewery@FreeBSD.org>, Current FreeBSD <freebsd-current@FreeBSD.org>, Peter <pmc@citylink.dinoex.sub.org>, <sjg@juniper.net> Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) Message-ID: <30.1677183903@kaos.jnpr.net> In-Reply-To: <B11DA944-90E7-42D9-81A4-145686767305@yahoo.com> References: <B74790D9-FBC2-4818-BEAF-34E5B705C460@yahoo.com> <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <DB0C7B41-2101-4C5C-BFC8-3C95CC0B9F6F@yahoo.com> <73088.1611797582@kaos.jnpr.net> <CB7040D0-3BF4-496F-A54F-87E5378016E0@yahoo.com> <F6BF110D-7855-4A10-A53F-52B34282234F@yahoo.com> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net> <B11DA944-90E7-42D9-81A4-145686767305@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Mark Millard <marklmi@yahoo.com> wrote:
> >> Also, OBJTOP is not constant over all the parts of
> >> buildworld buildkernel . Having the late-substitution
> >> form of notation ${OBJTOP} might not be appropriate
> >> for the content of .MAKE.META.IGNORE_PATHS .
> >
> > Fwiw .MAKE.META.IGNORE_PATHS is evaluated when meta_init() is
> > called after all the makefiles have been read - and had a
> > chance to influence .MAKE.MODE, so I'm not sure how varaiablity of
> > OBJTOP would matter?
> =
> To my knowledge, there is no place to find documentation
> about when/how-often the .MAKE.META.IGNORE_PATHS original
> text is reevaluated other than the above statement or
> source code inspection. (Others have a similar status.)
True. That should be fixed.
.MAKE.META.IGNORE_PATHS is only evaluated once by make
after all makefiles have been read. It is use to populate a list.
By contrast .MAKE.META.IGNORE_PATTERNS and .MAKE.META.IGNORE_FILTER
are evaluated for every line of filemon output - but again all that
happens after all makefiles have been read.
> -dV -V.MAKE.META.IGNORE_PATHS use does list ${__MAKE_SHELL}
> but lists /bin/sh without the -dV . (So -V use does not
> give a direct clue at what you report.)
One of the local changes to bmake in FreeBSD is that =
without -dV -V shows the fully resolved value.
> I got past the issue using :=3D before reading the above.
> (I'm also using MAKEOBJDIR instead of OBJTOP currently.)
Per my last response, I'd be pretty sure MAKEOBJDIR is incorrect.
> =
> >>> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/
> >>
> >> (Ignoring the variability of OBJTOP issue . . .)
> >>
> >> I do not expect that would work: ignoring things
> >> it likely should not.
> >
> > Sure, but it may be useful as an experiment to ensure things are
> > behaving as expected.
> =
> As a test:
> =
> .if ${.MAKE.LEVEL} =3D=3D 0
> .MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR:tA}/tmp/
> .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS}
> .endif
Lose the .if ${.MAKE.LEVEL} =3D=3D 0
it is almost certainly keeping things from working as expected.
> I still get things like:
> =
> /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENE=
RIC-NODBG/modules/usr/main-src/sys/modules/zlib/x86.meta: 23: file '/usr/o=
bj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/s=
bin/realpath' is newer than the target...
> Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64=
/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/zlib/x86
Because that will not be level 0 and so .MAKE.META.IGNORE_PATHS is not
set.
> =
> and:
> =
> /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENE=
RIC-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h.meta: 12: fil=
e '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/leg=
acy/usr/sbin/ln' is newer than the target...
> Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64=
/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h
> =
> for both of a pair of back-to-back runs of buildworld buildkernel.
> =
> FYI: The file system is zfs with mounts that look
> like:
> =
> zoptb /zoptb
> zoptb/BUILDs /usr/obj/BUILDs
> . . .
> zoptb/BUILDs/main-amd64-nodbg-clang /usr/obj/BUILDs/main-amd64-nodbg-cl=
ang
> . . .
> zoptb/ROOT/main-amd64 /
> . . .
> zoptb/tmp /tmp
> . . .
> =
> # bectl list
> BE Active Mountpoint Space Created
> 13S-amd64 - - 4.97G 2021-08-20 16:57
> 13_0R-amd64 - - 4.30G 2021-08-20 16:56
> 13_1R-amd64 - - 4.12G 2022-03-10 12:38
> main-amd64 NR / 7.42G 2023-02-19 15:37
> old-main-amd64 - - 2.25G 2023-02-09 19:07
> =
> (I use zfs in order to use bectl on a couple of
> systems, not for redundancy.)
> =
> =
> >> Also, I'd rather grow a smaller set of ignores
> >> gradually to make it easier to detect if an
> >> addition starts causing a problem and can be
> >> backed out. Starting with everything ignored
> >> would make things much harder to figure out
> >> when ignoring creates a problem.
> >
> > Yes.
> >
> >>
> >>> You might need ${OBJTOP:tA}/tmp/
> >>> or both.
> >
> > I found it necessary in the unit tests to add :tA to both TMPDIR
> > and .OBJDIR to get sane result on one test platform.
> >
> >>>> It is using paths that match the -dM output lines ( sbin
> >>>> use despite sbin -> ../bin being a symbolic link).
> >
> > use :tA if you want to ensure consistent results.
> =
> So, for each:
> =
> .MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_l=
egacy_tool}
> =
> I need to form an overall :tA on the path? Something
> like:
> =
> .if ${.MAKE.LEVEL} =3D=3D 0
I think you need to first get rid of that level 0 check before
worrying about anything else.
> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd eg=
rep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv patc=
h realpath rm sed sh touch truncate uudecode uuencode
> xargs
> IGNORELEGACY_${ignore_legacy_tool}=3D ${MAKEOBJDIR}/tmp/legacy/usr/sbin/=
${ignore_legacy_tool}
> .MAKE.META.IGNORE_PATHS+=3D ${IGNORELEGACY_${ignore_legacy_tool}:tA}
> .endfor
> .for ignore_other_tool in ctfconvert objcopy nm
> IGNOREOTHER_${ignore_other_tool}=3D ${MAKEOBJDIR}/tmp/usr/bin/${ignore_o=
ther_tool}
> .MAKE.META.IGNORE_PATHS+=3D ${IGNOREOTHER_${ignore_other_tool}:tA}
> .endfor
> .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS}
> .endif
> =
> Such seems to make no difference to the text reported via
> -dV -V.MAKE.META.IGNORE_PATHS in my context.
Yes, right now I think your main problem is only setting
.MAKE.META.IGNORE_PATHS at level 0
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?30.1677183903>