Date: Tue, 21 Feb 2023 19:31:01 -0800 From: Mark Millard <marklmi@yahoo.com> To: Peter <pmc@citylink.dinoex.sub.org> Cc: freebsd-stable@freebsd.org Subject: Re: [analysis] 13.2 BETA2: how do debug META_MODE? Message-ID: <BBAA94D6-0BE1-4A1D-9E1B-9FC35EBB12A2@yahoo.com> In-Reply-To: <Y/V%2BymMyb91kvnPM@disp.intra.daemon.contact> References: <Y/V%2BymMyb91kvnPM@disp.intra.daemon.contact>
next in thread | previous in thread | raw e-mail | index | archive | help
On Feb 21, 2023, at 18:32, Peter <pmc@citylink.dinoex.sub.org> wrote: > It appears as my issue goes away by itself at the fourth subsequent > build. > >> # cd /usr/src/ >> # env WITH_META_MODE=yes make buildworld >> # env WITH_META_MODE=yes make installworld >> # env WITH_META_MODE=yes make buildworld (again #0) >> ## no more rebuilds below? >> # env WITH_META_MODE=yes make buildworld (again #1) >> # env WITH_META_MODE=yes make buildworld (again #2) >> >> "again #0" will rebuild llvm/clang. The other two "again"s >> will not. > > Strangely, I observed rather the opposite: the issue was with those > nodes that do *not* get installed. Those that do get installed, they > behaved as expected. My older notes from 2021 were better at being explicit about the initial context assumed: QUOTE Given an already built, installed and booted system version, I've noted a big difference for META_MODE in 2 rebuild contexts (no source updates involved) END QUOTE In other words, all the build activity was rebuild-same-version activity or reinstall-same-version activity in my example sequence. So I expect that the attempted comparison happens to not have a matching context to make the comparison with. > Details: > > When starting a build, there are some programs in > /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ from the last build. > These might be suitable for the new build (or they might not, who > knows) - but then, very early in the buildworld, many of these files > are copied from the running base system, with their mtime preserved. > So now these files do not have an mtime of the last build, but of the > last *install* of this running instance. (And that may be too late.) > This is done here - and I have no idea how these files are selected: > >> -------------------------------------------------------------- >>>>> Rebuilding the temporary build tree >> -------------------------------------------------------------- > [...] >> Building /usr/obj/usr/src/amd64.amd64/tools/build/host-symlinks >> Linking host tools into /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin > > Over all, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ tends to become > a zoo: some files are copied from the base system, some are installed > during the build, some are not replaced at all. > > I tried to figure what exactly is happening, with the example of > the make-roken + roken.h. > I start the upgrade to 13.2 with an empty obj tree (because after > switching and rebasing branches, my mtimes are recreated from the > latest commit - and they will certainly not align with the old ones). > > In the first build the binaries in the obj tree have to be created. > But since the running system is still 13.1, these binaries get all > stamped for 13.1. > > In the second build -now running 13.2- META_MODE decides that the > mtimes are now fine and does not rebuild. > META_MODE has no notion of the currently running OS version, it only > considers mtimes and cannot detect that these dependencies are stamped > with an old OS version. In terms of your example, in case the earlier was unclear: In my test sequence, 13.2 would have already been built, installed, and booted before the buildworld/installworld sequence that I listed. So my notes do not match your full sequence. They are more of a rebuild-already-existing sequence to analyze the behavior just for that simpler context than a realistic overall normal-build sequence. Having a installworld is only there to see what differences it makes compared to later not having such between such rebuild-already-existing runs. The isntallworld was an example of reinstall-same-version-as-already-running. > In the third build META_MODE finds some mtimes obsolete again, and > does rebuild toolchain binaries. This would not have any effect > because "install" would not copy them to the bin-dir if the same > binaries are already there. But in this scenario the old binaries > are from 13.1, so they are different, and "install" copies the new > ones in - and now everything that somehow depends on them will also > rebuild. > > Finally in the fourth build everything appears to be fine. > > Conclusion: > For a safe upgrade (specifically for a major version change) it is > not so much necessary to delete the obj tree before the first build, > but rather *after* the first build, i.e. after the base system has > been upgraded. > > Some timings: > > The base gets installed into a clean DESTDIR and used for the next > pass. The obj-trees are individually kept. > > Initial (obj trees deleted) > ( 4 vcore) > 230217231308.base.pass1.sst: real 9339.64 base w/ kernels > 230217231308.base.pass2.sst: real 7982.69 > 230217231308.admn.pass1.jail.sst:real 9346.90 jail w/ compiler > 230217231308.admn.pass2.jail.sst:real 5460.16 > 230217231308.data.pass1.jail.sst:real 4094.04 jail w/o compiler > 230217231308.data.pass2.jail.sst:real 143.39 > 230217231308.iamk.pass1.jail.sst:real 8050.27 jail w/ compiler > 230217231308.iamk.pass2.jail.sst:real 5226.32 > 230217231308.oper.pass1.jail.sst:real 2910.28 jail w/o compiler > 230217231308.oper.pass2.jail.sst:real 92.05 > 230217231308.rail.pass1.jail.sst:real 3236.29 jail w/o compiler > 230217231308.rail.pass2.jail.sst:real 99.49 > 230217231308.tele.pass1.jail.sst:real 3170.34 jail w/o compiler > 230217231308.tele.pass2.jail.sst:real 180.65 > > pass3 > (10 vcore) > 230222000242.base.std.sst: real 1162.80 base w/ kernels > 230222000242.admn.std.jail.sst:real 1759.15 jail w/ compiler > 230222000242.data.std.jail.sst:real 155.54 jail w/o compiler > 230222000242.iamk.std.jail.sst:real 1715.07 jail w/ compiler > 230222000242.oper.std.jail.sst:real 149.51 jail w/o compiler > 230222000242.rail.std.jail.sst:real 151.73 jail w/o compiler > 230222000242.tele.std.jail.sst:real 150.52 jail w/o compiler > > pass4 > (10 vcore) > 230222021535.edge.std.sst: real 1018.79 base w/ kernels > 230222021535.admn.std.jail.sst:real 101.61 jail w/ compiler > 230222021535.data.std.jail.sst:real 67.47 jail w/o compiler > 230222021535.iamk.std.jail.sst:real 100.91 jail w/ compiler > 230222021535.oper.std.jail.sst:real 66.52 jail w/o compiler > 230222021535.rail.std.jail.sst:real 68.00 jail w/o compiler > 230222021535.tele.std.jail.sst:real 66.54 jail w/o compiler === Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BBAA94D6-0BE1-4A1D-9E1B-9FC35EBB12A2>