Date: Wed, 22 Feb 2023 03:32:42 +0100 From: Peter <pmc@citylink.dinoex.sub.org> To: freebsd-stable@freebsd.org Cc: Mark Millard <marklmi@yahoo.com> Subject: [analysis] Re: 13.2 BETA2: how do debug META_MODE? Message-ID: <Y/V%2BymMyb91kvnPM@disp.intra.daemon.contact>
next in thread | raw e-mail | index | archive | help
It appears as my issue goes away by itself at the fourth subsequent build. ># cd /usr/src/ ># env WITH_META_MODE=yes make buildworld ># env WITH_META_MODE=yes make installworld ># env WITH_META_MODE=yes make buildworld (again #0) >## no more rebuilds below? ># env WITH_META_MODE=yes make buildworld (again #1) ># env WITH_META_MODE=yes make buildworld (again #2) > >"again #0" will rebuild llvm/clang. The other two "again"s >will not. Strangely, I observed rather the opposite: the issue was with those nodes that do *not* get installed. Those that do get installed, they behaved as expected. Details: When starting a build, there are some programs in /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ from the last build. These might be suitable for the new build (or they might not, who knows) - but then, very early in the buildworld, many of these files are copied from the running base system, with their mtime preserved. So now these files do not have an mtime of the last build, but of the last *install* of this running instance. (And that may be too late.) This is done here - and I have no idea how these files are selected: > -------------------------------------------------------------- > >>> Rebuilding the temporary build tree > -------------------------------------------------------------- [...] > Building /usr/obj/usr/src/amd64.amd64/tools/build/host-symlinks > Linking host tools into /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin Over all, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/ tends to become a zoo: some files are copied from the base system, some are installed during the build, some are not replaced at all. I tried to figure what exactly is happening, with the example of the make-roken + roken.h. I start the upgrade to 13.2 with an empty obj tree (because after switching and rebasing branches, my mtimes are recreated from the latest commit - and they will certainly not align with the old ones). In the first build the binaries in the obj tree have to be created. But since the running system is still 13.1, these binaries get all stamped for 13.1. In the second build -now running 13.2- META_MODE decides that the mtimes are now fine and does not rebuild. META_MODE has no notion of the currently running OS version, it only considers mtimes and cannot detect that these dependencies are stamped with an old OS version. In the third build META_MODE finds some mtimes obsolete again, and does rebuild toolchain binaries. This would not have any effect because "install" would not copy them to the bin-dir if the same binaries are already there. But in this scenario the old binaries are from 13.1, so they are different, and "install" copies the new ones in - and now everything that somehow depends on them will also rebuild. Finally in the fourth build everything appears to be fine. Conclusion: For a safe upgrade (specifically for a major version change) it is not so much necessary to delete the obj tree before the first build, but rather *after* the first build, i.e. after the base system has been upgraded. Some timings: The base gets installed into a clean DESTDIR and used for the next pass. The obj-trees are individually kept. Initial (obj trees deleted) ( 4 vcore) 230217231308.base.pass1.sst: real 9339.64 base w/ kernels 230217231308.base.pass2.sst: real 7982.69 230217231308.admn.pass1.jail.sst:real 9346.90 jail w/ compiler 230217231308.admn.pass2.jail.sst:real 5460.16 230217231308.data.pass1.jail.sst:real 4094.04 jail w/o compiler 230217231308.data.pass2.jail.sst:real 143.39 230217231308.iamk.pass1.jail.sst:real 8050.27 jail w/ compiler 230217231308.iamk.pass2.jail.sst:real 5226.32 230217231308.oper.pass1.jail.sst:real 2910.28 jail w/o compiler 230217231308.oper.pass2.jail.sst:real 92.05 230217231308.rail.pass1.jail.sst:real 3236.29 jail w/o compiler 230217231308.rail.pass2.jail.sst:real 99.49 230217231308.tele.pass1.jail.sst:real 3170.34 jail w/o compiler 230217231308.tele.pass2.jail.sst:real 180.65 pass3 (10 vcore) 230222000242.base.std.sst: real 1162.80 base w/ kernels 230222000242.admn.std.jail.sst:real 1759.15 jail w/ compiler 230222000242.data.std.jail.sst:real 155.54 jail w/o compiler 230222000242.iamk.std.jail.sst:real 1715.07 jail w/ compiler 230222000242.oper.std.jail.sst:real 149.51 jail w/o compiler 230222000242.rail.std.jail.sst:real 151.73 jail w/o compiler 230222000242.tele.std.jail.sst:real 150.52 jail w/o compiler pass4 (10 vcore) 230222021535.edge.std.sst: real 1018.79 base w/ kernels 230222021535.admn.std.jail.sst:real 101.61 jail w/ compiler 230222021535.data.std.jail.sst:real 67.47 jail w/o compiler 230222021535.iamk.std.jail.sst:real 100.91 jail w/ compiler 230222021535.oper.std.jail.sst:real 66.52 jail w/o compiler 230222021535.rail.std.jail.sst:real 68.00 jail w/o compiler 230222021535.tele.std.jail.sst:real 66.54 jail w/o compiler
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Y/V%2BymMyb91kvnPM>