Date: Tue, 6 May 2025 16:15:43 +0100 From: Nuno Teixeira <eduardo@freebsd.org> To: Mark Millard <marklmi@yahoo.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: incremental bulds from scratch with beinstall.sh Message-ID: <CAFDf7ULuKpL_zxWKUkPCoHeQf-=YHdM=w43HBP7buinzJoh0Pw@mail.gmail.com> In-Reply-To: <28F2BDE7-5903-4C04-A570-6A407F19D5F2@yahoo.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Hello Mark, Definitely I'm not getting the results I want with WITH_META_MODE using BEs since most of the times I end up rebuilding almost everything in new BE. Should I stop using WITH_META_MODES a go straight to clean builds (clean /usr/obj) instead? My first concern is to speed up builds expecially on rpi4. Any hints are welcome. Thanks, Mark Millard <marklmi@yahoo.com> escreveu (segunda, 5/05/2025 à(s) 23:18): > Nuno Teixeira <eduardo_at_freebsd.org> wrote on > Date: Mon, 05 May 2025 20:37:09 UTC : > > > (...) > > > > Don't forget to `env NO_PKG_UPGRADE=yes beinstall.sh` to not mess with > > ports and stuff. > > > > Nuno Teixeira <eduardo@freebsd.org> escreveu (segunda, 5/05/2025 à(s) > > 21:34): > > > > > Hello, > > > > > > Using incremental WITH_META_MODE builds, after installation with > > > beinstall.sh, building same src, a complete compilation happens. > > > > > > If someone uses this script, could you please do the following test: > > > > > > WITH_META_MODE=yes (/etc/src-env.conf) > > > filemon module loaded > > > > > > # cd /usr/src > > > # make buildworld-jobs buildkernel-jobs > > The above used older commands and files from before > the following install. META_MODE recorded the use of > those commands. > > Example .meta mode file content: > > # Meta data file > /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++/libc++.a.meta > CMD @echo building static c++ library > CMD @rm -f libc++.a > CMD ar -crsD libc++.a algorithm.o any.o atomic.o barrier.o bind.o > charconv.o chrono.o condition_variable.o condition_variable_destructor.o > debug.o exception.o filesystem/directory_iterator.o filesyste > m/int128_builtins.o filesystem/operations.o functional.o future.o hash.o > ios.o iostream.o locale.o memory.o mutex.o mutex_destructor.o new.o > optional.o random.o random_shuffle.o regex.o shared_mutex.o > stdexcept.o string.o strstream.o system_error.o thread.o typeinfo.o > utility.o valarray.o variant.o vector.o cxxrt_auxhelper.o > cxxrt_dynamic_cast.o cxxrt_exception.o cxxrt_guard.o cxxrt_libelftc_dem_g > nu3.o cxxrt_memory.o cxxrt_stdexcept.o cxxrt_terminate.o cxxrt_typeinfo.o > CWD /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++ > TARGET libc++.a > -- command output -- > building static c++ library > > -- filemon acquired metadata -- > # filemon version 5 > # Target pid 22471 > # Start 1611359217.214996 > V 5 > E 22961 /bin/sh > R 22961 /etc/libmap.conf > R 22961 /var/run/ld-elf.so.hints > R 22961 /lib/libedit.so.7 > R 22961 /lib/libc.so.7 > R 22961 /lib/libncursesw.so.9 > R 22961 /usr/share/locale/C.UTF-8/LC_CTYPE > F 22961 22962 > E 22962 > /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/rm > R 22962 /etc/libmap.conf > R 22962 /var/run/ld-elf.so.hints > R 22962 /lib/libc.so.7 > R 22962 /usr/share/locale/C.UTF-8/LC_CTYPE > D 22962 libc++.a > X 22962 0 0 > . . . > > So META_MODE has lots of files that were used > that it can later detect being newer than the > prior build results, leading to rebuilds based > on those newer files. > > > > # ./tools/build/beinstall.sh > > The new be will have various updated files > that could be different by content and, for > commands, could behave differently than those > used to do the prior build. Detecting newer > time stamps on such used files leads to > rebuild activity. > > More than /usr/src/ and /usr/obj/ content > are involved as well. > > Note that the new be is based on somewhat > different files than the original > buildworld-jobs buildkernel-jobs was based > on. > > > > # reboot > > > > > (I presume booting into the new be here.) > > > > # cd /usr/src > > > # make buildworld-jobs buildkernel-jobs > > META_MODE will notice when commands are used > that are newer than when the prior build was > done. Similarly for other files that may be > read. It will make sure that the newer commands > and files are allowed to produce new results > that potentially could be distinct in content > from what the old context produced for results. > > > > > > > Since src and obj are the same from one BE to newer BE, minimal > > > compilation should happen, not a full one. > > META_MODE is more careful than that. > > Note: I'm not claiming that new behavior that is > needed is likely for lots of the files with new > dates. But META_MODE is biased to avoiding leaving > in place something that should have been updated. > > > > > > > Am I missing something here? > > > > > Note that make has a -dM option: > > M Print debugging information about “meta” mode > decisions > about targets. > > So, for example, > > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/awk' > is newer than the target... > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cap_mkdb' > is newer than the target... > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cat' > is newer than the target... > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cp' > is newer than the target... > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/crunchgen' > is newer than the target... > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/crunchide' > is newer than the target... > file > '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/dd' > is newer than the target... > . . . > > Various . . ./tmp/legacy/. . ./*bin/ actually were > links to files. > > Ultimately buildworld then installworld lead to new > dates for a bunch of files used. Later use of > META_MODE notices such and rebuilds based on the > newer files. (It is a lot of detail to go through > it all.) > > Back in 2021 and 2023 I got help with exploring > avoiding lots of these. But, in the end, it > involved use of experimental code in > share/mk/src.sys.obj.mk to provide a new > definition to use to build some paths with. > > The experiments were an unsupported activity that > produced an unsupported change to allow > configurable enabling of taking risks with not > updating files that possibly should be updated. > > === > Mark Millard > marklmi at yahoo.com > > -- Nuno Teixeira FreeBSD UNIX: <eduardo@FreeBSD.org> Web: https://FreeBSD.org [-- Attachment #2 --] <div dir="ltr"><div><div>Hello Mark,<br><br></div><span role="heading" class="gmail-yKMVIe">Definitely I'm not getting the results I want with WITH_META_MODE using BEs since most of the times I end up rebuilding almost everything in new BE.<br></span></div><div><span role="heading" class="gmail-yKMVIe"><br></span></div><div><span role="heading" class="gmail-yKMVIe">Should I stop using WITH_META_MODES a go straight to clean builds (clean /usr/obj) instead?<br></span></div><div>My first concern is to speed up builds expecially on rpi4.<br><br></div><div>Any hints are welcome.<br><br></div><div>Thanks,</div><div><span role="heading" class="gmail-yKMVIe"><br></span></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">Mark Millard <<a href="mailto:marklmi@yahoo.com">marklmi@yahoo.com</a>> escreveu (segunda, 5/05/2025 à(s) 23:18):<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Nuno Teixeira <<a href="http://eduardo_at_freebsd.org" rel="noreferrer" target="_blank">eduardo_at_freebsd.org</a>> wrote on<br> Date: Mon, 05 May 2025 20:37:09 UTC :<br> <br> > (...)<br> > <br> > Don't forget to `env NO_PKG_UPGRADE=yes beinstall.sh` to not mess with<br> > ports and stuff.<br> > <br> > Nuno Teixeira <<a href="mailto:eduardo@freebsd.org" target="_blank">eduardo@freebsd.org</a>> escreveu (segunda, 5/05/2025 à(s)<br> > 21:34):<br> > <br> > > Hello,<br> > ><br> > > Using incremental WITH_META_MODE builds, after installation with<br> > > beinstall.sh, building same src, a complete compilation happens.<br> > ><br> > > If someone uses this script, could you please do the following test:<br> > ><br> > > WITH_META_MODE=yes (/etc/src-env.conf)<br> > > filemon module loaded<br> > ><br> > > # cd /usr/src<br> > > # make buildworld-jobs buildkernel-jobs<br> <br> The above used older commands and files from before<br> the following install. META_MODE recorded the use of<br> those commands.<br> <br> Example .meta mode file content:<br> <br> # Meta data file /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++/libc++.a.meta<br> CMD @echo building static c++ library<br> CMD @rm -f libc++.a<br> CMD ar -crsD libc++.a algorithm.o any.o atomic.o barrier.o bind.o charconv.o chrono.o condition_variable.o condition_variable_destructor.o debug.o exception.o filesystem/directory_iterator.o filesyste<br> m/int128_builtins.o filesystem/operations.o functional.o future.o hash.o ios.o iostream.o locale.o memory.o mutex.o mutex_destructor.o new.o optional.o random.o random_shuffle.o regex.o shared_mutex.o<br> stdexcept.o string.o strstream.o system_error.o thread.o typeinfo.o utility.o valarray.o variant.o vector.o cxxrt_auxhelper.o cxxrt_dynamic_cast.o cxxrt_exception.o cxxrt_guard.o cxxrt_libelftc_dem_g<br> nu3.o cxxrt_memory.o cxxrt_stdexcept.o cxxrt_terminate.o cxxrt_typeinfo.o<br> CWD /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++<br> TARGET libc++.a<br> -- command output --<br> building static c++ library<br> <br> -- filemon acquired metadata --<br> # filemon version 5<br> # Target pid 22471<br> # Start 1611359217.214996<br> V 5<br> E 22961 /bin/sh<br> R 22961 /etc/libmap.conf<br> R 22961 /var/run/ld-elf.so.hints<br> R 22961 /lib/libedit.so.7<br> R 22961 /lib/libc.so.7<br> R 22961 /lib/libncursesw.so.9<br> R 22961 /usr/share/locale/C.UTF-8/LC_CTYPE<br> F 22961 22962<br> E 22962 /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/rm<br> R 22962 /etc/libmap.conf<br> R 22962 /var/run/ld-elf.so.hints<br> R 22962 /lib/libc.so.7<br> R 22962 /usr/share/locale/C.UTF-8/LC_CTYPE<br> D 22962 libc++.a<br> X 22962 0 0<br> . . .<br> <br> So META_MODE has lots of files that were used<br> that it can later detect being newer than the<br> prior build results, leading to rebuilds based<br> on those newer files.<br> <br> > > # ./tools/build/beinstall.sh<br> <br> The new be will have various updated files<br> that could be different by content and, for<br> commands, could behave differently than those<br> used to do the prior build. Detecting newer<br> time stamps on such used files leads to<br> rebuild activity.<br> <br> More than /usr/src/ and /usr/obj/ content<br> are involved as well.<br> <br> Note that the new be is based on somewhat<br> different files than the original<br> buildworld-jobs buildkernel-jobs was based<br> on.<br> <br> > > # reboot<br> > ><br> <br> (I presume booting into the new be here.)<br> <br> > > # cd /usr/src<br> > > # make buildworld-jobs buildkernel-jobs<br> <br> META_MODE will notice when commands are used<br> that are newer than when the prior build was<br> done. Similarly for other files that may be<br> read. It will make sure that the newer commands<br> and files are allowed to produce new results<br> that potentially could be distinct in content<br> from what the old context produced for results.<br> <br> > ><br> > > Since src and obj are the same from one BE to newer BE, minimal<br> > > compilation should happen, not a full one.<br> <br> META_MODE is more careful than that.<br> <br> Note: I'm not claiming that new behavior that is<br> needed is likely for lots of the files with new<br> dates. But META_MODE is biased to avoiding leaving<br> in place something that should have been updated.<br> <br> > ><br> > > Am I missing something here?<br> > ><br> <br> Note that make has a -dM option:<br> <br> M Print debugging information about “meta” mode decisions<br> about targets.<br> <br> So, for example,<br> <br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/awk' is newer than the target...<br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cap_mkdb' is newer than the target...<br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cat' is newer than the target...<br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/cp' is newer than the target...<br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/crunchgen' is newer than the target...<br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/crunchide' is newer than the target...<br> file '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/usr/sbin/dd' is newer than the target...<br> . . .<br> <br> Various . . ./tmp/legacy/. . ./*bin/ actually were<br> links to files.<br> <br> Ultimately buildworld then installworld lead to new<br> dates for a bunch of files used. Later use of<br> META_MODE notices such and rebuilds based on the<br> newer files. (It is a lot of detail to go through<br> it all.)<br> <br> Back in 2021 and 2023 I got help with exploring<br> avoiding lots of these. But, in the end, it<br> involved use of experimental code in<br> share/mk/<a href="http://src.sys.obj.mk" rel="noreferrer" target="_blank">src.sys.obj.mk</a> to provide a new<br> definition to use to build some paths with.<br> <br> The experiments were an unsupported activity that<br> produced an unsupported change to allow<br> configurable enabling of taking risks with not<br> updating files that possibly should be updated.<br> <br> ===<br> Mark Millard<br> marklmi at <a href="http://yahoo.com" rel="noreferrer" target="_blank">yahoo.com</a><br> <br> </blockquote></div><div><br clear="all"></div><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><font color="#888888">Nuno Teixeira</font></div><div><div><font color="#888888"> FreeBSD UNIX: <eduardo@FreeBSD.org> Web: <a href="https://FreeBSD.org" rel="noreferrer" target="_blank">https://FreeBSD.org</a><br></font></div></div></div></div>;help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFDf7ULuKpL_zxWKUkPCoHeQf-=YHdM=w43HBP7buinzJoh0Pw>