Date: Fri, 27 May 2016 17:16:25 -0700 From: Bryan Drewery <bdrewery@FreeBSD.org> To: current@FreeBSD.org Subject: [CFT] WITH_META_MODE: Working incremental build Message-ID: <e25bdb07-8a72-f955-ffb2-5d2609b9d3fe@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Lhkh0vHaJdFXOMEpFo3pLgebOGDGVfN3R Content-Type: multipart/mixed; boundary="BEVLhl9gCjDQDIN5xWHaUKQpdA6q28qhB" From: Bryan Drewery <bdrewery@FreeBSD.org> To: current@FreeBSD.org Message-ID: <e25bdb07-8a72-f955-ffb2-5d2609b9d3fe@FreeBSD.org> Subject: [CFT] WITH_META_MODE: Working incremental build --BEVLhl9gCjDQDIN5xWHaUKQpdA6q28qhB Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Buildworld/Buildkernel now supports a working incremental build. Using -DNO_CLEAN is not needed as it is the default with it (there is no -DCLEAN for it). The normal -DNO_CLEAN feature is broken and risky since it does not track a ton of the dependencies in the build such as the compiler, other build tools, csu/crt, CFLAGS, build commands, mk files, etc. Note that originally WITH_META_MODE initiated a new build system but that was renamed to WITH_DIRDEPS_BUILD and is something different entirel= y. The way that "meta mode" works is to enable the bmake "meta mode" feature that tracks the build command for each target and uses the filemon(4) [1] module to track all files read/written/executed for building target. If any of these change (see src.conf(5) WITH_META_MODE for details) then the target will be rebuilt. This means that even changes to CFLAGS or the cross compiler will be detected and cause rebuilds. This is quite an aggressive system but it works and removes the need for cleaning any of the tree. In this mode .depend.* files (WITH_FAST_DEPEND) are not generated since they are mostly redundant with filemon(4)'s tracking. An example in how powerful this is, is that I have today been testing buildworld with the in-tree clang and also doing builds on the same objdir with CROSS_TOOLCHAIN=3Damd64-gcc. Everything using a compiler would rebuild and anything not using a compiler would not rebuild. No cleaning needed. To use this you must either add WITH_META_MODE=3Dyes to your environment or add it into /etc/src-env.conf (not /etc/src.conf or /etc/make.conf). You will also need to load the filemon(4) module with 'kldload filemon'. The build will show far less information than the normal build but will otherwise follow the same flow as buildworld. If you wish to see the full output as before you can use -DNO_SILENT or add NO_SILENT=3Dyes to your /etc/src-env.conf. If a build error occurs you can view the .meta file for the target to see what the build command was. For example if you see 'Building /usr/obj/root/git/freebsd/world32/root/git/freebsd /lib/libdwarf/dwarf_arange.So' then see the file /usr/obj/root/git/freebsd/world32/root/git/freebsd/lib/libdwarf/dwarf_ara= nge.So.meta. If you wish to see why something is being rebuilt you can pass the -dM flag to make to see meta mode debug information. TODO: - There is currently a bug with library linking that can cause the next build to consider libraries out-of-date when they are not. It causes a chain reaction of relinking but not rebuilds. - Improve WORLDTMP populating such that it is not done redundantly. - It's possible rescue/rescue has some quirks still. [1] Over the years filemon(4) was quite unstable, insecure, and slow. A lot of work has gone into it over the past 6 months in each of these areas. After r300893 it should be fine for this use. --=20 Regards, Bryan Drewery --BEVLhl9gCjDQDIN5xWHaUKQpdA6q28qhB-- --Lhkh0vHaJdFXOMEpFo3pLgebOGDGVfN3R Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJXSONZAAoJEDXXcbtuRpfPThcH/iyq1nysZeOxWh1fDy57Oi8H TTX6mn1ohLiQEqviIRp3ptGJJycfsi7zNsvqsdumh7ijn7NPFkQKd7pGiaMyHca2 hdCGf+izbgg10HvYScAj6PJnq9NJfUrGi3nuesr0VXVybigln9nk8ZJLMOG2ufkP Qllx7MJHSGD9OGTJYV0sF5l9Y8RZMk5SknWGqEZKRAKa4UJbg+kQfZUZL1Oy2562 a8NWgggJZQHbLrBDfqsJdRipKTrUPRLICPAYi0pMQpbHCXkS998NMjVkzvV4Tywr mUfg2Rtok/FEVAiZl7foTRlIcSBnhKAIjFyZST8N6Tm1+UjQb+GcgX6kDwzUcVQ= =/yRE -----END PGP SIGNATURE----- --Lhkh0vHaJdFXOMEpFo3pLgebOGDGVfN3R--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e25bdb07-8a72-f955-ffb2-5d2609b9d3fe>