Date: Sat, 22 May 2021 16:51:31 -0700 From: Mark Millard via freebsd-arm <freebsd-arm@freebsd.org> To: tech-lists <tech-lists@zyxst.net> Cc: freebsd-arm@freebsd.org Subject: Re: RPi 4 build time Message-ID: <9A949E36-FDF2-40B3-A126-5538E41964D3@yahoo.com> In-Reply-To: <DB0011CC-3F99-4E34-B5DF-307AD707851A@yahoo.com> References: <YKgTB7Hf3dkQiW5c@vax.khramtsov.org> <YKgzeLxxZNhViwoi@ceres.zyxst.net> <0299DFBF-5497-4A06-978D-13E4FBD8B5F0@yahoo.com> <YKkqxrthZoHg87wV@ceres.zyxst.net> <DB0011CC-3F99-4E34-B5DF-307AD707851A@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-May-22, at 13:12, Mark Millard <marklmi at yahoo.com> wrote: > On 2021-May-22, at 09:01, tech-lists <tech-lists at zyxst.net> wrote: >=20 >> On Fri, May 21, 2021 at 03:51:35PM -0700, Mark Millard via = freebsd-arm wrote: >>=20 >>> So, if I read this right, you are reporting 4.5 hrs >>> for a "hot ccache" result, which I had mentioned as >>> one of the things leading to large variations in >>> reported build times. >>=20 >> Hi, >>=20 >> not sure what you mean by "hot cache" >=20 > The first time ccache is used it has no prior results > to use to avoid compiles/links: an empty cache (a form > of "cold" cache). Another form of "cold" cache could > result from changing compiler options that would change > the code generated for (nearly) every file produced so > that the cache becomes ineffective. >=20 > "hot" refers to having a significant amount of > "effective/used cache content" that makes a notable > difference in the build times. I'm not that impressed > with the terminology but it is was I've seen used the > most frequently for ccache. So I used it. >=20 >> - I always use devel/ccache-static >> as have tended to build from source throughout my time of using = freebsd. >> It provides tremendous speedups and generally i'll disable it only if = a >> problem arises and am debugging it, or crossing a version boundary = like >> from stable to current. What I'm saying is I don't know when ccache = was >> last used for building anything. >=20 > I'm confused how you can know it "provides tremendous > speedups" while simultaneously not knowing "when ccache > was last used for building anything". It sounds like you > think the 4.5 hr build might have not have been from > having a notable speed up from ccache? >=20 > Remember that when comparing to my "from scratch" > build times: in my build everything was compiled > and linked, no prior build materials around to be > reused. So I'm reporting a context where I know > how to interpret the result and I'm presenting > enough history to establish a repeatable context. >=20 >> 1. rpi4 here is clocked to 2.0GHz >> 2. ccache is in use and /var/cache/ccache has *not* been previously = cleared >> (i'll clear it for next test) >>=20 >> 3. make cleanworld cleandir clean has been run on /usr/src >> 4. sources are at 246839 >>=20 >> 5. this rpi4 has the following properties for its disk: >> [i] root-on-zfs >> [ii] boot-to-usb3 >> [iii] 4k sectorsize forced >> [iv] encrypted swapspace >> [v] entire filesystem encryption >=20 > FYI: My build-experiment boot media are never > encrypted for the file system or swap/paging > space. Another thing I'd not thought to comment > on. As I've reported, my UFS based and ZFS based > experiments get only minor variations in > build times (variations of minutes for from- > scratch builds that take hours). >=20 >> /etc/src.conf is >> https://cloud.zyxst.net/~john/FreeBSD/rpi4-main/src.conf >>=20 >> make -j10 cleanworld started on Sat May 22 15:41:58 BST 2021 >> make -j10 cleanworld completed on Sat May 22 15:43:23 BST 2021 >>=20 >> make -j10 cleandir started on Sat May 22 15:43:23 BST 2021 >> make -j10 cleandir completed on Sat May 22 15:43:50 BST 2021 >>=20 >> make -j10 clean started on Sat May 22 15:43:50 BST 2021 >> make -j10 clean completed on Sat May 22 15:44:11 BST 2021 >>=20 >> make -j6 buildworld started on Sat May 22 15:44:11 BST 2021 >> make -j6 buildworld completed on Sat May 22 16:20:48 BST 2021 >=20 > So between 36 min and 37 min to rebuild the same version > with the same build options and compiler/link command lines > (near[?] maximal effective-ccache content that leads to > near[?] maximal avoidance of rebuild activity). >=20 > Cool. >=20 > For META_MODE builds, seeing how long it takes to go through > and discover that little or nothing needs to be rebuild would > be the build times for 2nd build from doing back-to-back builds > (not even an install to the live system between). The META_MODE > use would then prevent most rebuild activity. I've not done > such a timing in a long time and it does not approximate any > normal build time for my typical rebuild patterns. So I do not > normally time that. >=20 > I'm not claiming META_MODE is similarly effective to ccache. > In fact, I know of issues where META_MODE rebuilds files that > ccache would avoid rebuilding the same file: for example, > doing an install of a build to the live system between the > rebuilds has side effects that lead META_MODE to rebuild far > more things. >=20 >> make -j6 buildkernel started on Sat May 22 16:20:48 BST 2021 >> make -j6 buildkernel completed on Sat May 22 16:49:18 BST 2021 >=20 > So between 28 min and 29 min to rebuild the same version with > the same build options and compiler/link command lines > (near[?] maximal effective-ccache content). >=20 > Total between 64 min and 66 min overall for buildworld buildkernel > for the near[?] maximal effective-ccache content and needing all > the files. >=20 > Good to know. Thanks. >=20 I happen to have ended up with an opportunity to do (no cleanout of old results after the first rebuild, no installationa of any of the builds): rebuild world reboot rebuild world The 2nd rebuild of world got: World built in 354 seconds, ncpu: 4, make -j4 So a little under 6 minutes via META_MODE. META_MODE does end up causing some rebuild activity, just not much. Much of it is re-linking. I did another "rebuild world" without a new reboot and got: World built in 293 seconds, ncpu: 4, make -j4 So, somewhat under 5 minutes for more context cached in RAM. A similar sequence for a debug build instead of non-debug build (building machine running non-debug) got: World built in 526 seconds, ncpu: 4, make -j4 So, somewhat under 9 minutes. Then (no reboot between): World built in 296 seconds, ncpu: 4, make -j4 So, somewhat under 5 minutes again. In general these figures are approximations of the low bound on a buildworld that is a (near) no-op but is not frequently approached in my normal activity. But it is rare for me to update the source tree again and rebuild after only a few source commits after what was originally rebuilt. For such, sub-half hour rebuilds can certainly occur via META_MODE use. The context happened to be the ZFS based one in all cases. Still no ccache use. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9A949E36-FDF2-40B3-A126-5538E41964D3>