Date: Thu, 24 Jan 2008 23:01:29 -0500 From: "Aryeh M. Friedman" <aryeh.friedman@gmail.com> To: Maxim Khitrov <mkhitrov@gmail.com> Cc: freebsd-questions@freebsd.org Subject: Re: speeding up buildworld/kernel Message-ID: <47995F19.7050905@gmail.com> In-Reply-To: <26ddd1750801241408w7b776c87g14b94cfa1433616a@mail.gmail.com> References: <4798F1D0.3090009@gmail.com> <26ddd1750801241232l51792bf3ydc02e17e4be49d6e@mail.gmail.com> <20080124210539.GA50047@owl.midgard.homeip.net> <26ddd1750801241408w7b776c87g14b94cfa1433616a@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maxim Khitrov wrote: > On Jan 24, 2008 4:05 PM, Erik Trulsson <ertr1013@student.uu.se> > wrote: >> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: >>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman >>> <aryeh.friedman@gmail.com> wrote: >>>> I update my sources at least once a day and do >>>> buildworld/kernel just as often... It seems some stuff that >>>> needs not be recompiled is on every single run for example >>>> gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there >>>> anything else I can do to speed stuff up... for ref here is >>>> my /etc/make.conf: >>>> >>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by >>>> use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 >>>> PERL_VERSION=5.8.8 >>>> >>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers >>>> Developer, not business, friendly >>>> http://www.flosoft-systems.com >>> I might be wrong, but NO_CLEAN seems like a bad idea except in >>> special circumstances. Install ccache, but make sure you set >>> CCACHE_HASH_COMPILER environment variable to 1. That will make >>> sure that the cache stays valid if the compiler executable is >>> overwritten by an identical copy (as it would be on >>> installworld). When the compiler changes the cache will be >>> repopulated on the next rebuild. >> You are indeed wrong. NO_CLEAN will work fine almost all the >> time - except in special circumstances. The few times it does >> not work one can always do a 'make clean' by hand first. (Or even >> faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some >> useful value you can do the same thing for the ports tree. >> Personally I always compile with -DNO_CLEAN and use 'rm -fr' to >> clean. I have never had problems originating with this. >> >> ccache is not very useful for buildworld, since among the first >> thing buildworld does is to build the compiler and then use the >> newly built compiler to compile the rest. I.e. the already >> installed compiler (which is the one ccache will handle) will not >> be used for most of the build thus removing almost all the >> advantage of ccache. It is supposed to be possible to use ccache >> for buildworld as well, but that would require a bit of hackery. > > That's not true. I just ran `make buildworld buildkernel` on my > firewall. Here are ccache stats when the operation finished: > > root@cerberus [/root]# ccache -s cache directory > /srv/.ccache cache hit 12056 cache miss > 38 called for link 461 multiple source files > 1 not a C/C++ file 1228 unsupported compiler > option 7 files in cache 117366 cache > size 679.6 Mbytes max cache size > 2.0 Gbytes > > Ccache is used through the entire build process and there is no > hackery involved. Just follow the directions for changing the > compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D > 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40 > seconds. I don't recall exactly what it was without ccache, but I > think it was around 3 hours. Just make sure that you set the > CCACHE_HASH_COMPILER variable, otherwise it will assume that the > compiler is different just because its modification time has > changed. > > - Max > I think Erik is correct here are some times (done in the order listed): After adding NO_PROFILE to make.conf: flosoft# cvs -q update -dP M lib/libc/stdlib/malloc.c flosoft# time make buildworld buildkernel installkernel installworld . . . 129.160u 49.686s 6:48.67 43.7% 1001+2748k 16259+6155io 29699pf+0w After installing ccache (first run): flosoft# setenv CCACHE_HASH_COMPILER 1 flosoft# set CCACHE_HASH_COMPILER=1 flosoft# time make buildworld buildkernel installkernel installworld . . . 117.765u 46.502s 4:56.24 55.4% 474+2667k 674+6151io 8269pf+0w flosoft# ccache -s cache directory /root/.ccache cache hit 0 cache miss 0 files in cache 0 cache size 0 Kbytes max cache size 976.6 Mbytes Second run: flosoft# time make buildworld buildkernel installkernel installworld . . . 118.318u 46.055s 4:46.64 57.3% 475+2644k 251+6145io 6203pf+0w flosoft# !cc ccache -s cache directory /root/.ccache cache hit 0 cache miss 0 files in cache 0 cache size 0 Kbytes max cache size 976.6 Mbytes After clearing out /usr/obj (with ccache turned off): Note: Even though I didn't time without NO_PROFILE this time I have in the past on the same machine and got about 1 hour 5 mins (so not a big savings) flosoft# rm -rf /usr/objflosoft# time make buildworld buildkernel installkernel installworld . . . 2549.561u 387.975s 58:08.69 84.2% 6352+7186k 27134+14972io 11234pf+0w Turning ccache back on (1st run): flosoft# rm -rf /usr/obj flosoft# setenv CCACHE_HASH_COMPILER 1 flosoft# set CCACHE_HASH_COMPILER=1 flosoft# time make buildworld buildkernel installkernel installworld . . . 2556.474u 392.932s 57:21.72 85.6% 6277+7130k 24444+15046io 6414pf+0w flosoft# ccache -s cache directory /root/.ccache cache hit 0 cache miss 0 files in cache 0 cache size 0 Kbytes max cache size 976.6 Mbytes Second run: flosoft# rm -rf /usr/obj flosoft# time make buildworld buildkernel installkernel installworld . . . 2647.683u 406.585s 1:29:41.15 56.7% 6265+7122k 18646+14931io 7350pf+0w flosoft# ccache -s cache directory /root/.ccache cache hit 0 cache miss 0 files in cache 0 cache size 0 Kbytes max cache size 976.6 Mbytes Just for sanity a final run without cleaning out /usr/obj and not having ccache: flosoft# time make buildworld buildkernel installkernel installworld . . . 166.330u 62.414s 17:28.56 21.8% 1830+2908k 5167+6726io 9397pf+0w Full Discloser Section: Hardware: e6850 CPU (dual core 3.0GHz) p35 chipset ihc9 disk controller 500 GB SATA/300 drive 4GB of RAM (DDR2 667) MSI Neo-F Mobo (Default jumper settings) nVidia 5200 FX (PCI, 256 MB) Not over clocked. Base system: FreeBSD flosoft.no-ip.biz 8.0-CURRENT FreeBSD 8.0-CURRENT #4: Thu Jan 24 14:47:29 EST 2008 aryeh@flosoft.no-ip.biz:/usr/obj/usr/src/sys/MONSTER amd64 Also running at various times: xfce4 (full install with dbus and hald running also) Allways on: firefox thunderbird miro deluge (with 1 download active) rythmbox Used during the 1st run of cleaning out /usr/obj and ccache open office 2 No reboots between runs and less then 1 minute pause between runs except between the 1st and second runs after cleaning /usr/obj (i.e. beofre and after ccache) No commands where issued on the terminal except the ones showed and ccache was installed with default settings (I didn't modify any files after doing "make install" for ccache) BTW I think the final sanity runs proves there is somekind of mem leak in xfce or xorg - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com "Free software != Free beer" Blog: http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHmV8ZQi2hk2LEXBARAnyNAJ94onhHiOghmgMwI1IPBAFLSlnLuACeKTgG UE2bfmtMYpfOcSPnqxdbasQ= =KcWD -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47995F19.7050905>