From owner-freebsd-questions@FreeBSD.ORG Fri Jan 25 06:04:20 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2178E16A417 for ; Fri, 25 Jan 2008 06:04:20 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.182]) by mx1.freebsd.org (Postfix) with ESMTP id C418F13C4EA for ; Fri, 25 Jan 2008 06:04:19 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by py-out-1112.google.com with SMTP id u52so847797pyb.10 for ; Thu, 24 Jan 2008 22:04:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding; bh=v26y5xeshZWZDIhlQGJwMwEwfmZ1FvCcEzIg8WpK6ps=; b=OYRJpABs0sFPXBrz93C35LU4QWf4xeDhlDo/cLiGlQ8dc3yYdP4HNBcY/3iX5PQteM9QRs75oC4M/3yGbeWykZBeLmaxxySXMg/FNdAjaHdKUOubFupHyhUqxeHHx1GqEY7CmYKgUQ/AR8lsZDaO5eQU5q7+oNPUplFlr6P6knA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding; b=ZPSSiPE3tyvfq6YArhX73baEcCQ52OYr7B6wn0BBSz7pppzEXiFs3dSmo1m35/sRP4h3KuakIr5+bnK0h1bq5WOesTxWNWECDMhixyqbq7WTYC6Ey19hZCBtu6ExIEQtm1lURYemEMjtWHmn4Br9AqwyFi/nsAS/8vtvxGU7pyY= Received: by 10.64.193.2 with SMTP id q2mr3475464qbf.51.1201241058569; Thu, 24 Jan 2008 22:04:18 -0800 (PST) Received: from flosoft.no-ip.biz ( [67.85.89.184]) by mx.google.com with ESMTPS id c6sm1447551qbc.11.2008.01.24.22.04.17 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 24 Jan 2008 22:04:18 -0800 (PST) Message-ID: <47997BE0.5020203@gmail.com> Date: Fri, 25 Jan 2008 01:04:16 -0500 From: "Aryeh M. Friedman" User-Agent: Thunderbird 2.0.0.9 (X11/20080121) MIME-Version: 1.0 To: Maxim Khitrov References: <4798F1D0.3090009@gmail.com> <26ddd1750801241232l51792bf3ydc02e17e4be49d6e@mail.gmail.com> <20080124210539.GA50047@owl.midgard.homeip.net> <26ddd1750801241408w7b776c87g14b94cfa1433616a@mail.gmail.com> <47995F19.7050905@gmail.com> <26ddd1750801242035j3b3cea5ek281fe7e00afbf217@mail.gmail.com> In-Reply-To: <26ddd1750801242035j3b3cea5ek281fe7e00afbf217@mail.gmail.com> X-Enigmail-Version: 0.95.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org Subject: Re: speeding up buildworld/kernel X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jan 2008 06:04:20 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maxim Khitrov wrote: > On Jan 24, 2008 11:01 PM, Aryeh M. Friedman > wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> >> >> Maxim Khitrov wrote: >>> On Jan 24, 2008 4:05 PM, Erik Trulsson >>> wrote: >>>> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov >>>> wrote: >>>>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman >>>>> wrote: >>>>>> I update my sources at least once a day and do >>>>>> buildworld/kernel just as often... It seems some stuff >>>>>> that needs not be recompiled is on every single run for >>>>>> example gcc and kerbos. I have NO_CLEAN in >>>>>> /etc/make.conf is there anything else I can do to speed >>>>>> stuff up... for ref here is my /etc/make.conf: >>>>>> >>>>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # >>>>>> added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 >>>>>> PERL_VERSION=5.8.8 >>>>>> >>>>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool >>>>>> Developers Developer, not business, friendly >>>>>> http://www.flosoft-systems.com >>>>> I might be wrong, but NO_CLEAN seems like a bad idea except >>>>> in special circumstances. Install ccache, but make sure you >>>>> set CCACHE_HASH_COMPILER environment variable to 1. That >>>>> will make sure that the cache stays valid if the compiler >>>>> executable is overwritten by an identical copy (as it would >>>>> be on installworld). When the compiler changes the cache >>>>> will be repopulated on the next rebuild. >>>> You are indeed wrong. NO_CLEAN will work fine almost all the >>>> time - except in special circumstances. The few times it >>>> does not work one can always do a 'make clean' by hand first. >>>> (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX >>>> to some useful value you can do the same thing for the ports >>>> tree. Personally I always compile with -DNO_CLEAN and use 'rm >>>> -fr' to clean. I have never had problems originating with >>>> this. >>>> >>>> ccache is not very useful for buildworld, since among the >>>> first thing buildworld does is to build the compiler and then >>>> use the newly built compiler to compile the rest. I.e. the >>>> already installed compiler (which is the one ccache will >>>> handle) will not be used for most of the build thus removing >>>> almost all the advantage of ccache. It is supposed to be >>>> possible to use ccache for buildworld as well, but that would >>>> require a bit of hackery. >>> That's not true. I just ran `make buildworld buildkernel` on my >>> firewall. Here are ccache stats when the operation finished: >>> >>> root@cerberus [/root]# ccache -s cache directory /srv/.ccache >>> cache hit 12056 cache miss 38 called >>> for link 461 multiple source files 1 not a >>> C/C++ file 1228 unsupported compiler option >>> 7 files in cache 117366 cache size >>> 679.6 Mbytes max cache size 2.0 Gbytes >>> >>> Ccache is used through the entire build process and there is no >>> hackery involved. Just follow the directions for changing the >>> compiler to /usr/local/libexec/ccache/world-cc. On this Celeron >>> D 1.8 GHz machine rebuilding world and kernel takes 45 minutes >>> and 40 seconds. I don't recall exactly what it was without >>> ccache, but I think it was around 3 hours. Just make sure that >>> you set the CCACHE_HASH_COMPILER variable, otherwise it will >>> assume that the compiler is different just because its >>> modification time has changed. >>> >>> - Max >>> >> No commands where issued on the terminal except the ones showed >> and ccache was installed with default settings (I didn't modify >> any files after doing "make install" for ccache) > > It doesn't work like that. You have to read > /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and configure > things properly before ccache is used for building the os. > > In /etc/make.conf you need to add the following: > > .if exists(/usr/local/libexec/ccache) && !defined(NOCCACHE) && \ > (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*)) CC= > /usr/local/libexec/ccache/world-cc CXX= > /usr/local/libexec/ccache/world-c++ .endif > > Unless you actually want to use /root/.ccache (and have enough > space for it), I would move that directory to some other partition. > In my /etc/csh.cshrc I have this configuration: > > setenv CCACHE_DIR /srv/.ccache setenv CCACHE_PATH > /usr/bin:/usr/local/bin setenv CCACHE_HASH_COMPILER 1 > > Once you've done all of this, rebuild the world. You can run ccache > -s during that process. If the hit/miss numbers stay at 0 then > ccache is not being used. Check your configuration and try again. > > The first run will be slower than normal (though not by much), > because the cache is being populated for the first time. On the > second run, however, clear ccache stats (ccache -z) and you should > see the same results as I've posted above. With the exception of > only a few files, just about everything should be obtained from the > cache and not compiled from scratch. > > - Max > Here are the results: First run (wo/ removing /usr/obj) using the above settings: 119.025u 46.448s 8:11.19 33.6% 487+2711k 15027+6126io 28785pf+0w flosoft# ccache -s cache directory /usr/.ccache cache hit 1 cache miss 3 called for link 3 unsupported compiler option 1 files in cache 6 cache size 188 Kbytes max cache size 976.6 Mbytes Second run: I miscopied and pasted before I relized it was wrong and had already started the next run but from memory: 6:?? minutes (< 6:30) 5 cache hits I cleaned out /usr/obj /usr/.ccache (where I store the cache) for the next run: it crashed: ===> lib/csu/i386-elf (obj,depend,all,install) rm -f .depend CC='/usr/local/libexec/ccache/world-cc' mkdep -f .depend -a - -I/usr/src/lib/csu/i386-elf/../common - -I/usr/src/lib/csu/i386-elf/../../libc/include /usr/src/lib/csu/i386-elf/crt1.c /usr/src/lib/csu/i386-elf/crti.S /usr/src/lib/csu/i386-elf/crtn.S /usr/local/libexec/ccache/world-cc -O2 -fno-strict-aliasing -pipe - -I/usr/src/lib/csu/i386-elf/../common - -I/usr/src/lib/csu/i386-elf/../../libc/include -Wsystem-headers - -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter - -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type - -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align - -Wunused-parameter -Wchar-subscripts -Winline -Wnested-externs - -Wredundant-decls -Wno-pointer-sign -c /usr/src/lib/csu/i386-elf/crt1.c {standard input}: Assembler messages: {standard input}:27: Error: suffix or operands invalid for `mov' *** Error code 1 Stop in /usr/src/lib/csu/i386-elf. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. 1713.072u 279.765s 58:14.38 57.0% 6179+8423k 22605+12323io 8667pf+0w - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com "Free software != Free beer" Blog: http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHmXvgQi2hk2LEXBARAohlAJ4kFVb/o7pHN2Q551n4iDMFhpprGwCbBkUe p5hwkE6sGOkpYEP7Taq+tM0= =llvM -----END PGP SIGNATURE-----