From owner-freebsd-current@FreeBSD.ORG Tue Nov 15 18:37:06 2005 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D2A8816A41F for ; Tue, 15 Nov 2005 18:37:06 +0000 (GMT) (envelope-from schweikh@schweikhardt.net) Received: from bremen.shuttle.de (bremen.shuttle.de [194.95.249.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D3C443D45 for ; Tue, 15 Nov 2005 18:37:05 +0000 (GMT) (envelope-from schweikh@schweikhardt.net) Received: by bremen.shuttle.de (Postfix, from userid 10) id 2A3DC3B9BF; Tue, 15 Nov 2005 19:37:04 +0100 (CET) Received: from hal9000.schweikhardt.net (localhost [127.0.0.1]) by hal9000.schweikhardt.net (8.13.4/8.13.4) with ESMTP id jAFIalKA099337; Tue, 15 Nov 2005 19:36:47 +0100 (CET) (envelope-from schweikh@hal9000.schweikhardt.net) Received: (from schweikh@localhost) by hal9000.schweikhardt.net (8.13.4/8.13.4/Submit) id jAFIal3s099336; Tue, 15 Nov 2005 19:36:47 +0100 (CET) (envelope-from schweikh) Date: Tue, 15 Nov 2005 19:36:47 +0100 From: Jens Schweikhardt To: Nicolas Blais Message-ID: <20051115183647.GA1783@schweikhardt.net> References: <200511141702.45491.nb_root@videotron.ca> <200511150542.34910.nb_root@videotron.ca> <4379BCEB.6030806@acidy.com> <200511150611.31743.nb_root@videotron.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200511150611.31743.nb_root@videotron.ca> User-Agent: Mutt/1.5.11 Cc: Steve Hodgson , freebsd-current@freebsd.org Subject: Using ccache for build{world, kernel} (was: can't compile -CURRENT since 2 days ago) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Nov 2005 18:37:07 -0000 On Tue, Nov 15, 2005 at 06:11:20AM -0500, Nicolas Blais wrote: # > Well, if ccache was clever enough to support kernel builds then it would # > be ok. But since I've never managed to get it to work reliably I use: Okay, getting the full effect of ccache for a buildworld and buildkernel requires solving two problems: 1) 'make buildworld' switches from the compilers found in /usr/bin to the ones it built below /usr/obj/usr/src/tmp/usr/bin at some point in time (simply by prepending this directory to PATH). 'make buildkernel' uses the new compiler right from the start. Make ccache somehow do the same. 2) Have ccache ignore (i.e. not hash) the modification time of the compiler because this changes with every new 'make buildworld'. Otherwise you invalidate your cache with each buildworld. #1 is done with make CC="/usr/local/bin/ccache cc" CXX="/usr/local/bin/ccache c++" buildworld make CC="/usr/local/bin/ccache cc" CXX="/usr/local/bin/ccache c++" buildkernel and NOT using any CCACHE_PATH, so ccache scans PATH for the real compiler. This automatically uses /usr/bin/{cc,c++} in the early buildworld stages, and the new compilers the moment PATH is prepended with the correct directories by 'make buildworld'. #2 is done with this patch (a week ago I sent it to the ccache developer mailing list bugs@ccache.samba.org in order to hopefully get it committed to the release version; so far no reply... maybe you guys join me in bugging them about it :-) Put this in /usr/ports/devel/ccache/files/patch-nohash-size-mtime and reinstall the port. Then export CCACHE_NOHASH_SIZE_MTIME=y during your makes for buildworld and buildkernel. This melts my buildworld time from 2700s (real@~100%CPU) to under 1000s and buildkernel from 800s to 250s, so you gain about a factor 3 for ab-initio builds. Sweet! --- ccache.c.orig Mon Oct 31 17:38:21 2005 +++ ccache.c Mon Oct 31 17:52:31 2005 @@ -331,8 +331,10 @@ hash_string(str_basename(args->argv[0])); } - hash_int(st.st_size); - hash_int(st.st_mtime); + if (!getenv("CCACHE_NOHASH_SIZE_MTIME")) { + hash_int(st.st_size); + hash_int(st.st_mtime); + } /* possibly hash the current working directory */ if (getenv("CCACHE_HASHDIR")) { --- ccache.1.orig Mon Oct 31 18:08:59 2005 +++ ccache.1 Mon Oct 31 18:09:02 2005 @@ -234,6 +234,16 @@ incorrect setting of this debug info rarely causes problems\&. If you strike problems with gdb not using the correct directory then enable this option\&. +.IP +.IP "\fBCCACHE_NOHASH_SIZE_MTIME\fP" +This tells ccache to not hash the real compiler's size and modification +time. Normally this is the mechanism to detect compiler upgrades. +There are situations however, where even though the compiler's size or +modification time has changed you can safely use the cached objects +(e.g. if as part of your build system the compiler is built as well +and the compiler's source has not changed; or if the compiler has only +changes that do not affect code generation). Use this feature only if +you know what you are doing. .IP .IP "\fBCCACHE_UNIFY\fP" If you set the environment variable CCACHE_UNIFY Regards, Jens -- Jens Schweikhardt http://www.schweikhardt.net/ SIGSIG -- signature too long (core dumped)