From owner-freebsd-current@FreeBSD.ORG Mon Jul 14 12:41:16 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D630A37B401; Mon, 14 Jul 2003 12:41:16 -0700 (PDT) Received: from kientzle.com (h-66-166-149-50.SNVACAID.covad.net [66.166.149.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 244FA43F75; Mon, 14 Jul 2003 12:41:16 -0700 (PDT) (envelope-from kientzle@acm.org) Received: from acm.org (ugly.x.kientzle.com [66.166.149.53]) by kientzle.com (8.12.9/8.12.9) with ESMTP id h6EJfFgY028922; Mon, 14 Jul 2003 12:41:15 -0700 (PDT) (envelope-from kientzle@acm.org) Message-ID: <3F130805.3060900@acm.org> Date: Mon, 14 Jul 2003 12:44:05 -0700 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.0.1) Gecko/20021005 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Gordon Tetlow References: <20030713214715.H15811@root.org> <20030714004042.A38507@hub.freebsd.org> <20030714160952.GD12996@roark.gnf.org> Content-Type: multipart/mixed; boundary="------------020706010603060304030809" cc: current@freebsd.org cc: Nate Lawson Subject: Re: Overdone rescue cleaning as part of buildworld? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: kientzle@acm.org List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jul 2003 19:41:17 -0000 This is a multi-part message in MIME format. --------------020706010603060304030809 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Gordon Tetlow wrote: > On Mon, Jul 14, 2003 at 12:40:42AM -0700, David O'Brien wrote: >>On Sun, Jul 13, 2003 at 09:49:46PM -0700, Nate Lawson wrote: >>>It appears /rescue is cleaning for way too much as part of buildworld. >>>For instance, groff is NOT part of /rescue (or we have other things to >>>discuss. :) This adds a bit of time to buildworld, can it be removed? Yeah, I took a few shortcuts; /rescue does build far more in OBJDIR than it needs to, and similarly cleans much more than it needs to. (Those extra dirs are never populated, but building and cleaning them does still take time.) I believe the attached patch addresses this issue; it trims down /usr/obj/usr/src/rescue/rescue/usr/src/... to just the directories actually needed. David's claim that /rescue is more than doubling the build time is surprising, though. Compiling all of /bin and /sbin (which is more-or-less what /rescue consists of) should not take more time than building the entire system including /bin and /sbin. Perhaps something else is going on here? > I've already started this process and I have some work in a local tree > to depessimize the build dramatically. Thank you for the reminder. Would > you be interested in taking a look at the patches? Gordon, I apologize that my shortcuts are causing you more work. If you've already solved these problems, feel free to ignore this and commit your work. On the other hand, since I _do_ understand how /rescue is built, I thought I might be able to save you some effort by feeding you fixes for this. I'm waiting on a buildworld with this patch to finish. I'll let you know if anything goes awry, but I believe it works. (Unfortunately, I _don't_ understand the parallel build issue. I strongly suspect that it only impacts dhclient, which has some rather unique build architecture. The draconian solution would be to carve dhclient out of the rescue crunchgen and build it separately, or just statically link the official copy in /sbin and put off the whole issue. I find both of these approaches expedient but unsatisfying.) Tim --------------020706010603060304030809 Content-Type: text/plain; name="kientzle_rescue_Makefile.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="kientzle_rescue_Makefile.diff" Index: Makefile =================================================================== RCS file: /usr/cvs/FreeBSD-CVS/src/rescue/rescue/Makefile,v retrieving revision 1.5 diff -u -r1.5 Makefile --- Makefile 30 Jun 2003 21:13:56 -0000 1.5 +++ Makefile 14 Jul 2003 19:25:02 -0000 @@ -64,7 +64,7 @@ # WARNING: Changing this list may require adjusting # /usr/include/paths.h as well! You were warned! # -CRUNCH_SRCDIRS+=$(.CURDIR)/../../bin $(.CURDIR)/../../usr.bin +CRUNCH_SRCDIRS+=$(.CURDIR)/../../bin CRUNCH_PROGS=cat chflags chio chmod cp date dd df domainname echo ed \ expr getfacl hostname kenv kill ln ls mkdir mv pax ps pwd \ realpath rm rmdir setfacl sh sleep stty sync test @@ -99,9 +99,6 @@ # WARNING: Changing this list may require adjusting # /usr/include/paths.h as well! You were warned! # -# Note that mdmfs and shutdown have their own private 'pathnames.h' -# headers in addition to the standard 'paths.h' header. -# CRUNCH_SRCDIRS+=$(.CURDIR)/../../sbin CRUNCH_PROGS+=atm adjkerntz atacontrol badsect bsdlabel camcontrol \ ccdconfig clri comcontrol conscontrol devfs dmesg dump \ @@ -159,28 +156,32 @@ CRUNCH_ALIAS_fsck_ffs=fsck_4.2bsd fsck_ufs CRUNCH_ALIAS_mount_std= mount_devfs mount_fdescfs mount_linprocfs mount_procfs -# dhclient has historically been troublesome... +# dhclient is troublesome... CRUNCH_PROGS+=dhclient CRUNCH_BUILDOPTS_dhclient=-DRELEASE_CRUNCH -Dlint ################################################################## # Programs from stock /usr/bin # -CRUNCH_SRCDIRS+=$(.CURDIR)/../../usr.bin -CRUNCH_SRCDIRS+=$(.CURDIR)/../../gnu/usr.bin CRUNCH_PROGS+=wall +CRUNCH_SRCDIR_wall+=$(.CURDIR)/../../usr.bin/wall CRUNCH_PROGS+=gzip +CRUNCH_SRCDIR_gzip+=$(.CURDIR)/../../gnu/usr.bin/gzip CRUNCH_ALIAS_gzip=gunzip gzcat zcat CRUNCH_PROGS+=bzip2 +CRUNCH_SRCDIR_bzip2+=$(.CURDIR)/../../usr.bin/bzip2 CRUNCH_ALIAS_bzip2=bunzip2 bzcat CRUNCH_LIBS+=-lbz2 CRUNCH_PROGS+=tar +CRUNCH_SRCDIR_tar+=$(.CURDIR)/../../gnu/usr.bin/tar + CRUNCH_PROGS+=vi CRUNCH_ALIAS_vi=ex +CRUNCH_SRCDIR_vi+=$(.CURDIR)/../../usr.bin/vi ################################################################## # The following is pretty nearly a generic crunchgen-handling makefile @@ -244,8 +245,6 @@ $(OUTPUTS): $(CONF) MAKEOBJDIRPREFIX=${CRUNCHOBJS} crunchgen -q -m $(OUTMK) -c $(OUTC) $(CONF) -# -m here forces make to treat the bsd.prog.mk and bsd.lib.mk in -# this directory as overrides for the standard shared ones. $(PROG): $(OUTPUTS) MAKEOBJDIRPREFIX=${CRUNCHOBJS} make -f $(OUTMK) @@ -260,14 +259,34 @@ .for D in $(CRUNCH_SRCDIRS) cd ${D} && MAKEOBJDIRPREFIX=${CANONICALOBJDIR} make ${.TARGET} .endfor +.for P in $(CRUNCH_PROGS) +.ifdef CRUNCH_SRCDIR_${P} + cd ${CRUNCH_SRCDIR_${P}} && \ + MAKEOBJDIRPREFIX=${CANONICALOBJDIR} make ${.TARGET} +.endif +.endfor clean: rm -f ${CLEANFILES} if [ -e ${.OBJDIR}/$(OUTMK) ]; then \ MAKEOBJDIRPREFIX=${CRUNCHOBJS} make -f $(OUTMK) clean; \ fi +.for P in $(CRUNCH_PROGS) +.ifdef CRUNCH_SRCDIR_${P} + cd ${CRUNCH_SRCDIR_${P}} && \ + MAKEOBJDIRPREFIX=${CRUNCHOBJS} make clean +.endif +.endfor .for D in $(CRUNCH_SRCDIRS) $(EXTRA_SRCDIRS) cd ${D} && MAKEOBJDIRPREFIX=${CRUNCHOBJS} make clean +.endfor + +# Someone should replace the bin/csh and bin/sh build-tools with +# shell scripts so we can remove this nonsense. +build-tools: +.for _tool in bin/csh bin/sh + cd ${.CURDIR}/../../${_tool}; \ + MAKEOBJDIRPREFIX=${CRUNCHOBJS} make DIRPRFX=rescue/${_tool} build-tools .endfor .include --------------020706010603060304030809--