From owner-freebsd-stable@FreeBSD.ORG Wed May 2 19:29:11 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8784616A410 for ; Wed, 2 May 2007 19:29:11 +0000 (UTC) (envelope-from marsgmiro@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.229]) by mx1.freebsd.org (Postfix) with ESMTP id 2A42B13C4D5 for ; Wed, 2 May 2007 19:29:05 +0000 (UTC) (envelope-from marsgmiro@gmail.com) Received: by nz-out-0506.google.com with SMTP id s1so265439nze for ; Wed, 02 May 2007 12:29:05 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Mj23v38viX1Pj6PllyllEW1yUpEVf5UlMtV1TP7HQYGO9cumd8njaDBQ9adScDXZaxKmqztaRp7J5cCD6xwOsYTpKZc7ZcUy3g720StgEALi3ovwsnZ2xruzprTdMZS+T1TAQ24ovoL2THldGGDMzkt8obMo4ac1UzWg5onXyhs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=RAbycKIj2jQOsY5c9ntRZt3b1cn8oISoMc7UIyJPWCI+X5vTdHFXVu5Yo3h3asKUjrFU0WWnBFAbMAaB80Jum8sd1Ix0xpQPtUpeF8fzORapwvHEKaKyhqOJu7MPze14vsOEEl12EuFi6fg6rfdpcr6X8Js15+GmBdGfP51Qgm8= Received: by 10.114.202.15 with SMTP id z15mr367192waf.1178134145285; Wed, 02 May 2007 12:29:05 -0700 (PDT) Received: by 10.114.59.16 with HTTP; Wed, 2 May 2007 12:29:05 -0700 (PDT) Message-ID: <28edec3c0705021229w50380f5cqf8608c694bbbf267@mail.gmail.com> Date: Thu, 3 May 2007 03:29:05 +0800 From: "Mars G. Miro" To: "Kris Kennaway" In-Reply-To: <20070502190732.GA42282@xor.obsecurity.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <28edec3c0705021139h45c785c2s915941c0afb93b5d@mail.gmail.com> <20070502190732.GA42282@xor.obsecurity.org> Cc: freebsd-stable@freebsd.org Subject: Re: mfs and buildworlds on da SunFire x4600 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 May 2007 19:29:11 -0000 On 5/3/07, Kris Kennaway wrote: > On Thu, May 03, 2007 at 02:39:55AM +0800, Mars G. Miro wrote: > > Greetz, > > > > I got my hands on a SunFire x4600. This is a dual-core 8-way > > Opteron (dmesg shows 16 CPUs) w/c has 32G RAM. FreeBSD 6.2/amd64 runs > > fine on it despite: > > - 'reboot' doesnt work, machine just hangs in there. > > There is a sysctl to specify an alternate reboot method. > > > - front USB ports wont work for a USB keyboard, just use da ports at da > > back. > > > > I've read a lot of threads in da past (and da BSDCan 2006 paper) > > regarding mfs on /usr/src and /usr/obj to speed up buildworlds, > > unfortunately it doesnt seem to improve things. I've tried both a > > malloc and swap based async non-softupdates mfs (and rebooted da box > > every after) but my buildworld speeds seem to be just da same as I do > > it on da MPT SAS disk w/c is just around ~18mins. > > It is probably a combination of issues: > > 1) There is still some disk access because of accessing things like > binaries in /usr. This will hurt more for parallel builds because > your disk hardware is single-threaded, so I/O delays will serialize > you. If you really wanted you could compare to a build in a md-hosted > chroot to completely eliminate the disk, but this is just a game. > > 2) Buildworld doesn't parallelize well to 16 CPUs. Large parts of it > are completely non-parallel. This is probably your main problem. > I guess that might explain it, see more below... > You definitely should be measuring some difference - you need to use > time(1) though instead of just estimating. > I did use time, here: make -j32 buildworld: disk 3229.975u 8981.269s 18:55.21 1075.6% 3804+2164k 19702+6594io 2158pf+0w malloc mfs ( mdmfs -SM -o async,noatime -s 2G md {src, obj} ) 3069.783u 9546.250s 18:57.58 1109.0% 3844+2147k 18504+4422io 2628pf+0w swap mfs ( mdmfs -S -o async,noatime -s 2G md {src, obj} ) 3061.682u 9682.776s 19:08.90 1109.2% 3849+2164k 13732+4402io 2053pf+0w make -j16 buildworld: disk 3218.722u 7855.885s 17:42.26 1042.5% 3833+2267k 16995+6620io 2445pf+0w malloc mfs ( mdmfs -SM -o async,noatime -s 2G md {src, obj} ) 3187.753u 8199.049s 17:57.63 1056.6% 3839+2238k 6890+4436io 2346pf+0w swap mfs ( mdmfs -S -o async,noatime -s 2G md {src, obj} ) 3200.400u 8213.312s 18:06.88 1050.1% 3860+2243k 12800+4397io 2303pf+0w > Kris > Thanks cheers mars