From owner-freebsd-stable@FreeBSD.ORG Mon Feb 10 16:07:40 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FA7AF6D; Mon, 10 Feb 2014 16:07:40 +0000 (UTC) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BED121B23; Mon, 10 Feb 2014 16:07:39 +0000 (UTC) Received: from glenbarber.us (c-71-224-221-174.hsd1.nj.comcast.net [71.224.221.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 290E821416; Mon, 10 Feb 2014 16:07:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us 290E821416 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Mon, 10 Feb 2014 11:07:36 -0500 From: Glen Barber To: Tim Daneliuk Subject: Re: And Here I Thought buildworld/makeworld Was IO Bound Message-ID: <20140210160736.GC1629@glenbarber.us> References: <52F84AF8.8050007@tundraware.com> <52F86768.9000109@googlemail.com> <52F8E8C6.2060909@tundraware.com> <20140210152407.GB1629@glenbarber.us> <52F8F426.9000003@tundraware.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qXCixuLMVvZDruUh" Content-Disposition: inline In-Reply-To: <52F8F426.9000003@tundraware.com> X-Operating-System: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Feb 2014 16:07:40 -0000 --qXCixuLMVvZDruUh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 10, 2014 at 09:45:42AM -0600, Tim Daneliuk wrote: > >>Well ... it has to hit the disk sooner or later. But, if the > >>frequency of physical writes is low because of aggregated > >>IO from the cache, I guess that would tend to make the whole > >>business more CPU bound than IO bound. I just found this surprising. > >> > > > >What is the underlying filesystem? > > >=20 > UFS w/Softupdates, no journaling. >=20 >=20 > [...] > >>Yeah, I wonder what other people are seeing for a full buildworld/kerne= l and/or what > >>the master machines at FreeBSD.org do in this regards. > >> > >>Would anyone else care to share with the class? > >> > > > >If you mean "what do the machines do with regard to tuning", the only > >specific tuning is turning off atime. > > >=20 >=20 > Could you comment a bit more about this please? How you do it, > rationale', etc. >=20 Each build happens in its own ZFS dataset, so atime is disabled during dataset creation (zfs create -o atime=3Doff zroot/11-amd64-GENERIC-snap). The tuning(7) manual has more details, but basically each time a file is read, a separate write happens to update the file access time. Turning off atime eliminates these excessive writes. > >If you mean "what do the machines do as far as overall build time", with > >a clean obj/ directory, 35 minutes sounds about right. I do not have > >exact numbers. > > > >The machine that currently produces weekly snapshot images uses '-j10' > >for buildworld and '-j6' for buildkernel for each build. It also runs > >the builds in parallel. Even with three parallel 'make -j10 buildworld' > >(one for head/, stable/10/, and stable/9/), disk IO is minimal. The > >only time the disk IO becomes a bottleneck is when creating the > >distribution files (base.txz, kernel.txz, etc.). >=20 >=20 > What sort of CPU/Mem/Disk is that machine? >=20 This machine is 48-core (4 x 12 cores) Opteron 6174 (2.2GHz), 128GB RAM, with 5 drives in a raidz1 on a PERC H700 controller. Glen --qXCixuLMVvZDruUh Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJS+PlIAAoJELls3eqvi17Qfm0QANJxYsvQLV3GZSceGcfYENMf dIUXehF3QAY34LiJKSX2lGbaXkmWMh9+yS0VaviOR96gaiywQeW8ncp86k/lsl3w ywsZQgcAFs6FeD/OrIgNkNQii4hYrlhE+Dhr+v+1glyO8L0HSqpLOgInJBi9C7lx +3GU38+b6TMRXiHi1NNG8KROjXYFYRsLXatNIBmEodCz5k3wUWc037VUQ4NUn7KQ /fTSr+td231VK7tlmdiC2c+gtWQQB4JUbRGmFSru34+Dl+ZWBa0w7Topc648LyRP w66XOcODhkmogyFnJdPkYSkctzZjou1KfGng65+0bE0cP4jnNu89fBHJUpvtoU98 8DeEDWEoerYaL1/gfN51Suo+b3EpL0ao8cf7PK+ioWC1tMZF8HTU8Be4K7wEo5wD 5qFFIs2S9o7Lpr2/ZhdFzmhxAY8AvoiuKMnaVSqnKsZD+VZbFpgw+jx8rOPw7qXQ FQ1c4epiooo2LaDbRVvyKR9mtBbWzramHzvR4t28cbST4a7Q8NnIFnwhKmOOPfzv /tUCYrYDE+CeSUIrGkRA2DL32DxHJP4w3UrkMbtaZRPsyTOTrYxE5+z4RmebAhJW ThagFoWT/t+g19HUrJs9HTjbsl3K4m9xh/d23IQ9heuejKGrNnmTVcPVkHljOHqt z86RnWlEzqoQIEEmDvJp =Wm6a -----END PGP SIGNATURE----- --qXCixuLMVvZDruUh--