From owner-freebsd-smp@FreeBSD.ORG Fri May 6 22:32:26 2005 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 88BCB16A4D4; Fri, 6 May 2005 22:32:26 +0000 (GMT) Received: from obsecurity.dyndns.org (CPE0050040655c8-CM00111ae02aac.cpe.net.cable.rogers.com [69.194.102.111]) by mx1.FreeBSD.org (Postfix) with ESMTP id E6F6B43D6E; Fri, 6 May 2005 22:32:25 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id C840E514C9; Fri, 6 May 2005 15:32:24 -0700 (PDT) Date: Fri, 6 May 2005 15:32:24 -0700 From: Kris Kennaway To: Kris Kennaway Message-ID: <20050506223224.GA22379@xor.obsecurity.org> References: <20050506183529.GA46411@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="r5Pyd7+fXNt84Ff3" Content-Disposition: inline In-Reply-To: <20050506183529.GA46411@xor.obsecurity.org> User-Agent: Mutt/1.4.2.1i cc: smp@FreeBSD.org cc: current@FreeBSD.org Subject: Re: Benchmarking mpsafevfs with parallel tarball extraction (ULE vs 4BSD) X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 May 2005 22:32:26 -0000 --r5Pyd7+fXNt84Ff3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, May 06, 2005 at 11:35:29AM -0700, Kris Kennaway wrote: > Here are my benchmark numbers for parallel tarball extraction > with/without mpsafevfs on a 12-processor E4500 running up-to-date 6.0. > Kernel was built without INVARIANTS and other debugging options, > without ADAPTIVE_GIANT (which causes about a 200% performance penalty > on system time in my testing, and has marginal impact on real or user > time) and with 4BSD scheduler (ULE causes spontaneous reboots on this > machine). The e4500 uses the esp SCSI controller, which runs > without Giant. I tried with ULE on a quad amd64 machine, which was stable enough to perform the extraction tests. Here is the data for one tarball extraction to md: x real.one.4bsd + real.one.ule +--------------------------------------------------------------------------+ | + | | + | | + | |+ + + + x | |+ + + + x x x x x x x x| | |MA_| |___MA___| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 9 2.43 2.5 2.46 2.4644444 0.022973415 + 11 2.15 2.18 2.16 2.1636364 0.010269106 Difference at 95.0% confidence -0.300808 +/- 0.0161686 -12.2059% +/- 0.656073% (Student's t, pooled s = 0.0171217) ...so ULE is 12% faster at extracting a single tarball to md 12 concurrent extractions: x real.4bsd + real.ule +--------------------------------------------------------------------------+ | x x + + | |x x x x xx x x + + + ++ + + + +| | |____AM___| |______MA_______| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 10 13.56 14.43 14.09 14.02 0.2641969 + 11 15.95 17.34 16.49 16.570909 0.40957184 Difference at 95.0% confidence 2.55091 +/- 0.318571 18.1948% +/- 2.27226% (Student's t, pooled s = 0.348356) ...but 18% slower with 12 concurrent extractions. The effective concurrency under 4BSD is 2.11 (same asymptote as on the 12-processor sparc64, suggesting something universal like VFS locking is the limitation) but under ULE it is only 1.57. With 4 concurrent extractions (= # CPUs) x real.4.4bsd + real.4.ule +--------------------------------------------------------------------------+ | x + | |x xxx x x x * x + + + ++ + + +| | |____________AM____________| |_______A_M_____| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 10 4.65 5.44 5.17 5.157 0.22400645 + 10 5.41 5.87 5.69 5.657 0.1374409 Difference at 95.0% confidence 0.5 +/- 0.174609 9.69556% +/- 3.38587% (Student's t, pooled s = 0.185834) ULE is still slower. This suggests that at the present time (and apart from the known instabilities) ULE may be better for filesystem performance on lightly loaded systems, but it degrades worse than 4BSD under concurrent filesystem load. Kris --r5Pyd7+fXNt84Ff3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFCe/B3Wry0BWjoQKURAt8DAJ9V+Gjl7bJQRb4j6LqINP719SuZywCgx4ml EKziuNUiB2TLSU3PMmzaid8= =TZ8C -----END PGP SIGNATURE----- --r5Pyd7+fXNt84Ff3--