From owner-freebsd-current@FreeBSD.ORG Sun Feb 25 05:24:44 2007 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7DECF16A403 for ; Sun, 25 Feb 2007 05:24:44 +0000 (UTC) (envelope-from zombyfork@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.227]) by mx1.freebsd.org (Postfix) with ESMTP id 27E7813C441 for ; Sun, 25 Feb 2007 05:24:44 +0000 (UTC) (envelope-from zombyfork@gmail.com) Received: by nz-out-0506.google.com with SMTP id i11so1007848nzh for ; Sat, 24 Feb 2007 21:24:40 -0800 (PST) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:references; b=DaVU/KUdpJ1ruTpna5dR/Ee5hP/Y5eS3NiLfzJdAOHpRoLSIxwQ4oZNtokra5w9mlsCj+ocEZITKKf7LIAdAHxyAd4tbMEZzCQNVPCW07ZjRubNVjihTTxZ+ya2Nag3k8CEXGy3+OLhL8c9fksJsVXpD+FJenw07BRkzJCyYof4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:references; b=m4uur+LyKmPnoci8IFDP8pO5oku3FySw0atllgA6jZMltPvoyk+1xj5T9cKzzkLP2LukK2yTzjDJoXMUnhspL1UdoE2MEribtzIo6Xkb2nImitVDIX3anQ1K3iQP+Wngm89/Jk+CptBzD6B6rkVMNUYtnVdcY14JXfNjNnoG8UQ= Received: by 10.114.111.1 with SMTP id j1mr1666790wac.1172379635166; Sat, 24 Feb 2007 21:00:35 -0800 (PST) Received: by 10.115.108.15 with HTTP; Sat, 24 Feb 2007 21:00:35 -0800 (PST) Message-ID: <346a80220702242100i7ec22b5h4b25cc7d20d03e98@mail.gmail.com> Date: Sat, 24 Feb 2007 22:00:35 -0700 From: "Coleman Kane" To: "Kris Kennaway" In-Reply-To: <20070224213111.GB41434@xor.obsecurity.org> MIME-Version: 1.0 References: <20070224213111.GB41434@xor.obsecurity.org> X-Mailman-Approved-At: Sun, 25 Feb 2007 12:29:28 +0000 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: smp@freebsd.org, hackers@freebsd.org, current@freebsd.org Subject: Re: Progress on scaling of FreeBSD on 8 CPU systems X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: cokane@cokane.org List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2007 05:24:44 -0000 On 2/24/07, Kris Kennaway wrote: > > Now that the goals of the SMPng project are complete, for the past > year or more several of us have been working hard on profiling FreeBSD > in various multiprocessor workloads, and looking for performance > bottlenecks to be optimized. > > We have recently made significant progress on optimizing for MySQL > running on an 8-core amd64 system. The graph of results may be found > here: > > http://www.freebsd.org/~kris/scaling/scaling.png > > This shows the graph of MySQL transactions/second performed by a > multi-threaded client workload against a local MySQL database with > varying numbers of client threads, with identically configured FreeBSD > and Linux systems on the same machine. > > The test was run on FreeBSD 7.0, with the latest version of the ULE > 2.0 scheduler, the libthr threading library, and an uncommitted patch > from Jeff Roberson [1] that addresses poor scalability of file > descriptor locking (using a new sleepable mutex primitive); this patch > is responsible for almost all of the performance and scaling > improvements measured. It also includes some other patches (collected > in my kris-contention p4 branch) that have been shown to help > contention in MySQL workloads in the past (including a UNIX domain > socket locking pushdown patch from Robert Watson), but these were > shown to only give small individual contributions, with a cumulative > effect on the order of 5-10%. > > With this configuration we are able to achieve performance that is > consistent with Linux at peak (the graph shows Linux 2% faster, but > this is commensurate with the margin of error coming from variance > between runs, so more data is needed to distinguish them), with 8 > client threads (=1 thread/CPU core), and significantly outperforms > Linux at higher than peak loads, when running on the same hardware. > > Specifically, beyond 8 client threads FreeBSD has only minor > performance degradation (an 8% drop from peak throughput at 8 clients > to 20 clients), but Linux collapses immediately above 8 threads, and > above 14 threads asymptotes to essentially single-threaded levels. At > 20 clients FreeBSD outperforms Linux by a factor of 4. > > We see this result as part of the payoff we are seeing from the hard > work of many developers over the past 7 years. In particular it is a > significant validation of the SMP and locking strategies chosen for > the FreeBSD kernel in the post-FreeBSD 4.x world. > > More configuration details and discussion about the benchmark may be > found here: > > http://people.freebsd.org/~kris/scaling/mysql.html > > Kris > > What does the performance curve look like for the in-CVS 7-CURRENT tree with 4BSD or ULE ? How do those stand up against the Linux SMP scheduler for scalability. It would be nice to see the comparison displayed to see what the performance improvements of the aforementioned patch were realized to. This would likely be a nice graphics for the SMPng project page, BTW... -- Coleman