From owner-freebsd-current@FreeBSD.ORG Tue May 9 18:23:36 2006 Return-Path: X-Original-To: current@FreeBSD.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B1B8D16A9DE; Tue, 9 May 2006 18:23:36 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id B39DF43D64; Tue, 9 May 2006 18:23:31 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 9C9071A3C26; Tue, 9 May 2006 11:23:31 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 13BEA515BB; Tue, 9 May 2006 14:23:31 -0400 (EDT) Date: Tue, 9 May 2006 14:23:30 -0400 From: Kris Kennaway To: Greg 'groggy' Lehey Message-ID: <20060509182330.GB92714@xor.obsecurity.org> References: <20060506150622.C17611@fledge.watson.org> <20060506221908.GB51268@xor.obsecurity.org> <20060507210426.GA4422@xor.obsecurity.org> <20060507214153.GA5275@xor.obsecurity.org> <20060507230430.GA6872@xor.obsecurity.org> <20060508065207.GA20386@xor.obsecurity.org> <20060509004328.GB55852@xor.obsecurity.org> <20060509011109.GA57060@xor.obsecurity.org> <20060509181302.GD3636@eucla.lemis.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hQiwHBbRI9kgIhsi" Content-Disposition: inline In-Reply-To: <20060509181302.GD3636@eucla.lemis.com> User-Agent: Mutt/1.4.2.1i Cc: performance@FreeBSD.org, Robert Watson , current@FreeBSD.org, Kris Kennaway Subject: Re: Fine-grained locking for POSIX local sockets (UNIX domain sockets) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 May 2006 18:23:37 -0000 --hQiwHBbRI9kgIhsi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 09, 2006 at 11:13:02AM -0700, Greg 'groggy' Lehey wrote: > On Monday, 8 May 2006 at 21:11:09 -0400, Kris Kennaway wrote: > > On Mon, May 08, 2006 at 08:43:28PM -0400, Kris Kennaway wrote: > >> On Mon, May 08, 2006 at 02:52:07AM -0400, Kris Kennaway wrote: > >>> OK, David's patch fixes the umtx thundering herd (and seems to give a > >>> 4-6% boost). I also fixed a thundering herd in FILEDESC_UNLOCK (which > >>> was also waking up 2-7 CPUs at once about 30% of the time) by doing > >>> s/wakeup/wakeup_one/. This did not seem to give a performance impact > >>> on this test though. > >> > >> Turning down kern.hz from 1000 to 100 also made a big difference on 12 > >> CPUs (+6.1%). > >> > >> Note also that the system is no less than 40% idle during the runs (at > >> any load), so the bottlenecks are serious. > > > > top -H shows the threads mostly in umtx state. >=20 > This doesn't lend much support to the idea that the gettimeofday() > calls are a bottleneck. There are at least several issues here: * Factor of >two difference in performance across the board (all loads) relative to Linux. This may be general issues like gettimeofday() on Linux vs FreeBSD; clearly there is something *very big* to blame here. Mysql does do *lots* of such calls, so the cost of them is surely a component in performance, the only question is if it's the main one. * When you get some of the locking out of the way (per this thread) FreeBSD has 44% better peak performance on Sven's test on amd64, but tails off by about 33% at higher loads compared to unpatched. I see similar changes on 12-CPU E4500, but not as much performance gain (may be due to other reasons). i.e. optimizing the locking allows a new, bigger bottleneck to suck on center stage. This is the basis for my observation about libthr at high loads. It is not the same as the above issue. Kris --hQiwHBbRI9kgIhsi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEYN4iWry0BWjoQKURAnNGAKDveYJUb9GfgI60pnxB+lxlgwWXiACdEy5h j5L+YV64tSOPEWKpN7+/iWE= =kYah -----END PGP SIGNATURE----- --hQiwHBbRI9kgIhsi--