Date: Tue, 9 May 2006 14:23:30 -0400 From: Kris Kennaway <kris@obsecurity.org> To: Greg 'groggy' Lehey <grog@FreeBSD.org> Cc: performance@FreeBSD.org, Robert Watson <rwatson@FreeBSD.org>, current@FreeBSD.org, Kris Kennaway <kris@obsecurity.org> Subject: Re: Fine-grained locking for POSIX local sockets (UNIX domain sockets) Message-ID: <20060509182330.GB92714@xor.obsecurity.org> In-Reply-To: <20060509181302.GD3636@eucla.lemis.com> References: <20060506150622.C17611@fledge.watson.org> <20060506221908.GB51268@xor.obsecurity.org> <20060507210426.GA4422@xor.obsecurity.org> <20060507214153.GA5275@xor.obsecurity.org> <20060507230430.GA6872@xor.obsecurity.org> <20060508065207.GA20386@xor.obsecurity.org> <20060509004328.GB55852@xor.obsecurity.org> <20060509011109.GA57060@xor.obsecurity.org> <20060509181302.GD3636@eucla.lemis.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--hQiwHBbRI9kgIhsi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 09, 2006 at 11:13:02AM -0700, Greg 'groggy' Lehey wrote: > On Monday, 8 May 2006 at 21:11:09 -0400, Kris Kennaway wrote: > > On Mon, May 08, 2006 at 08:43:28PM -0400, Kris Kennaway wrote: > >> On Mon, May 08, 2006 at 02:52:07AM -0400, Kris Kennaway wrote: > >>> OK, David's patch fixes the umtx thundering herd (and seems to give a > >>> 4-6% boost). I also fixed a thundering herd in FILEDESC_UNLOCK (which > >>> was also waking up 2-7 CPUs at once about 30% of the time) by doing > >>> s/wakeup/wakeup_one/. This did not seem to give a performance impact > >>> on this test though. > >> > >> Turning down kern.hz from 1000 to 100 also made a big difference on 12 > >> CPUs (+6.1%). > >> > >> Note also that the system is no less than 40% idle during the runs (at > >> any load), so the bottlenecks are serious. > > > > top -H shows the threads mostly in umtx state. >=20 > This doesn't lend much support to the idea that the gettimeofday() > calls are a bottleneck. There are at least several issues here: * Factor of >two difference in performance across the board (all loads) relative to Linux. This may be general issues like gettimeofday() on Linux vs FreeBSD; clearly there is something *very big* to blame here. Mysql does do *lots* of such calls, so the cost of them is surely a component in performance, the only question is if it's the main one. * When you get some of the locking out of the way (per this thread) FreeBSD has 44% better peak performance on Sven's test on amd64, but tails off by about 33% at higher loads compared to unpatched. I see similar changes on 12-CPU E4500, but not as much performance gain (may be due to other reasons). i.e. optimizing the locking allows a new, bigger bottleneck to suck on center stage. This is the basis for my observation about libthr at high loads. It is not the same as the above issue. Kris --hQiwHBbRI9kgIhsi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEYN4iWry0BWjoQKURAnNGAKDveYJUb9GfgI60pnxB+lxlgwWXiACdEy5h j5L+YV64tSOPEWKpN7+/iWE= =kYah -----END PGP SIGNATURE----- --hQiwHBbRI9kgIhsi--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060509182330.GB92714>