From owner-freebsd-performance@FreeBSD.ORG Mon May 8 18:43:32 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AF80616A400; Mon, 8 May 2006 18:43:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2A37243D4C; Mon, 8 May 2006 18:43:27 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 271391A3C1A; Mon, 8 May 2006 11:43:25 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 73733515DC; Mon, 8 May 2006 14:43:22 -0400 (EDT) Date: Mon, 8 May 2006 14:43:22 -0400 From: Kris Kennaway To: David Xu Message-ID: <20060508184322.GA49344@xor.obsecurity.org> References: <20060506150622.C17611@fledge.watson.org> <20060507230430.GA6872@xor.obsecurity.org> <20060508065207.GA20386@xor.obsecurity.org> <200605081843.31825.davidxu@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3V7upXqbjpZ4EhLz" Content-Disposition: inline In-Reply-To: <200605081843.31825.davidxu@freebsd.org> User-Agent: Mutt/1.4.2.1i Cc: performance@freebsd.org, freebsd-performance@freebsd.org, Robert Watson , current@freebsd.org, Kris Kennaway Subject: Re: Fine-grained locking for POSIX local sockets (UNIX domain sockets) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 May 2006 18:43:32 -0000 --3V7upXqbjpZ4EhLz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 08, 2006 at 06:43:31PM +0800, David Xu wrote: > On Monday 08 May 2006 14:52, Kris Kennaway wrote: > > OK, David's patch fixes the umtx thundering herd (and seems to give a > > 4-6% boost). I also fixed a thundering herd in FILEDESC_UNLOCK (which > > was also waking up 2-7 CPUs at once about 30% of the time) by doing > > s/wakeup/wakeup_one/. This did not seem to give a performance impact > > on this test though. > >.... > > filedesc contention is down by a factor of 3-4, with corresponding > > reduction in the average hold time. The process lock contention > > coming from the signal delivery wakeup has also gone way down for some > > reason. > > >=20 > I found that mysqld frequently calls alarm() in its file thr_alarm.c and= =20 > thr_kill() to send SIGALRM to its timer thread to wake it up, the timer= =20 > thread itself is being blocked in sigwait(), normally the alarm timer will > be expired in a second, so the kernel will periodically call psignal to f= ind > a thread which can handle the signal, it means kernel has to periodically > walk through thread list with process lock and scheduler held, this is > very expensive. >=20 > thr_kill will in most time wake up the timer thread earlier, in thr_kill > syscall, kernel has to walk through thread list to find a thread whose > thread is matching the given id, the function thread_find() > uses a linear searching algorithm, it is slow, if there are lots of threa= d in > the process, the process lock will be holden too long, I think that's th= e=20 > reason why you have seen so many process lock contention, if you > define USE_ALARM_THREAD in mysql header file, the contention should > be decreased ( I hope ), patch: Hmm, with this patch mysql 4.1 seems to crash at startup. I haven't yet had time to investigate. Is anyone else seeing this? Kris --3V7upXqbjpZ4EhLz Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEX5FJWry0BWjoQKURAs44AJ9xvSlgI2Ged4dpfI5Au728e4UxJQCfQ+MG Re10oKdTr9I47XG0rxx3yvA= =xvgt -----END PGP SIGNATURE----- --3V7upXqbjpZ4EhLz--