From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 14:40:56 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BE9616A4CE for ; Mon, 22 Mar 2004 14:40:56 -0800 (PST) Received: from mail3.speakeasy.net (mail3.speakeasy.net [216.254.0.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 593AE43D2D for ; Mon, 22 Mar 2004 14:40:56 -0800 (PST) (envelope-from jhb@FreeBSD.org) Received: (qmail 25723 invoked from network); 22 Mar 2004 22:40:55 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 22 Mar 2004 22:40:55 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.10/8.12.10) with ESMTP id i2MMen28000256; Mon, 22 Mar 2004 17:40:50 -0500 (EST) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Seigo Tanimura Date: Mon, 22 Mar 2004 14:20:39 -0500 User-Agent: KMail/1.6 References: <200403160519.i2G5J0V6023193@urban> <200403161009.48938.john@baldwin.cx> <200403220657.i2M6vCrS097750@shojaku.t.axe-inc.co.jp> In-Reply-To: <200403220657.i2M6vCrS097750@shojaku.t.axe-inc.co.jp> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200403221420.39732.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@FreeBSD.org Subject: Re: Is MTX_CONTESTED evil? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2004 22:40:56 -0000 On Monday 22 March 2004 01:57 am, Seigo Tanimura wrote: > On Tue, 16 Mar 2004 10:09:48 -0500, > John Baldwin said: > > john> On Tuesday 16 March 2004 12:19 am, Seigo Tanimura wrote: > >> _mtx_unlock_sleep() currently wakes up only one thread being blocked, > >> and leaves MTX_CONTESTED on a mutex. According to Solaris Internals, > >> that strategy adds an overhead to check for MTX_CONTESTED on a mutex, > >> even though it is not held by any thread. The thread waken up cannot > >> grab the mutex immediately by _obtain_lock() and have to go through > >> _mtx_lock_sleep(). The penalty tends to be large for a mutex with a > >> high contention, and we have at least one of such a mutex - Giant. > >> > >> What would it be like if we axed MTX_CONTEST and let > >> _mtx_unlock_sleep() wake up all of the blocked threads? > > john> We wouldn't be able to axe MTX_CONTEST. We also use it to determine > on unlock john> if we can unlock easily or if we have waiters that we need > to awake. The john> only way we might be able to axe MTX_CONTEST would be > to penalize every john> unlock operation requiring a turnstile lookup (spin > lock acquire/release + john> hash table lookup) even unlocks of an > uncontested mutex. However, what I john> think you want to do is get rid > of the mtx_lock == MTX_CONTESTED case and use john> turnstile_wakeup() > rather than turnstile_signal()? Is that what you are > > Yes. What I an wondering is whether the reduction of the cost due to > a mutex with waiters and no holders can beat the cost of waking up all > the waiters on the turnstile. > > > john> asking? That is something we can try at some point in the future, > but we john> would need to benchmark both ways. What we might can do is > add a kernel john> option MUTEX_WAKE_ALL or some such that uses the Solaris > behavior. Having it john> be an option like ADAPTIVE_MUTEXES makes it > easier to benchmark both cases. > > > On the detection of the waiters by MTX_CONTEST, maybe we can test > MTX_CONTEST on mtx_lock before performing _release_lock(). If the > test succeeds, _mtx_unlock_sleep() must be called and we do not need > to perform an atomic test-and-set. A race can occur if the mutex is > locked after the MTX_CONTEST test, but _release_lock() should then > cover the case. > > Pseudocode: > > mtx_unlock(m) > { > if (m->mtx_lock & MTX_CONTEST || !_release_lock(m)) > _mtx_unlock_sleep(m); > } I would just always do the atomic op to avoid penalizing the common fast case (not contested, not recursed). BTW, last week I did implement the MUTEX_WAKE_ALL kernel option in the jhb_lock p4 branch. I've been very busy with work stuff though for several days now, but when I get some free time again I'll post the patch so people can play with it. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org