Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 22 Mar 2004 14:20:39 -0500
From:      John Baldwin <jhb@FreeBSD.org>
To:        Seigo Tanimura <tanimura@tanimura.dyndns.org>
Cc:        arch@FreeBSD.org
Subject:   Re: Is MTX_CONTESTED evil?
Message-ID:  <200403221420.39732.jhb@FreeBSD.org>
In-Reply-To: <200403220657.i2M6vCrS097750@shojaku.t.axe-inc.co.jp>
References:  <200403160519.i2G5J0V6023193@urban> <200403161009.48938.john@baldwin.cx> <200403220657.i2M6vCrS097750@shojaku.t.axe-inc.co.jp>

next in thread | previous in thread | raw e-mail | index | archive | help
On Monday 22 March 2004 01:57 am, Seigo Tanimura wrote:
> On Tue, 16 Mar 2004 10:09:48 -0500,
>   John Baldwin <john@baldwin.cx> said:
>
> john> On Tuesday 16 March 2004 12:19 am, Seigo Tanimura wrote:
> >> _mtx_unlock_sleep() currently wakes up only one thread being blocked,
> >> and leaves MTX_CONTESTED on a mutex.  According to Solaris Internals,
> >> that strategy adds an overhead to check for MTX_CONTESTED on a mutex,
> >> even though it is not held by any thread.  The thread waken up cannot
> >> grab the mutex immediately by _obtain_lock() and have to go through
> >> _mtx_lock_sleep().  The penalty tends to be large for a mutex with a
> >> high contention, and we have at least one of such a mutex - Giant.
> >>
> >> What would it be like if we axed MTX_CONTEST and let
> >> _mtx_unlock_sleep() wake up all of the blocked threads?
>
> john> We wouldn't be able to axe MTX_CONTEST.  We also use it to determine
> on unlock john> if we can unlock easily or if we have waiters that we need
> to awake.  The john> only way we might be able to axe MTX_CONTEST would be
> to penalize every john> unlock operation requiring a turnstile lookup (spin
> lock acquire/release + john> hash table lookup) even unlocks of an
> uncontested mutex.  However, what I john> think you want to do is get rid
> of the mtx_lock == MTX_CONTESTED case and use john> turnstile_wakeup()
> rather than turnstile_signal()?  Is that what you are
>
> Yes.  What I an wondering is whether the reduction of the cost due to
> a mutex with waiters and no holders can beat the cost of waking up all
> the waiters on the turnstile.
>
>
> john> asking?  That is something we can try at some point in the future,
> but we john> would need to benchmark both ways.  What we might can do is
> add a kernel john> option MUTEX_WAKE_ALL or some such that uses the Solaris
> behavior.  Having it john> be an option like ADAPTIVE_MUTEXES makes it
> easier to benchmark both cases.
>
>
> On the detection of the waiters by MTX_CONTEST, maybe we can test
> MTX_CONTEST on mtx_lock before performing _release_lock().  If the
> test succeeds, _mtx_unlock_sleep() must be called and we do not need
> to perform an atomic test-and-set.  A race can occur if the mutex is
> locked after the MTX_CONTEST test, but _release_lock() should then
> cover the case.
>
> Pseudocode:
>
> mtx_unlock(m)
> {
> 	if (m->mtx_lock & MTX_CONTEST || !_release_lock(m))
> 		_mtx_unlock_sleep(m);
> }

I would just always do the atomic op to avoid penalizing the common fast case 
(not contested, not recursed).  BTW, last week I did implement the 
MUTEX_WAKE_ALL kernel option in the jhb_lock p4 branch.  I've been very busy 
with work stuff though for several days now, but when I get some free time 
again I'll post the patch so people can play with it.

-- 
John Baldwin <jhb@FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200403221420.39732.jhb>