From owner-cvs-src@FreeBSD.ORG  Tue Nov  9 21:11:07 2004
Return-Path: <owner-cvs-src@FreeBSD.ORG>
Delivered-To: cvs-src@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id D297F16A4CE; Tue,  9 Nov 2004 21:11:07 +0000 (GMT)
Received: from fledge.watson.org (fledge.watson.org [204.156.12.50])
	by mx1.FreeBSD.org (Postfix) with ESMTP
	id 6A4AE43D2F; Tue,  9 Nov 2004 21:11:07 +0000 (GMT)
	(envelope-from robert@fledge.watson.org)
Received: from fledge.watson.org (localhost [127.0.0.1])
	by fledge.watson.org (8.13.1/8.13.1) with ESMTP id iA9L9twg061070;
	Tue, 9 Nov 2004 16:09:55 -0500 (EST)
	(envelope-from robert@fledge.watson.org)
Received: from localhost (robert@localhost)iA9L9tJn061067;
	Tue, 9 Nov 2004 21:09:55 GMT	(envelope-from robert@fledge.watson.org)
Date: Tue, 9 Nov 2004 21:09:55 +0000 (GMT)
From: Robert Watson <rwatson@FreeBSD.org>
X-Sender: robert@fledge.watson.org
To: Stephan Uphoff <ups@tree.com>
In-Reply-To: <1100024464.29384.30.camel@palm.tree.com>
Message-ID: <Pine.NEB.3.96L.1041109210808.60848B-100000@fledge.watson.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
cc: src-committers@FreeBSD.org
cc: John Baldwin <jhb@FreeBSD.org>
cc: Alan Cox <alc@FreeBSD.org>
cc: cvs-src@FreeBSD.org
cc: Mike Silbersack <silby@silby.com>
cc: cvs-all@FreeBSD.org
cc: Julian Elischer <julian@elischer.org>
Subject: Re: cvs commit: src/sys/i386/i386 pmap.c
X-BeenThere: cvs-src@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: CVS commit messages for the src tree <cvs-src.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/cvs-src>,
	<mailto:cvs-src-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/cvs-src>
List-Post: <mailto:cvs-src@freebsd.org>
List-Help: <mailto:cvs-src-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/cvs-src>,
	<mailto:cvs-src-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 09 Nov 2004 21:11:08 -0000


On Tue, 9 Nov 2004, Stephan Uphoff wrote:

> > Can you explain why a spin mutex is more expensive than a sleep mutex (I 
> > assume this is uncontested)?
> 
> cli() and sti() used for the critical section are expensive.  ( The spin
> mutex includes the critical section) 
> 
> I recall a USENIX paper about avoiding the cost of cli(),sti() by just
> setting an in memory flag. The interrupt handler was modified to honor
> the flag and delay interrupt processing until the flag was cleared. 
> This may have the potential to drastically decrease the cost of a spin
> mutex if interrupts during critical regions are infrequent. 

This is present in FreeBSD 4.x, where spls use soft interrupt levels
rather than hardware disabling of interrupts.  It's also present in John's
jhb_lock branch, which hopefully he will merge soon.  It gets critical
section enter/exit, with the exception of around spinlocks, down to about
30-40 cycles on the P4.  I have a modified version of UMA in the
rwatson_percpu branch that uses critical sections instead of mutexes to
protect the per-CPU caches, which presumably makes the most sense when
combined with optimized critical sections once that's committed.  FWIW,
although it all micro-benchmarks much faster, I haven't yet demonstrated
that combining the light-weight critical sections with critical sections
around UMA per-CPU caches actually runs faster.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org      Principal Research Scientist, McAfee Research