From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 17:55:26 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A39E3106567E; Mon, 13 Apr 2009 17:55:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0120E8FC24; Mon, 13 Apr 2009 17:55:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AC66246B51; Mon, 13 Apr 2009 13:55:24 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 14CB18A050; Mon, 13 Apr 2009 13:54:49 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 13 Apr 2009 12:53:36 -0400 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200904131253.37019.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 13 Apr 2009 13:54:49 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.1 required=4.2 tests=RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Robert Watson , Andrew Brampton Subject: Re: FreeBSD memguard + spinlocks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2009 17:55:26 -0000 On Saturday 11 April 2009 5:03:58 pm Andrew Brampton wrote: > 2009/4/11 Robert Watson : > > On Sat, 11 Apr 2009, Andrew Brampton wrote: > > > > Your understanding is mostly right. =C2=A0The missing bit is this: ther= e are=20 two > > kinds of interrupt contexts -- fast/filter interrupt handlers, which=20 borrow > > the stack and execution context of the kernel thread they preempt, and > > interrupt threads, which get their own complete thread context. > > > > Fast interrupt handlers are allowed unlock to acquire spinlocks so as to > > avoid deadlock because of the borrowed context. =C2=A0This means they c= an't > > perform any sort of sleep, or acquire any locks that might sleep, since= =20 the > > thread they've preempted may hold conflicting locks, or be the one that > > would have woken up the sleep that the handler performed. =C2=A0Almost = no code > > will run in fast handlers -- perhaps checking some device registers, do= ing > > work on a lockless or spinlock-protected queue, and waking up a worker > > thread. > > > > This is why, BTW, spin locks disable interrupt: they need to control > > preemption by other interrupt handlers to avoid deadlock, but they are = not > > intended for use except when either in the scheduler, in a few related = IPI > > contexts, or when synchronizing between normal kernel code and a fast > > handler. > > > > Full interrupt thread contexts are permitted to perform short lock slee= ps, > > such as those performed when contending default mutexes, rwlocks, and > > rmlocks. They are permitted to invoke kernel services such as malloc(9), > > UMA(9), the network stack, etc, as long as they use M_NOWAIT and don't > > invoke msleep(9) or similar unbounded sleeps -- again to avoid the > > possibility of deadlocks, since you don't want an interrupt thread=20 sleeping > > waiting for an event that only it can satisfy. > > > > So the first question, really, is whether you are or mean to be using > > fast/filter interrupt handler. =C2=A0Device drivers will never call mem= ory > > allocation, free, etc, from there, but will defer it to an ithread usin= g=20 the > > filter mechanism in 8.x, or to a task queue or other worker in 7.x and > > earlier. =C2=A0If you're using a regular INTR_MPSAFE ithread, you shoul= d be=20 able > > to use only default mutexes (a single atomic operation if uncontended) > > without disabling interrupts, etc. > > > > Robert N M Watson > > Computer Laboratory > > University of Cambridge > > >=20 > Anyway, that is why I also asked about a lighter weight spin lock > (perhaps similar to this one). I tempted to replace this custom > spinlock with the standard MTX_DEF, however I'm unsure of its impact. > The custom spin lock seems quick and light to acquire, and it does not > concern me that a interrupt can potentially interrupt the code. You should just use a MTX_DEF mutex. Also, if you use M_NOWAIT, you will n= eed=20 to handle malloc() returning NULL. In general I try to allocate things whi= le=20 not holding any locks when possible and only acquire the lock to initialize= =20 the memory returned from malloc(). =2D-=20 John Baldwin