From owner-freebsd-arch@FreeBSD.ORG Wed Apr 30 23:31:21 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6938837B401; Wed, 30 Apr 2003 23:31:21 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id ACC9E43FB1; Wed, 30 Apr 2003 23:31:19 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id QAA22993; Thu, 1 May 2003 16:31:09 +1000 Date: Thu, 1 May 2003 16:31:08 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Andrew Gallatin In-Reply-To: <16047.59314.532227.475952@grasshopper.cs.duke.edu> Message-ID: <20030501144708.I18220@gamplex.bde.org> References: <200304290438.h3T4cdHE069528@arch20m.dellroad.org> <16047.59314.532227.475952@grasshopper.cs.duke.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-arch@freebsd.org Subject: Re: lots of malloc(M_WAITOK)'s in interrupt context from camisr X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 May 2003 06:31:21 -0000 On Wed, 30 Apr 2003, Andrew Gallatin wrote: > John Baldwin writes: > > > If you need to do more work in your interrupt routine than just wakeups > > and dinking with registers, you can always wake up a software interrupt > > handler or some other random kthread to do things that take a long amount (This is about normal interrupt handlers, not INTR_FAST ones.) > Dumb question: Exactly what is one allowed to do in an INTR_FAST > interrupt context? Obviously, you can't sleep. But can you call > wakeup()? You may access driver memory and device i/o space that you have suitably locked. That's all. You may not call any functions that may access other memory (other than the stack) or that may do unsuitable locking. In practice, this means that you should only call bus-space i/o functions and lower-level i/o functions (only after locking i/o accesses of course). The locking is too difficult or expensive to call more. Suitable locking methods include: (1) h/w disabling of interrupts for the !SMP case. (2) simple spinlocks for the SMP case. These must be combined with h/w disabling of interrupts to avoid deadlock if the lock contention occurs on the same CPU. Examples of this may be found in the sio driver in RELENG_4. The only known problems with this are that the lock is too gigantic, yet is not gigantic enough to give unbroken locking for the rule-breaking calls to the non-i/o functions pps_event() and microtime(). Unsuitable locking methods include: (1) Everything in the mtx family. These are used in -current, but that is a bug in -current. They are basically higher level functions. Using them at the lowest (INTR_FAST) level adds complications and overheads to both this level and higher levels. In practice, mtx spinlocks are not very different from the simple spinlocks used in RELENG_4. They just have extra complications and overheads to make them more general and then to reduce them back to simple spinlocks when they are used in INTR_FAST handlers. (2) Using locks for non-driver/non-device memory. These (notably sched_lock) are used in -current, but that is a bug in -current. It is much larger than the bug in (1). It makes all INTR_FAST handlers non-fast at least in the !SMP case, since they may be blocked by other INTR_FAST handlers that are blocked by the general locks even though they don't all have this bug. E.g., exit() holds sched_lock for a long time; clock interrupt handlers are blocked by sched_lock and (in the !SMP case) block all other INTR_FAST handlers while they are blocked; thus exit() may block all INTR_FAST handlers for a long time. In RELENG_4, exit() doesn't block fast interrupt handlers at all. wakeup() is one layer away from being callable, unless the bugs in -current permit it. Bruce