From owner-freebsd-current@FreeBSD.ORG Mon May 3 10:10:29 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E05B616A4E5 for ; Mon, 3 May 2004 10:10:29 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6278D43D58 for ; Mon, 3 May 2004 10:10:16 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i43HAALF019206; Mon, 3 May 2004 13:10:10 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i43H9x3f018983; Mon, 3 May 2004 13:10:10 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Mon, 3 May 2004 13:09:59 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Bruce Evans In-Reply-To: <20040503131929.K3770@gamplex.bde.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: bm@malepartus.de cc: freebsd-current@freebsd.org cc: "M. Warner Losh" Subject: Re: sio: lots of silo overflows on Asus K8V with Moxa Smartio C104H/PCI X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2004 17:10:30 -0000 On Mon, 3 May 2004, Bruce Evans wrote: > On Sun, 2 May 2004, M. Warner Losh wrote: > > > In message: <20040502130027.O1806@gamplex.bde.org> > > Bruce Evans writes: > > : No, but siointr() doesn't go near the tty layer or almost any other > > : layer. It uses pseudo-dma and schedules a SWI to transfer data between > > : its buffers and the tty layer. Fast interrupt handlers cannot call > > : other layers because they cannot reasonable lock other layers or > > : vice versa. (Even the call to the scheduler to schedule a SWI in the > > : current implementation is a layering violation.) > > > > So it is also safe to call the tty layer w/o giant held from the SWI? > > I guess that was my question. Or is the SWI wrapped in Giant when I > > wasn't looking... > > Unsafe. SWIs are wrapped by Giant unless they are created using > swi_add(... INTR_MPSAFE ...). There are 26 calls to swi_add() in the > tree, and the slow progress of Giant removal is shown by only 3 of > these using INTR_MPSAFE. The ones in sio are not in the 3. The 3 > are all just for multiplexors that push down deciding whether to use > Giant to lower layers: Actually, it's a little more complex than that: most of the callers if swi_add() with INTR_MPSAFE are actually via bus_setup_intr() which accepts similar flags. You'll find that a much larger number of interrupts are capable of running mpsafe if you search for that. Also, unless debug.mpsafenet is turned on, the interrupt setup code will ignore the INTR_MPSAFE flag for interrupts with INTR_TYPE_NET set. Obviously, you don't want to run with debug.mpsafenet set unless you're also running with the network stack locking patches :-). > - softclock: handles timeout. CALLOUT_MPSAFE specifies that Giant > is not needed (or has been pushed down further) for individual > timeouts. > - swi_net: calls netisrs. NETISR_MPSAFE specifies that Giant is not > needed (or ...) for individual netisrs. > - taskqueue_swi_run: calls queued MPSAFE tasks. Non-MPSAFE tasks are > on another queue. I've been working through some of the callouts in the system -- we have a number of low-hanging fruit, such as some of the kernel log subsystem pieces, that we might be able to handle pretty easily. I have the NFS server largely running without Giant (with the exception of entry into VFS, where it has to grab Giant) and probably ought to merge those patches sometime soon. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research