From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 10:09:19 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7029E16A4B3 for ; Sat, 20 Sep 2003 10:09:19 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 828F643FFD for ; Sat, 20 Sep 2003 10:09:18 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.9p1/8.12.3) with ESMTP id h8KH9Bsd056568; Sat, 20 Sep 2003 10:09:11 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.9p1/8.12.3/Submit) id h8KH9BCV056567; Sat, 20 Sep 2003 10:09:11 -0700 (PDT) (envelope-from rizzo) Date: Sat, 20 Sep 2003 10:09:11 -0700 From: Luigi Rizzo To: John Polstra Message-ID: <20030920100911.B55993@xorpc.icir.org> References: <606095639.1063987155@melange.errno.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from jdp@polstra.com on Sat, Sep 20, 2003 at 09:48:20AM -0700 cc: Sam Leffler cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Sep 2003 17:09:19 -0000 On Sat, Sep 20, 2003 at 09:48:20AM -0700, John Polstra wrote: ... > > Note that the fxp driver holds its driver lock for an average 112 us in one > > spot. This turns out to be in fxp_tick and the likely culprit is the call > > to mii_tick which is done with the lock held. > > Agreed. The MII calls usually have lots of DELAY() calls in them. > > > I'm not sure if it's simple to move MII operations outside of the driver > > lock but it might be worth investigating. > > It seems like it ought to be possible to at least lock at a finer > grain for those calls. The driver lock is held because the MII the main problem, as i see it, is that when there are PHY events you still need to do some expensive work while holding a lock that blocks interrupts, with very bad impact on the worst-case response of the system. One option in these cases could be to do the following: 1. disable the offending device while doing the expensive work (e.g. reconfigure the PHY); 2. acquire a different lock (lock2) on the device; 3. release the lock that prevents (other) interrupts to be delivered 4. do the thing, hoever long it takes 5. re-enable the device 6. release the lock2 this would work fine for PHY events because the interface is presumably not in a state to process traffic anyways, so disabling it should have no bad side effects; maybe other drivers (e.g. parallel ports, or devices where you have to do bit-banging to move data without too many timing constraints) could use a similar approach. cheers luigi > routines call back into the driver and use it to do the bit-twiddling > necessary to shift data into and out of the PHY registers via the > PHY's serial interface. That's normally pretty independent of > anything else that's going on in the chip, so it could probably be > done under a separate lock. > > Another idea would be to substitute something else for DELAY() calls > of more than a few microseconds. I am not very up-to-date on -current > these days, but as far as I know DELAY() is still a spin-wait. If > the delay is for more than a few microseconds it would probably be > better to switch to a different runnable thread and come back later. > In fact, CPU speeds may have reached the point where that is always > the right thing to do. > > Finally, the entire mii_tick() nonsense could be thrown out for many > modern network adapters. Both the Broadcom and Intel gigabit chips > are capable of interrupting on interesting link events, so there is no > need at all to poll the PHY every second. That doesn't appear to be > true of the fxp devices, though. They need to be polled. Still, you > can avoid the expensive PHY reads and writes most of the time. For > example, if you have received any packets on the interface in the past > second, you can assume that link is good and never touch the PHY at > all. > > John > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"