Date: Sat, 20 Sep 2003 10:09:11 -0700 From: Luigi Rizzo <rizzo@icir.org> To: John Polstra <jdp@polstra.com> Cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking Message-ID: <20030920100911.B55993@xorpc.icir.org> In-Reply-To: <XFMail.20030920094820.jdp@polstra.com>; from jdp@polstra.com on Sat, Sep 20, 2003 at 09:48:20AM -0700 References: <606095639.1063987155@melange.errno.com> <XFMail.20030920094820.jdp@polstra.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Sep 20, 2003 at 09:48:20AM -0700, John Polstra wrote:
...
> > Note that the fxp driver holds its driver lock for an average 112 us in one
> > spot. This turns out to be in fxp_tick and the likely culprit is the call
> > to mii_tick which is done with the lock held.
>
> Agreed. The MII calls usually have lots of DELAY() calls in them.
>
> > I'm not sure if it's simple to move MII operations outside of the driver
> > lock but it might be worth investigating.
>
> It seems like it ought to be possible to at least lock at a finer
> grain for those calls. The driver lock is held because the MII
the main problem, as i see it, is that when there are PHY events you
still need to do some expensive work while holding a lock that
blocks interrupts, with very bad impact on the worst-case
response of the system.
One option in these cases could be to do the following:
1. disable the offending device while doing the expensive work
(e.g. reconfigure the PHY);
2. acquire a different lock (lock2) on the device;
3. release the lock that prevents (other) interrupts to be delivered
4. do the thing, hoever long it takes
5. re-enable the device
6. release the lock2
this would work fine for PHY events because the interface is
presumably not in a state to process traffic anyways, so disabling
it should have no bad side effects;
maybe other drivers (e.g. parallel ports, or devices where you have
to do bit-banging to move data without too many timing constraints)
could use a similar approach.
cheers
luigi
> routines call back into the driver and use it to do the bit-twiddling
> necessary to shift data into and out of the PHY registers via the
> PHY's serial interface. That's normally pretty independent of
> anything else that's going on in the chip, so it could probably be
> done under a separate lock.
>
> Another idea would be to substitute something else for DELAY() calls
> of more than a few microseconds. I am not very up-to-date on -current
> these days, but as far as I know DELAY() is still a spin-wait. If
> the delay is for more than a few microseconds it would probably be
> better to switch to a different runnable thread and come back later.
> In fact, CPU speeds may have reached the point where that is always
> the right thing to do.
>
> Finally, the entire mii_tick() nonsense could be thrown out for many
> modern network adapters. Both the Broadcom and Intel gigabit chips
> are capable of interrupting on interesting link events, so there is no
> need at all to poll the PHY every second. That doesn't appear to be
> true of the fxp devices, though. They need to be polled. Still, you
> can avoid the expensive PHY reads and writes most of the time. For
> example, if you have received any packets on the interface in the past
> second, you can assume that link is good and never touch the PHY at
> all.
>
> John
>
> _______________________________________________
> freebsd-arch@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030920100911.B55993>