Date: Wed, 30 Apr 2003 20:50:36 +1000 (EST) From: Bruce Evans <bde@zeta.org.au> To: Andrew Gallatin <gallatin@cs.duke.edu> Cc: "M. Warner Losh" <imp@bsdimp.com> Subject: Re: cvs commit: src/sys/dev/fxp if_fxp.c if_fxpvar.h Message-ID: <20030430195827.V15323@gamplex.bde.org> In-Reply-To: <16046.60754.563011.201060@grasshopper.cs.duke.edu> References: <16046.55840.945040.149302@grasshopper.cs.duke.edu> <16046.60754.563011.201060@grasshopper.cs.duke.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 29 Apr 2003, Andrew Gallatin wrote:
> John Baldwin writes:
> > ...
> > I would need to hold the ithread lock around the TAILQ_FOREACH()
> > bit and drop it around the execution of the handlers, but I probably
> > need that anyways. Of course, all this really does, is take your
> > gone test and move it up in the stack. You are still checking a flag
> > on every interrupt handler invocation.
>
> Since you we already need to check for IH_DEAD, you could avoid extra
> compares by doing
>
> if (ih->ih_flags & (IH_DISABLED | IH_DISABLING | IH_DEAD)) {
> /* sort out rare cases outside the critical path */
> }
>
> Now we're doing only one check in the common case on something that
> you apparently need to check anyway. I'd be all for something like
> this.
I think the orphan handler idea can be used to remove all deadness tests
from the fast path. Something like:
In detach function:
- change handler to a non-orphan handler that disables further device
interrupts if possible, else calls the main handler. Since this handler
is not in the fast path, it can afford to do lots of state checking and
locking to decide what to do.
- wait until main handler is not running directly.
- wait/combine with non-orphan handler to disable device interrupts.
- change handler to orphan handler than does nothing when we know that
further (shared) interrupts are not for us (or that null handling of
(possibly non-shared) ones for us is correct). Loop back to previous
step and share locks with it as necessary.
The logic for this can be in bus_teardown_intr() (ithread_remove_handler())
or a prerequisite for bus_teardown_intr(). The device-specific parts need
to be in each driver of course.
In ithread_remove_handler():
- just assert that we always have the easy case where all handlers are already
dead (ophan).
This doesn't solve the problem that an interrupt handler for gone-away
hardware may spin forever reading garbage registers.
> > > No, in theory increased performance should come from increased
> > > parallelism with no increased overhead. Any increased overhead is a
> > > bug. Linux 2.4 runs a thread safe kernel with less overhead than we
> > > have in 4.x. Its possible.
> >
> > How are we going to achieve increased paralellism w/o increased overhead?
> > Discounting redesigning algo's which would have been a win in the
> > non-parallel kernel as well. A mutex is far more expensive than an spl.
> > You have to protect against more things. Of course overhead is going to
> > go up.
>
> I have no idea really. All I know is that linux 2.4 is more parallel
> than 5.x is today, yet it outperforms 4.x for things like system call
> latency and networking packets/sec through a single interface.
> Therefore it must be possible.
Heavyweight context switches and timestamps to make them super-heavy
don't help (except for debugging), but I'm more concerned about 5.x
having monotonically increasing overhead with x as finer grained locks
are added.
Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030430195827.V15323>