From owner-freebsd-current  Tue Apr 17 17:47:35 2001
Delivered-To: freebsd-current@freebsd.org
Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67])
	by hub.freebsd.org (Postfix) with ESMTP id A623637B423
	for <current@FreeBSD.ORG>; Tue, 17 Apr 2001 17:47:31 -0700 (PDT)
	(envelope-from dillon@earth.backplane.com)
Received: (from dillon@localhost)
	by earth.backplane.com (8.11.2/8.11.2) id f3I0lN615938;
	Tue, 17 Apr 2001 17:47:23 -0700 (PDT)
	(envelope-from dillon)
Date: Tue, 17 Apr 2001 17:47:23 -0700 (PDT)
From: Matt Dillon <dillon@earth.backplane.com>
Message-Id: <200104180047.f3I0lN615938@earth.backplane.com>
To: Greg Lehey <grog@lemis.com>
Cc: Alfred Perlstein <bright@wintelcom.net>,
	"Justin T. Gibbs" <gibbs@scsiguy.com>,
	Doug Barton <DougB@DougBarton.net>,
	"current @ freebsd . org" <current@FreeBSD.ORG>
Subject: Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)
References: <200104160259.f3G2xqs06321@aslan.scsiguy.com> <200104160616.f3G6GI973782@earth.backplane.com> <20010417011957.W976@fw.wintelcom.net> <20010418093212.A80877@wantadilla.lemis.com>
Sender: owner-freebsd-current@FreeBSD.ORG
Precedence: bulk
X-Loop: FreeBSD.ORG


:*sigh* Couldn't you have changed the subject line when discussing
:something of this importance?
:
:Greg

    Sorry.  Now I am so much in a huff I'm thinking about how all this
    could be implemented from scratch with the 4.x base.  I know, I know,
    good luck Matt...

    For example, this business about interrupts as threads.  BSDI had an
    interesting solution, but what I liked most about it was that if an
    interrupt could be completed right then and there it was just like a
    normal interrupt.  It's the bit about switching stacks, detecting a
    mutex interlock, and blocking as a thread which was the complex part.
    But I think I just now came up with a better one (and to be fair, I
    just came up with this now).

    Interrupts by definition know precisely what they are going to do, so by
    definition they know precisely which mutexes (if any) they may need
    to get.  This means that, in fact, it is possible to implement a check
    to determine if any of the mutexes an interrupt might want to get are
    already being held by the SAME cpu or not, and if they are to do the
    equivalent of what our delayed-interrupt stuff does in the stable's
    spl/splx code, but instead do the check when a mutex is released.

    The result is:  No need for an idle process to support interrupt
    contexts, no need to implement interrupts as threads, and no need
    to implement fancy scheduling tricks or Giant handling.

    4.x:

	mainline kernel {
	    s = splblah();
	    .
	    masked interrupt occurs, sets bit and immediately irets
	    .
	    .
	    splx(s); 
	    (bit found to be set and delayed interrupt is issued here)
	}

    Proposed:

	mainline kernel {
	    get_spin_mutex(&somemutex);
	    .
	    .
	    masked interrupt occurs, interrupt structure contains array
	    of mutexes the interrupt will need.  Check said mutexes, one
	    found to be held by current cpu.  Set interrupt-pending bit
	    in mutex and iret immediately.
	    .
	    .
	    release_spin_mutex(&somemutex)
	    (bit found to be set in mutex, triggers interrupt reissuing code)
	}

    And there you have it.  The mutex/array test is takes very little time
    being a read-only test that requires no bus locking, and the collision
    case is cheap also because the current cpu already owns the mutex, allowing
    us to set the interrupt-pending bit in that mutex without any bus
    locking.  The check during the release of the mutex is two instructions,
    no bus locking required.  The whole thing can be implemented without any
    additional bus locking and virtually no contention.

    The case could be further optimized by requiring that interrupts only
    use a single mutex, period.  This would allow the mainline interrupt
    routine to obtain the mutex on entry to the interrupt and allow the 
    reissuing code to reissue the interrupt without freeing the mutex that
    caused the reissue, so the mutex is held throughout and then freed by
    the interrupt itself.

    Holy shit.  I think that's it!  I don't think it can get much better then
    that.  It solves all of BDE's issues, solves the interrupt-as-thread
    issue (by not using threads for interrupts at all), and removes a huge
    amount of unnecessary complexity from the system.  We could even get rid
    of the idle processes if we wanted to.

							-Matt


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message