Date: Wed, 27 Feb 2002 15:49:13 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: Bosko Milekic <bmilekic@unixdaemons.com> Cc: Jeff Roberson <jroberson@chesapeake.net>, arch@FreeBSD.ORG Subject: Re: Slab allocator Message-ID: <200202272349.g1RNnDF31841@apollo.backplane.com> References: <20020227172755.W59764-100000@mail.chesapeake.net> <200202272307.g1RN75T31581@apollo.backplane.com> <20020227183124.B62898@unixdaemons.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:..
:> FAST interrupts do not usually call into other subsystems, to
:> prevent unexpected alterations of the per-cpu data. They should
:> not be thought of as a mechanism that blocks interrupts.
:
: Well, we're going to be doing context-stealing soon, don't forget
:about that. That means that if we're in a critical section, we won't be
:able to take advantage of stealing by taking the context and running the
:handler immediately. So, to schedule an ithread we will need to get some
:sort of sched_lock. Sure, perhaps it won't be _the_ sched_lock, but it
:will have to be some sort of lock for the run queue we will be placing
:our ithread on when not performing context stealing. Personally, I don't
:believe that having the fast-path allocation be a critical section.
:We're headed in the pre-emption path and disabling pre-emption every
:time I want to allocate something is pretty unreasonable [*].
:
:[*] Yes, I know that you don't like pre-emption.
:
:--
:Bosko Milekic
If an i-scheduled interrupt is blocked by a critical section which only
lasts 1 uS, there will be no detrimental effect on system performance.
If our codebase is pockmarked with thousands of little 1uS-long
critical sections it will have no cumulative effect whatsoever on
interrupt latencies. None. This may seem counter-intuitive but it's
true, and anyone who has ever written embedded software will tell you
the same thing. It isn't the number of places you disable preemption
that kills you, it's the LONGEST place you disable preemption that
kills you. It's true on everything from a little 8 bit microcomputer
to an IBM mainframe.
Or, to put it another way, every time the system takes an interrupt
it disrupts code flow and pollutes the L1 and L2 caches of the cpu
if the cpu is otherwise doing something useful (like running a system
call). If an interrupt does not require real time response, there is
no point disrupting the system in order to give it real time response.
The system will actually operate more efficiently if you allow it to
continue whatever it was already doing, at least for a little while.
This doesn't mean that preemption is bad, just that it is bad if
you go overboard trying to do it. Being rabid about critical sections
classifies as "going overboard". Critical sections can save far, far
more clock cycles then non-conflicting mutexes without any effect on
interrupt performance if used properly.
Also keep in mind that a critical section != sched_lock. Just using
a critical section in something like the SLAB allocator does not
have the same detrimental effect that holding the sched_lock
(which obtains a critical section) has. Bruce has demonstrated this
and it should be an obvious truism to anyone in SMP land. We need
to work on reducing the time we hold sched_lock, but that is a very
different issue then using a critical section to protect a per-process
cache.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200202272349.g1RNnDF31841>