Date: Thu, 02 May 2002 10:31:55 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: John Baldwin <jhb@FreeBSD.org> Cc: freebsd-smp@FreeBSD.org, Andrew Gallatin <gallatin@cs.duke.edu>, Jonathan Mini <mini@FreeBSD.org> Subject: Re: hlt when idle? Message-ID: <3CD1780B.2CC97DDB@mindspring.com> References: <XFMail.20020502131842.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
John Baldwin wrote:
> > It definitely does, provided you are guaranteed that the clock
> > interrupt handler can always execute in less time than the
> > interval between clock interrupts. Then you don't have to
> > contend on the global handler lock to ensure it only runs once.
>
> Well, with our current interrupts the foo_process() (should be foo_thread())
> versions still need the global sched_lock to modify some flag fields and what
> not. (Some of this could be eliminated by splitting td_flags into two parts,
> one that needs the lock and one that doesn't.) Thus, if they all fire at once,
> even if it's broadcast they all contend with each other, wherease if they are
> broadcast but staggered such as on the alpha they don't in theory contend with
> each other at all. (Provided the handler is fast enough, which it should be.)
Ah. I omitted a piece because it was obvious to me, when in
retrospect it's really non-obvious.
You reprogram the top 32 bits of the ICR in the APIC for the
interrupt each time as part of the interrupt handler, so it's
actually unicast to a single processor to get the "round robin"
effect ("bother that CPU instead of me next time").
IMO, going round-robin on this sort of thing is a bad idea;
L1 and L2 cache busting are a biggie for me -- but then I want
to scale to an infinite number of processors. 8-).
If you wanted to broadcast a clock interrupt, which makes more
sense for me, then the system "clock interrupt" processing
would be handled on a single CPU.
Alternately, you could rewrite the upper 32 bits, rewrite the
lower 32 bits (which would trigger the IPI to be sent), and
then rewrite the upper 32 bits again, to target the next guy
in line.
I think the confusion is because it's useful for the clock to
be broadcast for one set of reasons, and it's useful to hit on
only one processor for global handler processing for another
set of reasons.
I think a lot of the desire to hit a CPU over the head to have
it look at the scheduler queue comes because the scheduler
queues aren't per-CPU, and thus CPU migration is not a really
rare event, like it should be (in order to get CPU affinity).
In the migration case, an IPI targetting only the migrated-to
CPU, only if it's HLT'ed, is probably acceptable overhead.
Actually, here's an incredibly good reference for Intel APIC
and SMP processing, from Purdue University:
Multiprocessing Support for Hobby OSes Explained
Ben L. Titzer
http://expert.cc.purdue.edu/~titzer/redpants/mpx86.html
It goes into the details of ICR programming, which is where the
black magic is for broadcast vs. non-broadcast, etc..
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CD1780B.2CC97DDB>