Date: Thu, 12 Sep 1996 14:04:27 -0600 From: Steve Passe <smp@csn.net> To: Peter Wemm <peter@spinner.dialix.com> Cc: rv@groa.uct.ac.za (Russell Vincent), freebsd-smp@freebsd.org, Terry Lambert <terry@lambert.org> Subject: Re: Intel XXpress - some SMP benchmarks Message-ID: <199609122004.OAA19225@clem.systemsix.com> In-Reply-To: Your message of "Thu, 12 Sep 1996 12:17:16 MDT." <199609121817.MAA18639@clem.systemsix.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
>Hmm. I'd rather avoid an extra indirection map if possible. Considering
>that we are #define'ing things like curproc and some other heavily used
>variables to lookup the cpuid for array indexes, we would better spend the
>effort to just deal with a sparse ID set properly.
further thoughts on dense vs sparse ID mapping:
the simple indirection to get an int from a table of 16 ints is pretty cheap.
I suggest cleanup of any areas of code that repeatedly call macros like
curproc. I did a quick grep in /kern and found a few offenders like:
curproc->p_flag |= P_PHYSIO;
(I wonder if gcc will optimize this to only 1 APIC read?) Most of the
code does a 1 time assignment to a struct ptr via curproc and uses that.
if we have sparse tables, ie 16 entries (most of which will be empty), code
like the following becomes very inefficient:
#ifdef SMP
for (j = i = 0; i < SPARSE_TABLE_SIZE; i++) {
if (p == SMPcurproc[i])
j++;
}
...
---
of course all the above is assumming my current belief that we can't get away
with reprogramming the APIC IDs to a dense sequence. I know that the APICs
initially get their IDs from the state of 4 pins during hardware (motherboard)
RESET. Whether there are problems when the ID you reprogam to are different
I haven't a clue...
---
2nd STARTUP
pentium proc. vol 1, 20.1.1.4, par 5:
"It is the responsibility of the system software to resend the STARTUP IPI
if there is an error... [enable APIC, use LVT3 ERROR vector to check it ]
Otherwise the system software would have to poll the delivery status bit ...
to determine if IPI is pending ... and resend STARTUP IPI if the IPI
remains pending after an appropiate amount of time."
since we poll this bit I think that (logically anyways) we should be safe
only sending the one STARTUP IPI. From the XXPRESS port we know that if
there isn't an APIC sitting there @ that ID the loop hangs forever.
If we go to a broadcast scheme then polling may or may not work.
I could see it working like a "wired-or", ie stays pending till all CPUs
have received it, OR screwing up totally.
Does anyone out there reading this list have a board with more than 2 CPUs?
We might want to run some tests before we paint ourselves into either corner.
---
# of IDs:
pentium proc. vol 3, 19.3.1.5, 'Physical Dest Mode':
"... A single destination (ID = 0 thru 14) or a broadcast to all (ID = 15)
can be specified in the physical mode. ... Note that in this mode, the Pentium
Processor ... APIC supports up to 15 agents."
So this is probably why the XXPRESS IDs its IO APIC #14, ie at the top end.
The board has a 2nd IO APIC for the 2nd PIC bus (we never looked at the MP
table while it was enabled) which probably would get ID #13. Ie., I am
guessing they start IDing IO APICs at the top and work down. Why they
skip ID #1 for CPUs I have not a clue!!!
Terry earlier wondered how the Sequent supported 32 CPUs. There is a more
complex 'heirarxchial model' that supports up to 60 APICs. This model
requires additional custom hardware support. I don't think I wanna go there.
---
I'm going to cleanup mptable.c next, and will post it in an hour or 2. It
is the user level program that we used to dump the MP tables I published
earlier today.
--
Steve Passe | powered by
smp@csn.net | FreeBSD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199609122004.OAA19225>