Date: Thu, 12 Sep 1996 12:17:16 -0600 From: Steve Passe <smp@csn.net> To: Peter Wemm <peter@spinner.dialix.com> Cc: freebsd-smp@freebsd.org Subject: Re: Intel XXpress - some SMP benchmarks Message-ID: <199609121817.MAA18639@clem.systemsix.com> In-Reply-To: Your message of "Thu, 12 Sep 1996 19:00:34 %2B0800." <199609121100.TAA03569@spinner.DIALix.COM>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
>BTW, Some other things we do not do.. We don't set the ERROR LVT to handle
>a non-delivered or failed message.
no, but my original code for the apic_startup() had checks on the APIC_ESR
register, never saw errors. I will put that back later today.
>I noticed you took out the second STARTUP IPI.. The docs I've been
>reading say "the startup IPI can only be used once after a reset or INIT",
> ...
>the second one is for insurance in case the first one was missed, and that
>the second will normally be ignored. I notice that there is no way in the
I guess the second one couldn't hurt, but I would rather use a better
means than "if I do it often enough its gotta work". A greater concern
is the INIT/RESET of the 'run bootMP' flavor that the XXPRESS demanded.
If we add the correct timings whats to prevent the STARTUP IPI from
re-running a CPU once it has already started (via RESET), perhaps
double incrementing mp_ncpus?
>system is probably missing the single startup IPI. Time do do some more
>accurate times I think.
I agree. I added a zero to each of the timing loops compared to the
original code. Realy need a usleep() of some sort.
>The pppbios.pdf specifically says: "The BSP sends a StartUp APIC message
>broadcast......" One of the various other tables in the P5 docs say that
>startup IPI broadcasts are always edge triggered when used in "all but
>self" mode, so who knows.. :-)
I see that now, I'm willing to believe it might be doable.
>Another thought.. We do not use the timer on the apic. It has a 32 bit
>read/write register for the "initial count". We could cheat and use that
>as a 32 bit pointer to a cpu-specific data page with each cpu's scratch
>area etc,
are you certain that we won't want to use it in the future?
another issue is that erich claims accessing the APIC registers is relatively
expensive time-wise. He pointed out that one should NOT write code like:
do_this( cpu_number() );
do_that( cpu_number() );
but instead:
id = cpu_number();
do_this( id );
do_that( id );
---
> better off putting in a check for an 82489 apic in the boot code and
both mine and Russel's boards show an IO APIC version of 17. His CPUs
show APIC vewrsion 16, while mine shows 17 (all numbers taken from the
MP table, NOT the APIC version registers). I can't find a # for the
82379AB anywhere in its manual.
>From the manuals:
82093AA: 11
82379AB: ??
82489DX: 01
P5: 1x
P6: ??
---
I got supped and "worlded" late last nite. Observations:
-
we seem to be out of sync with ps:
# ps
ps: proc size mismatch (13640 total, 632 chunks)
-
when I tried to halt the code realized it was running on CPU#2, claimed someing
about "freezing", then did! It was big red time. the mp_lock was declared
to be 01000001, so I guess the 2nd CPU froze while holding the lock.
-
making all in sys has problem with spl.h:
# make all
===> i386/boot
===> i386/boot/biosboot
cc -O2 -DDO_BAD144 -DBOOTWAIT=5000 -DTIMEOUT= -DCOMCONSOLE=0x3F8
-DBOOTSEG=0x1000 -DBOOTSTACK=0xFFF0 -c /usr/src/sys/i386/boot/biosboot/io.c
In file included from /usr/include/machine/cpufunc.h:425,
from /usr/src/sys/i386/boot/biosboot/io.c:31:
/usr/include/machine/spl.h:39: opt_smp.h: No such file or directory
*** Error code 1
copied opt_smp.h to i386/include:
# make all
===> i386/boot
===> i386/boot/biosboot
cc -O2 -DDO_BAD144 -DBOOTWAIT=5000 -DTIMEOUT= -DCOMCONSOLE=0x3F8
-DBOOTSEG=0x1000 -DBOOTSTACK=0xFFF0 -c /usr/src/sys/i386/boot/biosboot/io.c
In file included from /usr/include/machine/cpufunc.h:425,
from /usr/src/sys/i386/boot/biosboot/io.c:31:
/usr/include/machine/spl.h:39: opt_smp.h: No such file or directory
*** Error code 1
checked current state of i386/include:
# grep opt_smp *h
pcb.h:#include "opt_smp.h"
smp.h:#include "opt_smp.h"
spl.h:#include <opt_smp.h>
changing the line in spl.h to:
#include "opt_smp.h"
fixes that. still have a problem in that the makefiles for biosboot etc.
don't search the right path, ie opt_smp is only going to be in the kernel
compile specific directory, is it not? THis only worked because I copied
opt_smp.h to i386/include.
---
FYI, Heres my (GA586DX512) MP table:
--------------------------------------------------------------------------
MP Floating Pointer Structure:
location: BIOS
physical address: 0x000f0c80
signature: '_MP_'
length: 16 bytes
version: 1.1
checksum: 0xf4
mode: Virtual Wire
--------------------------------------------------------------------------
MP Config Table Header:
physical address: 0x000f0c94
signature: 'PCMP'
base table length: 292
version: 1.1
checksum: 0x31
OEM ID: 'OEM00000'
Product ID: 'PROD00000000'
OEM table pointer: 0x00000000
OEM table size: 0
entry count: 28
local APIC address: 0xfee00000
extended table length: 0
extended table checksum: 0
--------------------------------------------------------------------------
MP Config Base Table Entries:
--
Processor
apic ID: 0, version: 17
CPU is usable, CPU is the bootstrap processor
family: 5, model: 2, stepping: 1
feature flags: 0x000007bf
--
Processor
apic ID: 1, version: 17
CPU is usable, CPU is NOT the bootstrap processor
family: 5, model: 2, stepping: 1
feature flags: 0x000007bf
--
Bus
bus ID: 0, bus type: ISA
--
Bus
bus ID: 1, bus type: PCI
--
I/O APIC
apic ID: 2, version: 17
APIC is usable
apic address: 0xfec00000
--
I/O INT
INT type: 3, flags: 0x0000
source bus ID: 0, IRQ: 0
destination APIC ID: 2, INT: 0
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 1
destination APIC ID: 2, INT: 1
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 0
destination APIC ID: 2, INT: 2
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 3
destination APIC ID: 2, INT: 3
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 4
destination APIC ID: 2, INT: 4
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 5
destination APIC ID: 2, INT: 5
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 6
destination APIC ID: 2, INT: 6
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 7
destination APIC ID: 2, INT: 7
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 8
destination APIC ID: 2, INT: 8
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 9
destination APIC ID: 2, INT: 9
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 10
destination APIC ID: 2, INT: 10
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 11
destination APIC ID: 2, INT: 11
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 12
destination APIC ID: 2, INT: 12
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 13
destination APIC ID: 2, INT: 13
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 14
destination APIC ID: 2, INT: 14
--
I/O INT
INT type: 0, flags: 0x0000
source bus ID: 0, IRQ: 15
destination APIC ID: 2, INT: 15
--
I/O INT
INT type: 0, flags: 0x000f
source bus ID: 1, IRQ: 32
destination APIC ID: 2, INT: 16
--
I/O INT
INT type: 0, flags: 0x000f
source bus ID: 1, IRQ: 36
destination APIC ID: 2, INT: 17
--
I/O INT
INT type: 0, flags: 0x000f
source bus ID: 1, IRQ: 40
destination APIC ID: 2, INT: 18
--
I/O INT
INT type: 0, flags: 0x000f
source bus ID: 1, IRQ: 48
destination APIC ID: 2, INT: 19
--
I/O INT
INT type: 2, flags: 0x0000
source bus ID: 0, IRQ: 0
destination APIC ID: 2, INT: 23
--
Local INT
INT type: 3, flags: 0x0005
source bus ID: 0, IRQ: 0
destination APIC ID: 255, INT: 0
--
Local INT
INT type: 1, flags: 0x0005
source bus ID: 0, IRQ: 0
destination APIC ID: 255, INT: 1
--------------------------------------------------------------------------
Here's Russel's (Intel XXPRESS):
--------------------------------------------------------------------------
MP Floating Pointer Structure:
location: BIOS
physical address: 0x000f7ba0
signature: '_MP_'
length: 16 bytes
version: 1.4
checksum: 0x66
mode: Virtual Wire
--------------------------------------------------------------------------
MP Config Table Header:
physical address: 0x000f7bb0
signature: 'PCMP'
base table length: 268
version: 1.4
checksum: 0xdd
OEM ID: 'INTEL '
Product ID: 'XXPRESS '
OEM table pointer: 0x00000000
OEM table size: 0
entry count: 25
local APIC address: 0xfee00000
extended table length: 220
extended table checksum: 190
--------------------------------------------------------------------------
MP Config Base Table Entries:
--
Processor
apic ID: 0, version: 16
CPU is usable, CPU is the bootstrap processor
family: 5, model: 2, stepping: 11
feature flags: 0x000003bf
--
Processor
apic ID: 2, version: 16
CPU is usable, CPU is NOT the bootstrap processor
family: 5, model: 2, stepping: 11
feature flags: 0x000003bf
--
Bus
bus ID: 0, bus type: PCI
--
Bus
bus ID: 1, bus type: PCI
--
Bus
bus ID: 18, bus type: XPRESS
--
Bus
bus ID: 19, bus type: EISA
--
I/O APIC
apic ID: 14, version: 17
APIC is usable
apic address: 0xfec00000
--
I/O INT
INT type: 3, flags: 0x0005
source bus ID: 19, IRQ: 0
destination APIC ID: 14, INT: 0
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 1
destination APIC ID: 14, INT: 1
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 0
destination APIC ID: 14, INT: 2
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 3
destination APIC ID: 14, INT: 3
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 4
destination APIC ID: 14, INT: 4
--
I/O INT
INT type: 0, flags: 0x000c
source bus ID: 19, IRQ: 5
destination APIC ID: 14, INT: 5
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 6
destination APIC ID: 14, INT: 6
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 7
destination APIC ID: 14, INT: 7
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 8
destination APIC ID: 14, INT: 8
--
I/O INT
INT type: 0, flags: 0x000c
source bus ID: 19, IRQ: 9
destination APIC ID: 14, INT: 9
--
I/O INT
INT type: 0, flags: 0x000c
source bus ID: 19, IRQ: 10
destination APIC ID: 14, INT: 10
--
I/O INT
INT type: 0, flags: 0x000c
source bus ID: 19, IRQ: 11
destination APIC ID: 14, INT: 11
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 12
destination APIC ID: 14, INT: 12
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 13
destination APIC ID: 14, INT: 13
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 14
destination APIC ID: 14, INT: 14
--
I/O INT
INT type: 0, flags: 0x0005
source bus ID: 19, IRQ: 15
destination APIC ID: 14, INT: 15
--
Local INT
INT type: 3, flags: 0x0005
source bus ID: 19, IRQ: 0
destination APIC ID: 255, INT: 0
--
Local INT
INT type: 1, flags: 0x0005
source bus ID: 0, IRQ: 0
destination APIC ID: 255, INT: 1
--------------------------------------------------------------------------
MP Config Extended Table Entries:
--
bus ID: 0 address type: memory address
address base: 0xe8000
address range: 0x4000
--
bus ID: 0 address type: memory address
address base: 0xa0000
address range: 0x20000
--
bus ID: 1 address type: prefetch address
address base: 0xc0100000
address range: 0x100000
--
bus ID: 1 address type: memory address
address base: 0xc0000000
address range: 0x100000
--
bus ID: 1 address type: I/O address
address base: 0x7000
address range: 0x1000
--
bus ID: 0 address type: memory address
address base: 0x6000000
address range: 0xba000000
--
bus ID: 0 address type: memory address
address base: 0xc0200000
address range: 0x3fe00000
--
bus ID: 0 address type: I/O address
address base: 0x0
address range: 0x7000
--
bus ID: 0 address type: I/O address
address base: 0x8000
address range: 0x8000
--
bus ID: 19 bus info: 0x01 parent bus ID: 0--
bus ID: 0 address modifier: add
predefined range: 0x00000000--
bus ID: 0 address modifier: add
predefined range: 0x00000001--
bus ID: 1 address modifier: subtract
predefined range: 0x00000000--
bus ID: 1 address modifier: subtract
predefined range: 0x00000001
--------------------------------------------------------------------------
--
Steve Passe | powered by
smp@csn.net | FreeBSD
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: 2.6.2
mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE
04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX
WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR
tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+
=ds99
-----END PGP PUBLIC KEY BLOCK-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199609121817.MAA18639>