Date: Sun, 24 Oct 1999 04:11:56 -0500 From: Chris Csanady <cc@137.org> To: freebsd-smp@freebsd.org Cc: freebsd-hackers@freebsd.org Subject: IO APIC misdirecting irq (Re: SMP/interrupt problems..) Message-ID: <3812CD5C.1E990C73@ameslab.gov> References: <381159FF.D38E76FD@ameslab.gov> <38115BD9.6B8BE3BA@ameslab.gov>
next in thread | previous in thread | raw e-mail | index | archive | help
Sorry for the post to both lists--the original message probably belonged on
freebsd-smp. Anyways, please follow up there only.
I have tracked down the problem with the Megaraid controller in SMP, and it
seems it is due to an irq being misrouted. While the controller thinks it is
on irq 11, all the interrupts are actually going to apic irq 17. I have
confirmed
this by hardwiring amr0's interrupt to be irq 17--in which case it works
perfectly. :)
It is not clear to me whether the irq should or should not be routed to irq 17
though--only that it is what is happening with the current configuration. Does
anyone know in particular how this should work? (Or better yet, have a fix? ;)
It is kind of weird, because the amr0 controller is actually the same device
as the pci-pci bridge, just a different function. So, I assume it is just on
the host bus. In any case, even though the irq/int pin match in the mptable,
the device (slot) does not. :\
I have included my dmesg, pciconf output, and mptable output. The device in
question is the vendor=0x8086, dev=0x1960. Anyone have any ideas?
Chris
Copyright (c) 1992-1999 The FreeBSD Project.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
FreeBSD 4.0-CURRENT #14: Sat Oct 23 19:15:37 CDT 1999
ccsanady@friley-160-235.res.iastate.edu:/usr/src/sys/compile/EUROPA
Calibrating clock(s) ... TSC clock: 198655681 Hz, i8254 clock: 1193129 Hz
CLK_USE_I8254_CALIBRATION not specified - using default frequency
Timecounter "i8254" frequency 1193182 Hz
CLK_USE_TSC_CALIBRATION not specified - using old calibration method
CPU: Pentium Pro (198.67-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x619 Stepping = 9
Features=0xfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV>
real memory = 134217728 (131072K bytes)
Physical memory chunk(s):
0x00001000 - 0x0009efff, 647168 bytes (158 pages)
0x00322000 - 0x07ff7fff, 130899968 bytes (31958 pages)
avail memory = 126820352 (123848K bytes)
Programming 24 pins in IOAPIC #0
SMP: CPU0 apic_initialize():
lint0: 0x00000700 lint1: 0x00010400 TPR: 0x00000010 SVR: 0x000001ff
FreeBSD/SMP: Multiprocessor motherboard
cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfec08000
cpu1 (AP): apic id: 12, version: 0x00040011, at 0xfec08000
io0 (APIC): apic id: 13, version: 0x00170011, at 0xfec00000
bios32: Found BIOS32 Service Directory header at 0xc00fd970
bios32: Entry = 0xfd980 (c00fd980) Rev = 0 Len = 1
pcibios: PCI BIOS entry at 0xd9a1
pnpbios: Found PnP BIOS data at 0xc00fa170
pnpbios: Entry = f0000:a270 Rev = 1.0
Other BIOS signatures found:
ACPI: 00000000
Preloaded elf kernel "kernel" at 0xc0306000.
Pentium Pro MTRR support enabled
SMP: CPU0 bsp_apic_configure():
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000010 SVR: 0x000001ff
pci_open(1): mode 1 addr port (0x0cf8) is 0x80000058
pci_open(1a): mode1res=0x80000000 (0x80000000)
pci_cfgcheck: device 0 [class=060000] [hdr=00] is there (id=12378086)
npx0: <math processor> on motherboard
npx0: INT 16 interface
pci_open(1): mode 1 addr port (0x0cf8) is 0x00000000
pci_open(1a): mode1res=0x80000000 (0x80000000)
pci_cfgcheck: device 0 [class=060000] [hdr=00] is there (id=12378086)
pcib0: <Host to PCI bridge> on motherboard
found-> vendor=0x8086, dev=0x1237, revid=0x02
class=06-00-00, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
Freeing (NOT implemented) redirected PCI irq 9.
found-> vendor=0x8086, dev=0x1229, revid=0x02
class=02-00-00, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
intpin=a, irq=18
map[0]: type 1, range 32, base ffbae000, size 12
map[1]: type 1, range 32, base 0000ff40, size 5
map[2]: type 1, range 32, base fef00000, size 20
found-> vendor=0x8086, dev=0x7000, revid=0x01
class=06-01-00, hdrtype=0x00, mfdev=1
subordinatebus=0 secondarybus=0
found-> vendor=0x8086, dev=0x7010, revid=0x00
class=01-01-80, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
map[4]: type 1, range 32, base 0000ffa0, size 4
found-> vendor=0x8086, dev=0x7020, revid=0x01
class=0c-03-00, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
intpin=d, irq=9
map[4]: type 1, range 32, base 0000ff80, size 5
Freeing (NOT implemented) redirected PCI irq 11.
found-> vendor=0x9004, dev=0x8078, revid=0x00
class=01-00-00, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
intpin=a, irq=17
map[0]: type 1, range 32, base 0000f400, size 8
map[1]: type 1, range 32, base ffbaf000, size 12
Freeing (NOT implemented) redirected PCI irq 10.
found-> vendor=0x1000, dev=0x000f, revid=0x14
class=01-00-00, hdrtype=0x00, mfdev=1
subordinatebus=0 secondarybus=0
intpin=a, irq=16
map[0]: type 1, range 32, base 0000f800, size 8
map[1]: type 1, range 32, base ffbea800, size 8
map[2]: type 1, range 32, base ffbe8000, size 12
Freeing (NOT implemented) redirected PCI irq 10.
found-> vendor=0x1000, dev=0x000f, revid=0x14
class=01-00-00, hdrtype=0x00, mfdev=1
subordinatebus=0 secondarybus=0
intpin=a, irq=16
map[0]: type 1, range 32, base 0000fc00, size 8
map[1]: type 1, range 32, base ffbeac00, size 8
map[2]: type 1, range 32, base ffbe9000, size 12
found-> vendor=0x8086, dev=0x0960, revid=0x03
class=06-04-00, hdrtype=0x01, mfdev=1
subordinatebus=1 secondarybus=1
found-> vendor=0x8086, dev=0x1960, revid=0x03
class=01-00-00, hdrtype=0x00, mfdev=1
subordinatebus=0 secondarybus=0
intpin=a, irq=11
map[0]: type 1, range 32, base ffbd0000, size 16
Freeing (NOT implemented) redirected PCI irq 9.
found-> vendor=0x109e, dev=0x0350, revid=0x12
class=04-00-00, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
intpin=a, irq=18
map[0]: type 1, range 32, base ffbeb000, size 12
Freeing (NOT implemented) redirected PCI irq 9.
found-> vendor=0x102b, dev=0x0519, revid=0x01
class=03-00-00, hdrtype=0x00, mfdev=0
subordinatebus=0 secondarybus=0
intpin=a, irq=19
map[0]: type 1, range 32, base ffbec000, size 14
map[1]: type 1, range 32, base ff000000, size 23
pci0: <PCI bus> on pcib0
fxp0: <Intel EtherExpress Pro 10/100B Ethernet> irq 18 at device 6.0 on pci0
fxp0: Ethernet address 00:a0:c9:55:9e:4f
bpf: fxp0 attached
isab0: <Intel 82371SB PCI to ISA bridge> at device 7.0 on pci0
I/O Recovery Timing: 8-bit 1 clocks, 16-bit 1 clocks
Extended BIOS: enabled
Lower BIOS: enabled
Coprocessor IRQ13: enabled
Mouse IRQ12: disabled
Interrupt Routing: A: IRQ10, B: IRQ11, C: IRQ9, D: IRQ9
MB0: , MB1:
Trying Read_Port at 203
CSC0000: start dependant
CSC0000: adding dma mask 0x2
CSC0000: adding dma mask 0x9
CSC0000: adding irq mask 0x20
CSC0000: adding io range 0x534-0x537, size=0x4, align=0x4
CSC0000: adding io range 0x388-0x38b, size=0x4, align=0x8
CSC0000: adding io range 0x220-0x22f, size=0x10, align=0x20
CSC0000: start dependant
CSC0000: adding dma mask 0xa
CSC0000: adding dma mask 0xb
CSC0000: adding irq mask 0x9aa0
CSC0000: adding io range 0x534-0xfff, size=0x4, align=0x4
CSC0000: adding io range 0x388-0x38b, size=0x4, align=0x8
CSC0000: adding io range 0x220-0x26f, size=0x10, align=0x20
CSC0000: start dependant
CSC0000: adding dma mask 0xb
CSC0000: adding irq mask 0x9aa0
CSC0000: adding io range 0x534-0xfff, size=0x4, align=0x4
CSC0000: adding io range 0x388-0x3fb, size=0x4, align=0x8
CSC0000: adding io range 0x220-0x30f, size=0x10, align=0x20
CSC0000: end dependant
CSC0001: start dependant
CSC0001: adding io range 0x200-0x207, size=0x8, align=0x8
CSC0001: start dependant
CSC0001: adding io range 0x208-0x20f, size=0x8, align=0x8
CSC0001: end dependant
CSC0010: adding io range 0x120-0xfff, size=0x8, align=0x8
CSC0003: start dependant
CSC0003: adding irq mask 0x200
CSC0003: adding io range 0x330-0x331, size=0x2, align=0x8
CSC0003: start dependant
CSC0003: adding irq mask 0x9a00
CSC0003: adding io range 0x300-0x3f9, size=0x2, align=0x8
CSC0003: end dependant
isa0: <ISA bus> on isab0
ata-pci0: <Intel PIIX3 IDE controller> at device 7.1 on pci0
ata-pci0: Busmastering DMA supported
ata0: iobase=0x01f0 altiobase=0x03f6
ata0: mask=03 status0=50 status1=00
ata0: mask=03 status0=50 status1=00
ata0: devices = 0x1
ata0 at 0x01f0 irq 14 on ata-pci0
ata1: iobase=0x0170 altiobase=0x0376
ata1: mask=00 status0=ff status1=ff
chip1: <Intel 82371SB (PIIX3) USB controller> irq 9 at device 7.2 on pci0
ahc0: <Adaptec aic7880 Ultra SCSI adapter> irq 17 at device 9.0 on pci0
ahc0: Reading SEEPROM...done.
ahc0: Low byte termination Enabled
ahc0: High byte termination Enabled
ahc0: aic7880 Wide Channel A, SCSI Id=7, 16/255 SCBs
ahc0: Downloading Sequencer Program... 411 instructions downloaded
ncr0: <ncr 53c875 fast20 wide scsi> irq 16 at device 11.0 on pci0
ncr0: minsync=12, maxsync=137, maxoffs=16, 128 dwords burst, large dma fifo
ncr0: single-ended, open drain IRQ driver, using on-chip SRAM
ncr1: <ncr 53c875 fast20 wide scsi> irq 16 at device 11.1 on pci0
ncr1: minsync=12, maxsync=137, maxoffs=16, 128 dwords burst, large dma fifo
ncr1: single-ended, open drain IRQ driver, using on-chip SRAM
using shared irq16.
pcib1: <PCI to PCI bridge (vendor=8086 device=0960)> at device 15.0 on pci0
pci1: <PCI bus> on pcib1
amr0: <AMI MegaRAID> irq 11 at device 15.1 on pci0
amr0: firmware GH6B bios 1.41 32MB memory, chipset 61
amrd0: <MegaRAID logical drive> on amr0
amrd0: 1553MB (3180544 sectors), state 0x2 properties 0x1
Creating DISK amrd0
amrd1: <MegaRAID logical drive> on amr0
amrd1: 3072MB (6291456 sectors), state 0x2 properties 0x0
Creating DISK amrd1
bktr0: <BrookTree 848A> irq 18 at device 17.0 on pci0
using shared irq18.
iicbb0: <I2C generic bit-banging driver> on bti2c0
iicbus0: <Philips I2C bus> on iicbb0 master-only
iicbus: iic devclass not found
smbus0: <System Management Bus> on bti2c0
smbus: smb devclass not found
brooktree0: PCI bus latency is 136.
bktr0: buffer size 3555328, addr 0x5000000
bktr: GPIO is 0x003ffffb
bktr0: Hauppauge Model 56131 E
Hauppauge WinCast/TV, Philips FR1236 NTSC FM tuner, dbx stereo.
vga-pci0: <Matrox MGA 2064W graphics accelerator> irq 19 at device 19.0 on pci0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <keyboard controller (i8042)> at port 0x60-0x6f on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
atkbd: the current kbd controller command byte 0065
atkbd: keyboard ID 0x41ab (2)
kbdc: RESET_KBD return code:00fa
kbdc: RESET_KBD status:00aa
kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000
psm0: current command byte:0065
kbdc: TEST_AUX_PORT status:0000
kbdc: RESET_AUX return code:00fa
kbdc: RESET_AUX status:00aa
kbdc: RESET_AUX ID:0000
psm: status 00 02 64
psm: status b1 03 c8
psm: status b1 03 c8
psm: status b1 03 c8
psm: status 00 00 3c
psm: data 08 00 00
psm: status 00 02 64
psm: data 08 00 00
psm: status 00 02 64
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model Generic PS/2 mouse, device ID 0-00, 3 buttons
psm0: config:00000000, flags:00000000, packet size:3
psm0: syncmask:c0, syncbits:00
vga0: <Generic ISA VGA> at port 0x3b0-0x3df iomem 0xa0000-0xbffff on isa0
fb0: vga0, vga, type:VGA (5), flags:0x7007f
fb0: port:0x3b0-0x3df, crtc:0x3d4, mem:0xa0000 0x20000
fb0: init mode:24, bios mode:3, current mode:24
fb0: window:0xc00b8000 size:32k gran:32k, buf:0 size:32k
VGA parameters upon power-up
50 18 10 00 00 00 03 00 02 67 60 4f 50 83 55 81
bf 1f 00 4f 0d 0e 00 00 06 40 9c 8e 8f 28 1f 96
b9 a3 ff 00 01 02 03 04 05 14 07 38 39 3a 3b 3c
3d 3e 3f 0c 00 0f 08 00 00 00 00 00 10 0e 00 ff
VGA parameters in BIOS for mode 24
50 18 10 00 10 00 03 00 02 67 60 4f 50 83 55 81
bf 1f 00 4f 0d 0e 00 00 00 00 9c 8e 8f 28 1f 96
b9 a3 ff 00 01 02 03 04 05 14 07 38 39 3a 3b 3c
3d 3e 3f 0c 00 0f 08 00 00 00 00 00 10 0e 00 ff
EGA/VGA parameters to be used for mode 24
50 18 10 00 10 00 03 00 02 67 60 4f 50 83 55 81
bf 1f 00 4f 0d 0e 00 00 00 00 9c 8e 8f 28 1f 96
b9 a3 ff 00 01 02 03 04 05 14 07 38 39 3a 3b 3c
3d 3e 3f 0c 00 0f 08 00 00 00 00 00 10 0e 00 ff
sc0: <System console> on isa0
sc0: VGA <16 virtual consoles, flags=0x200>
sc0: fb0 kbd0
sio0: irq maps: 0x8063 0x8073 0x8063 0x8063
sio0 at port 0x3f8-0x3ff irq 4 on isa0
sio0: type 16550A
sio1: irq maps: 0x8063 0x806b 0x8063 0x8063
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
pcm0: <CS4236B> at port 0x534-0x537,0x388-0x38b,0x220-0x22f irq 5 drq 1,0 on
isa0
pcm: setmap 40000, ff00; 0xc88d3000 -> 40000
pcm: setmap 50000, ff00; 0xc88e3000 -> 50000
unknown0: <GAME> at port 0x200-0x207 on isa0
unknown1: <CTRL> at port 0x120-0x127 on isa0
unknown2: <MPU> at port 0x330-0x331 irq 9 on isa0
SMP: enabled INTs: 1, 3, 4, 5, 6, 11, 12, 14, 16, 17, 18, apic_imen: 0x00f8a785
BIOS Geometries:
0:03fefe3f 0..1022=1023 cylinders, 0..254=255 heads, 1..63=63 sectors
1:00c4fe3f 0..196=197 cylinders, 0..254=255 heads, 1..63=63 sectors
2:0186fe3f 0..390=391 cylinders, 0..254=255 heads, 1..63=63 sectors
0 accounted for
Device configuration finished.
APIC_IO: routing 8254 via 8259 on pin 0
bpf: lo0 attached
Linux-ELF exec handler installed
SMP: AP CPU #1 Launched!
SMP: CPU1 apic_initialize():
lint0: 0x00010700 lint1: 0x00010400 TPR: 0x00000010 SVR: 0x000001ff
ata0: master: success setting up WDMA2 mode on PIIX4 chip
ad0: piomode=4 dmamode=2 udmamode=2
ad0: <IBM-DTTA-351010/T56OA73A> ATA-4 disk at ata0 as master
ad0: 9671MB (19807200 sectors), 19650 cyls, 16 heads, 63 S/T, 512 B/S
ad0: 16 secs/int, 31 depth queue, DMA
Creating DISK ad0
Creating DISK wd0
Waiting 2 seconds for SCSI devices to settle
[scsi device probing deleted..]
chip0@pci0:0:0: class=0x060000 card=0x00000000 chip=0x12378086 rev=0x02 hdr=0x00
fxp0@pci0:6:0: class=0x020000 card=0x00000000 chip=0x12298086 rev=0x02 hdr=0x00
isab0@pci0:7:0: class=0x060100 card=0x00000000 chip=0x70008086 rev=0x01 hdr=0x00
ata-pci0@pci0:7:1: class=0x010180 card=0x00000000 chip=0x70108086 rev=0x00
hdr=0x00
chip1@pci0:7:2: class=0x0c0300 card=0x00000000 chip=0x70208086 rev=0x01 hdr=0x00
ahc0@pci0:9:0: class=0x010000 card=0x00000000 chip=0x80789004 rev=0x00 hdr=0x00
ncr0@pci0:11:0: class=0x010000 card=0x87601092 chip=0x000f1000 rev=0x14 hdr=0x00
ncr1@pci0:11:1: class=0x010000 card=0x87601092 chip=0x000f1000 rev=0x14 hdr=0x00
pcib1@pci0:15:0: class=0x060400 card=0x00000000 chip=0x09608086 rev=0x03
hdr=0x01
amr0@pci0:15:1: class=0x010000 card=0x0466101e chip=0x19608086 rev=0x03 hdr=0x00
bktr0@pci0:17:0: class=0x040000 card=0x00000000 chip=0x0350109e rev=0x12
hdr=0x00
vga-pci0@pci0:19:0: class=0x030000 card=0x00000000 chip=0x0519102b rev=0x01
hdr=0x00
===============================================================================
MPTable, version 2.0.15
-------------------------------------------------------------------------------
MP Floating Pointer Structure:
location: BIOS
physical address: 0x000f80f0
signature: '_MP_'
length: 16 bytes
version: 1.4
checksum: 0x10
mode: Virtual Wire
-------------------------------------------------------------------------------
MP Config Table Header:
physical address: 0x000f8100
signature: 'PCMP'
base table length: 260
version: 1.4
checksum: 0x30
OEM ID: 'INTEL '
Product ID: 'PR440FX '
OEM table pointer: 0x00000000
OEM table size: 0
entry count: 24
local APIC address: 0xfec08000
extended table length: 120
extended table checksum: 15
-------------------------------------------------------------------------------
MP Config Base Table Entries:
--
Processors: APIC ID Version State Family Model Step Flags
0 0x11 BSP, usable 6 1 9 0xfbff
12 0x11 AP, usable 6 1 9 0xfbff
--
Bus: Bus ID Type
0 PCI
1 PCI
18 ISA
--
I/O APICs: APIC ID Version State Address
13 0x11 usable 0xfec00000
--
I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN#
ExtINT active-hi edge 18 0 13 0
INT active-hi edge 18 1 13 1
INT active-hi edge 18 3 13 3
INT active-hi edge 18 4 13 4
INT active-hi edge 18 5 13 5
INT active-hi edge 18 6 13 6
INT active-hi edge 18 7 13 7
INT active-hi edge 18 8 13 8
INT active-hi edge 18 12 13 12
INT active-hi edge 18 14 13 14
INT active-hi edge 18 15 13 15
INT active-lo level 0 19:A 13 19
INT active-lo level 0 17:A 13 18
INT active-lo level 0 11:A 13 16
INT active-lo level 0 9:A 13 17
INT active-lo level 0 6:A 13 18
--
Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN#
ExtINT active-hi edge 18 0 255 0
NMI active-hi edge 0 0:A 255 1
-------------------------------------------------------------------------------
MP Config Extended Table Entries:
--
bus ID: 0 address type: memory address
address base: 0xd4000
address range: 0x4000
--
bus ID: 0 address type: memory address
address base: 0xd8000
address range: 0x4000
--
bus ID: 0 address type: memory address
address base: 0xdc000
address range: 0x4000
--
bus ID: 0 address type: memory address
address base: 0xa0000
address range: 0x20000
--
bus ID: 0 address type: memory address
address base: 0x8000000
address range: 0xf8000000
--
bus ID: 0 address type: I/O address
address base: 0x0
address range: 0x10000
-------------------------------------------------------------------------------
# SMP kernel config file options:
# Required:
options SMP # Symmetric MultiProcessor Kernel
options APIC_IO # Symmetric (APIC) I/O
# Optional (built-in defaults will work in most cases):
#options NCPU=2 # number of CPUs
#options NBUS=3 # number of busses
#options NAPIC=1 # number of IO APICs
#options NINTR=24 # number of INTs
===============================================================================
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3812CD5C.1E990C73>