Date: Thu, 23 May 2002 11:11:23 +0200 From: Holger Kipp <holger.kipp@alogis.com> To: freebsd-smp@FreeBSD.ORG Cc: Pete French <pfrench@firstcallgroup.co.uk>, frank@exit.com, maildrop@qwest.net, stable@FreeBSD.ORG Subject: Re: 4.6-RC system hangs (fxp0, smp, sym) (UPDATE) Message-ID: <3CECB23B.5DD164A1@alogis.com> References: <3CEA128A.AB7FAC93@alogis.com> <E17A6HG-000GVa-00@mailhost.firstcallgroup.co.uk> <15594.22920.415872.835007@moe.cs.duke.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
I did some more testing, and finally found something interesting:
I had IO-APIC disabled. All errors went away after I enabled IO-APIC
in the BIOS. Now for the guesswork:
The only real difference I can see here is the assignment of IRQs to
controllers (SCSI and NIC). See below. I only have one SMP machine
with said problems to test this.
If you encounter similar problems, could you please check if your
configuration fits this pattern? If it doesn't, I'm out of ideas...
dmesg and mptable for the working setup below.
Regards,
Holger
--- 8< ---------------------------------------------------------------------
==> I noticed that with IO-APIC, sym0, sym1 and fxp0 have different IRQs
assigned (all errors gone), whilst with the old setup, they all have the
same IRQ.
errors no errors
sym0 IRQ 11 at 13.0 IRQ 2 at 13.0
sym1 IRQ 11 at 13.1 IRQ 11 at 13.1
fxp0 IRQ 11 at 15.0 IRQ 16 at 15.0
==> Could this explain the problems I (and others) see?
- I have all disks on sym0, but maybe copying from sym0 to sym1 could
give errors if they share the same IRQ (if timing is critical)?
- have only one NIC, but with several NICs on the same IRQ, could
this lead to problems as seen by others?
- maybe new NIC-Problems actually are the result of optimized drivers
(changed timing)?
- is difference between IRQ-handling NCR <-> SYM sufficient to explain
the behaviour differences between those two I experience?
dmesg.sym.apic.txt
--- 8< ---------------------------------------------------------------------
Copyright (c) 1992-2002 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.6-RC #0: Wed May 22 19:56:42 CEST 2002
root@idefix.intern.hkipp.de:/usr/obj/usr/src/sys/IDEFIXSYM
Timecounter "i8254" frequency 1193182 Hz
CPU: Pentium II/Pentium II Xeon/Celeron (349.13-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x652 Stepping = 2
Features=0x183fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR>
real memory = 134217728 (131072K bytes)
avail memory = 127340544 (124356K bytes)
Programming 24 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
FreeBSD/SMP: Multiprocessor motherboard
cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000
cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000
io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000
Preloaded elf kernel "kernel" at 0xc033b000.
Pentium Pro MTRR support enabled
md0: Malloc disk
Using $PIR table, 8 entries at 0xc00fdf50
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Intel 82443BX host to PCI bridge (AGP disabled)> on motherboard
IOAPIC #0 intpin 21 -> irq 2
IOAPIC #0 intpin 22 -> irq 11
IOAPIC #0 intpin 20 -> irq 16
pci0: <PCI bus> on pcib0
sym0: <875> port 0xf800-0xf8ff mem 0xfedff000-0xfedfffff,0xfedfec00-0xfedfecff irq 2 at device 13.0 on pci0
sym0: No NVRAM, ID 7, Fast-20, SE, parity checking
sym1: <875> port 0xf400-0xf4ff mem 0xfedfd000-0xfedfdfff,0xfedfe800-0xfedfe8ff irq 11 at device 13.1 on pci0
sym1: No NVRAM, ID 7, Fast-20, SE, parity checking
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xfce0-0xfcff mem 0xfeb00000-0xfebfffff,0xfedfa000-0xfedfafff irq 16 at device 15.0 on pci0
fxp0: Ethernet address 00:a0:c9:ec:70:26
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
isab0: <Intel 82371AB PCI to ISA bridge> at device 18.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 ATA33 controller> port 0xfcb0-0xfcbf at device 18.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
pci0: <Intel 82371AB/EB (PIIX4) USB controller> at 18.2 irq 11
Timecounter "PIIX" frequency 3579545 Hz
chip1: <Intel 82371AB Power management controller> port 0x2180-0x218f at device 18.3 on pci0
pci0: <Cirrus Logic GD5480 SVGA controller> at 20.0
isa0: too many dependant configs (8)
orm0: <Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xcb7ff,0xcb800-0xcbfff on isa0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model Generic PS/2 mouse, device ID 0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1: configured irq 3 not in bitmap of probed irqs 0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
APIC_IO: Testing 8254 interrupt delivery
APIC_IO: routing 8254 via IOAPIC #0 intpin 2
Waiting 3 seconds for SCSI devices to settle
SMP: AP CPU #1 Launched!
(probe4:sym0:0:4:0): phase change 6-7 6@01a7518c resid=4.
sym0: unexpected disconnect
pass2 at sym0 bus 0 target 4 lun 0
pass2: <TOSHIBA SMC2.0 A08> Fixed Processor SCSI-2 device
pass2: 3.300MB/s transfers
Mounting root from ufs:/dev/da0s1a
cd0 at sym1 bus 0 target 6 lun 0
cd0: <TOSHIBA CD-ROM XM-6201TA 1037> Removable CD-ROM SCSI-2 device
cd0: 10.000MB/s transfers (10.000MHz, offset 16)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da0 at sym0 bus 0 target 0 lun 0
da0: <IBM DDRS-39130W S97B> Fixed Direct Access SCSI-2 device
da0: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da0: 8715MB (17850000 512 byte sectors: 255H 63S/T 1111C)
da1 at sym0 bus 0 target 1 lun 0
da1: <IBM DDRS-39130W S97B> Fixed Direct Access SCSI-2 device
da1: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da1: 8707MB (17833616 512 byte sectors: 255H 63S/T 1110C)
uhci0: <Intel 82371AB/EB (PIIX4) USB controller> port 0xfcc0-0xfcdf irq 11 at device 18.2 on pci0
usb0: <Intel 82371AB/EB (PIIX4) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
mptable.apic.txt
--- 8< ---------------------------------------------------------------------
===============================================================================
MPTable, version 2.0.15
-------------------------------------------------------------------------------
MP Floating Pointer Structure:
location: BIOS
physical address: 0x000f76d0
signature: '_MP_'
length: 16 bytes
version: 1.4
checksum: 0x46
mode: Virtual Wire
-------------------------------------------------------------------------------
MP Config Table Header:
physical address: 0x0009f160
signature: 'PCMP'
base table length: 276
version: 1.4
checksum: 0x06
OEM ID: 'INTEL '
Product ID: 'Nightshade '
OEM table pointer: 0x00000000
OEM table size: 0
entry count: 26
local APIC address: 0xfee00000
extended table length: 144
extended table checksum: 192
-------------------------------------------------------------------------------
MP Config Base Table Entries:
--
Processors: APIC ID Version State Family Model Step Flags
1 0x11 BSP, usable 6 5 2 0x183fbff
0 0x11 AP, usable 6 5 2 0x183fbff
--
Bus: Bus ID Type
0 PCI
1 ISA
--
I/O APICs: APIC ID Version State Address
2 0x11 usable 0xfec00000
--
I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN#
ExtINT active-hi edge 1 0 2 0
INT active-hi edge 1 1 2 1
INT active-hi edge 1 0 2 2
INT active-hi edge 1 3 2 3
INT active-hi edge 1 4 2 4
INT active-hi edge 1 5 2 5
INT active-hi edge 1 6 2 6
INT active-hi edge 1 7 2 7
INT active-hi edge 1 8 2 8
INT active-hi edge 1 9 2 9
INT active-hi edge 1 10 2 10
INT active-hi edge 1 12 2 12
INT active-hi edge 1 13 2 13
INT active-hi edge 1 14 2 14
INT active-hi edge 1 15 2 15
INT active-lo level 0 13:A 2 21
INT active-lo level 0 13:B 2 22
INT active-lo level 0 15:A 2 20
INT active-lo level 0 18:D 2 22
--
Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN#
ExtINT active-hi edge 1 0 255 0
NMI active-hi edge 0 0:A 255 1
-------------------------------------------------------------------------------
MP Config Extended Table Entries:
--
System Address Space
bus ID: 0 address type: I/O address
address base: 0x0
address range: 0x10000
--
System Address Space
bus ID: 0 address type: memory address
address base: 0x8000000
address range: 0xf4000000
--
System Address Space
bus ID: 0 address type: prefetch address
address base: 0xfc000000
address range: 0x2dfb000
--
System Address Space
bus ID: 0 address type: memory address
address base: 0xfedfb000
address range: 0x1205000
--
System Address Space
bus ID: 0 address type: memory address
address base: 0xa0000
address range: 0x20000
--
System Address Space
bus ID: 0 address type: memory address
address base: 0xd4000
address range: 0x14000
--
Bus Heirarchy
bus ID: 1 bus info: 0x01 parent bus ID: 0
--
Compatibility Bus Address
bus ID: 0 address modifier: add
predefined range: 0x00000000
--
Compatibility Bus Address
bus ID: 0 address modifier: add
predefined range: 0x00000001
===============================================================================
--
Holger Kipp, Dipl.-Math., Systemadministrator | alogis AG
Fon: +49 (0)30 / 43 65 8 - 114 | Berliner Strasse 26
Fax: +49 (0)30 / 43 65 8 - 214 | D-13507 Berlin Tegel
email: holger.kipp@alogis.com | http://www.alogis.com
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CECB23B.5DD164A1>