From owner-freebsd-smp Thu May 16 4:40:34 2002 Delivered-To: freebsd-smp@freebsd.org Received: from klima.physik.uni-mainz.de (klima.Physik.Uni-Mainz.DE [134.93.180.162]) by hub.freebsd.org (Postfix) with ESMTP id 4624037B40C for ; Thu, 16 May 2002 04:40:06 -0700 (PDT) Received: from klima.Physik.Uni-Mainz.DE (administrator@klima.Physik.Uni-Mainz.DE [134.93.180.162]) by klima.physik.uni-mainz.de (8.12.3/8.12.2) with ESMTP id g4GBe4b0017940 for ; Thu, 16 May 2002 13:40:05 +0200 (CEST) (envelope-from administrator@klima.physik.uni-mainz.de) Date: Thu, 16 May 2002 13:40:04 +0200 (CEST) From: Administrator IPA To: freebsd-smp@freebsd.org Subject: RE: SMP problems with Fujitsu Siemens Primergy P200 (fwd) Message-ID: <20020516133347.W17595-100000@klima.physik.uni-mainz.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Dear Sirs. On older SMP main PCBs I experienced a similar behaviour. The only solution was to disable a second or third NIC (I have in one machine two Intel fxp0 and fxp1 NICs). The second I did was changing several cards in PCI slots. Our problem occured when the firmware of our AMI Enterprise 1600 RAID controler was updated. It looked like an IRQ routing problem, I think this is a very often reported phenomenon on high end server boards and FreeBSD. After permutating the additional NIC on the 33MHz PCI slots, disabling all ACPI stuff (in BIOS and in kernel) and moving the RAID controller to its appropriate 66MHz PCI slot the obscure freezing of the system has been banned. I must add to this that we didn't activate the built in NIC anymore, so maybe the problem will come back if this nick gets reanimated again ... -- MfG O. Hartmann ohartman@klima.physik.uni-mainz.de ---------------------------------------------------------------- IT-Administration des Institut fuer Physik der Atmosphaere (IPA) ---------------------------------------------------------------- Johannes Gutenberg Universitaet Mainz Becherweg 21 55099 Mainz Tel: +496131/3924662 (Maschinensaal) Tel: +496131/3924144 FAX: +496131/3923532 ---------- Forwarded message ---------- Date: Mon, 13 May 2002 13:52:48 +0200 From: Simon L. Nielsen To: freebsd-stable@FreeBSD.ORG Subject: SMP problems with Fujitsu Siemens Primergy P200 Hello I'm having some problems using SMP on a Fujitsu Siemens Primergy P200 with FreeBSD 4.5-RELEASE-p4. It's a dual Intel Pentium 3 1266MHz with 1GB ECC RAM. It has a Mylex AcceleRAID 352 controller connected to 4 SCSI disks, and a normal ATAPI CD-ROM drive. It also has a on-board Symbios SCSI controller but this is not being used at present. When booting a SMP kernel it simply hangs after initializing the second CPU. The computer is not completely frozen since I can break to the kernel debugger, but it never gets any further. However, I don't really have enough information about the internals of the kernel to do anything useful in the kernel debugger... The computer works fine on a single CPU kernel. I have also tried to boot on another computer with the exact same hardware and still there is no positive result, so it would not appear to be "normal" defective hardware. I have tried booting a 4.6-PRERELEASE (from today) and 5.0DP1 without success. The 4.6 fails in the same way as 4.5, but it does detect the CD-ROM drive after it has written "AP CPU #1 Launched" (I have not included the output since it is almost the same and rather large). The 5.0DP1 just does a kernel after detecting the Mylex controller but I don't think that is really related to the main problem. I have included output from from a verbose boot and output from mptable. Any hints on what might be wrong or suggestions how to fix the problem would be very much appreciated. Verbose boot message from 4.5-RELEASE-p4 : SMAP type=01 base=00000000 00000000 len=00000000 0009e400 SMAP type=02 base=00000000 0009e400 len=00000000 00001c00 SMAP type=02 base=00000000 000ca000 len=00000000 00002000 SMAP type=02 base=00000000 000e0000 len=00000000 00020000 SMAP type=01 base=00000000 00100000 len=00000000 3fdf0000 SMAP type=03 base=00000000 3fef0000 len=00000000 0000f000 SMAP type=04 base=00000000 3feff000 len=00000000 00001000 SMAP type=01 base=00000000 3ff00000 len=00000000 00100000 SMAP type=02 base=00000000 fec00000 len=00000000 00010000 SMAP type=02 base=00000000 fee00000 len=00000000 00001000 SMAP type=02 base=00000000 ffc00000 len=00000000 00400000 Copyright (c) 1992-2002 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.5-RELEASE-p4 #2: Fri May 10 23:01:17 GST 2002 root@hd1.test.siemens.com:/usr/src/sys/compile/DEM-HARVESTER2 Calibrating clock(s) ... TSC clock: 1260490417 Hz, i8254 clock: 1193080 Hz CLK_USE_I8254_CALIBRATION not specified - using default frequency Timecounter "i8254" frequency 1193182 Hz CLK_USE_TSC_CALIBRATION not specified - using old calibration method CPU: Pentium III/Pentium III Xeon/Celeron (1260.61-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6b1 Stepping = 1 Features=0x383fbff real memory = 1073741824 (1048576K bytes) Physical memory chunk(s): 0x00001000 - 0x0009dfff, 643072 bytes (157 pages) 0x004db000 - 0x3feeffff, 1067536384 bytes (260629 pages) 0x3ff00000 - 0x3fff7fff, 1015808 bytes (248 pages) avail memory = 1040273408 (1015892K bytes) Programming 16 pins in IOAPIC #0 Programming 16 pins in IOAPIC #1 SMP: CPU0 apic_initialize(): lint0: 0x00000700 lint1: 0x00010400 TPR: 0x00000010 SVR: 0x000001ff FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 3, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 1, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 2, version: 0x000f0011, at 0xfec01000 bios32: Found BIOS32 Service Directory header at 0xc00f6ba0 bios32: Entry = 0xfd890 (c00fd890) Rev = 0 Len = 1 pcibios: PCI BIOS entry at 0x11a pnpbios: Found PnP BIOS data at 0xc00f6c30 pnpbios: Entry = f0000:9dd0 Rev = 1.0 Other BIOS signatures found: ACPI: 000f6c00 Preloaded elf kernel "kernel.old" at 0xc04b1000. Pentium Pro MTRR support enabled md0: Malloc disk Creating DISK md0 Math emulator present SMP: CPU0 bsp_apic_configure(): lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000010 SVR: 0x000001ff pci_open(1): mode 1 addr port (0x0cf8) is 0x80000270 pci_open(1a): mode1res=0x80000000 (0x80000000) pci_cfgcheck: device 0 [class=060000] [hdr=80] is there (id=00081166) Using $PIR table, 15 entries at 0xc00fded0 pcib-: pcib1 exists, using next available unit number pcib-: pcib2 exists, using next available unit number npx0: on motherboard npx0: INT 16 interface pcib1: on motherboard IOAPIC #1 intpin 13 -> irq 5 Freeing (NOT implemented) redirected PCI irq 10. found-> vendor=0x1000, dev=0x0021, revid=0x01 class=01-00-00, hdrtype=0x00, mfdev=0 subordinatebus=0 secondarybus=0 intpin=a, irq=5 pci1: on pcib1 sym0: <1010-66> irq 5 at device 10.0 on pci1 sym0: failed to allocate MMIO resources device_probe_and_attach: sym0 attach returned 6 pcib0: on motherboard found-> vendor=0x1166, dev=0x0008, revid=0x23 class=06-00-00, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 found-> vendor=0x1166, dev=0x0008, revid=0x01 class=06-00-00, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 found-> vendor=0x1166, dev=0x0006, revid=0x01 class=06-00-00, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 found-> vendor=0x1166, dev=0x0006, revid=0x01 class=06-00-00, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 found-> vendor=0x1002, dev=0x4752, revid=0x27 class=03-00-00, hdrtype=0x00, mfdev=0 subordinatebus=0 secondarybus=0 map[10]: type 1, range 32, base f5000000, size 24 map[14]: type 1, range 32, base 00001000, size 8 map[18]: type 1, range 32, base f4020000, size 12 IOAPIC #1 intpin 14 -> irq 9 Freeing (NOT implemented) redirected PCI irq 11. found-> vendor=0x8086, dev=0x1229, revid=0x09 class=02-00-00, hdrtype=0x00, mfdev=0 subordinatebus=0 secondarybus=0 intpin=a, irq=9 map[10]: type 1, range 32, base f4021000, size 12 map[14]: type 1, range 32, base 00001400, size 6 map[18]: type 1, range 32, base f4000000, size 17 found-> vendor=0x1166, dev=0x0200, revid=0x51 class=06-01-00, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 found-> vendor=0x1166, dev=0x0211, revid=0x00 class=01-01-8a, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 map[20]: type 1, range 32, base 00001800, size 4 IOAPIC #1 intpin 12 -> irq 10 Freeing (NOT implemented) redirected PCI irq 9. found-> vendor=0x1166, dev=0x0220, revid=0x04 class=0c-03-10, hdrtype=0x00, mfdev=1 subordinatebus=0 secondarybus=0 intpin=a, irq=10 map[10]: type 1, range 32, base f4022000, size 12 pci0: on pcib0 pci0: (vendor=0x1002, dev=0x4752) at 4.0 fxp0: port 0x1400-0x143f mem 0xf4000000-0xf401ffff,0xf4021000-0xf4021fff irq 9 at device 10.0 on pci0 fxp0: using memory space register mapping fxp0: Ethernet address 00:30:05:29:07:61 fxp0: PCI IDs: 8086 1229 110a 004b 0009 fxp0: Dynamic Standby mode is disabled inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto bpf: fxp0 attached isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x1800-0x180f at device 15.1 on pci0 ata0: iobase=0x01f0 altiobase=0x03f6 bmaddr=0x1800 ata0: mask=03 status0=20 status1=30 ata0: mask=03 ostat0=20 ostat2=30 ata0-master: ATAPI probe a=20 b=20 ata0-slave: ATAPI probe a=30 b=30 ata0: mask=03 status0=20 status1=30 ata0-master: ATA probe a=25 b=25 ata0-slave: ATA probe a=25 b=25 ata0: devices=00 ata0: at 0x1f0 irq 14 on atapci0 ata1: iobase=0x0170 altiobase=0x0376 bmaddr=0x1808 ata1: mask=03 status0=50 status1=00 ata1: mask=03 ostat0=50 ostat2=00 ata1-master: ATAPI probe a=14 b=eb ata1-slave: ATAPI probe a=14 b=eb ata1: mask=03 status0=00 status1=00 ata1: devices=0c ata1: at 0x170 irq 15 on atapci0 ohci0: mem 0xf4022000-0xf4022fff irq 10 at device 15.2 on pci0 ohci0: (New OHCI DeviceId=0x02201166) usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: (unknown) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered pcib2: on motherboard found-> vendor=0x1011, dev=0x0026, revid=0x02 class=06-04-00, hdrtype=0x01, mfdev=0 subordinatebus=3 secondarybus=3 IOAPIC #1 intpin 9 -> irq 11 found-> vendor=0x1069, dev=0x0050, revid=0x02 class=01-04-00, hdrtype=0x00, mfdev=0 subordinatebus=0 secondarybus=0 intpin=a, irq=11 map[10]: type 1, range 32, base f8000000, size 26 pci2: on pcib2 pcib4: at device 10.0 on pci2 IOAPIC #1 intpin 11 -> irq 16 Freeing (NOT implemented) redirected PCI irq 5. found-> vendor=0x9004, dev=0x6915, revid=0x03 class=02-00-00, hdrtype=0x00, mfdev=0 subordinatebus=0 secondarybus=0 intpin=a, irq=16 map[10]: type 1, range 32, base f6400000, size 19 map[14]: type 1, range 32, base 00002000, size 8 IOAPIC #1 intpin 8 -> irq 17 Freeing (NOT implemented) redirected PCI irq 9. found-> vendor=0x9004, dev=0x6915, revid=0x03 class=02-00-00, hdrtype=0x00, mfdev=0 subordinatebus=0 secondarybus=0 intpin=a, irq=17 map[10]: type 1, range 32, base f6480000, size 19 map[14]: type 1, range 32, base 00002400, size 8 pci3: on pcib4 sf0: port 0x2000-0x20ff mem 0xf6400000-0xf647ffff irq 16 at device 4.0 on pci3 sf0: Ethernet address: 00:00:d1:9d:b7:a4 miibus1: on sf0 ukphy0: on miibus1 ukphy0: OUI 0x0005be, model 0x0003, rev. 1 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto bpf: sf0 attached sf1: port 0x2400-0x24ff mem 0xf6480000-0xf64fffff irq 17 at device 5.0 on pci3 sf1: Ethernet address: 00:00:d1:9d:b7:a5 miibus2: on sf1 ukphy1: on miibus2 ukphy1: OUI 0x0005be, model 0x0003, rev. 1 ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto bpf: sf1 attached mly0: mem 0xf8000000-0xfbffffff irq 11 at device 12.0 on pci2 mly0: AcceleRAID 352 , 2 channels, firmware 6.00-17-00 (20020218), 64MB RAM mly0: Siemens AcceleRAID 352 (1e), 33MHz 64-bit PCI mly0: 64MB 66MHz 64-bit SDRAM+ECC, cache 56MB mly0: CPU: i960RN @ 100MHZ mly0: 4MB 66MHz 64-bit private SDRAM+ECC mly0: battery backup not installed mly0: maximum data transfer 2048 blocks, maximum sg entries/command 257 mly0: logical devices present/critical/offline 1/0/0 mly0: physical devices present 7 mly0: physical disks present/offline 4/0 mly0: 2 physical channels, 2 virtual channels of 2 possible mly0: 512 parallel commands supported mly0: 1MB flash ROM, 0 of 100000 maximum cycles pci-: pci3 exists, using next available unit number pcib3: on motherboard pci4: on pcib3 ex_isa_identify() ata-: ata0 exists, using next available unit number ata-: ata1 exists, using next available unit number Trying Read_Port at 203 Trying Read_Port at 243 Trying Read_Port at 283 Trying Read_Port at 2c3 Trying Read_Port at 303 Trying Read_Port at 343 Trying Read_Port at 383 Trying Read_Port at 3c3 isa_probe_children: disabling PnP devices isa_probe_children: probing non-PnP devices orm0: