From owner-freebsd-stable@FreeBSD.ORG Tue Mar 8 03:04:21 2005 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1D4D416A4CE for ; Tue, 8 Mar 2005 03:04:21 +0000 (GMT) Received: from server.tmk.com (server.tmk.com [204.141.35.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4CE7643D48 for ; Tue, 8 Mar 2005 03:04:18 +0000 (GMT) (envelope-from terry@tmk.com) Received: from tmk.com by tmk.com (PMDF V6.2-X27 #37010) id <01LLMI1DXGYO004H9Z@tmk.com> for freebsd-stable@freebsd.org; Mon, 07 Mar 2005 22:04:16 -0400 (EDT) Date: Mon, 07 Mar 2005 21:51:15 -0400 (EDT) From: Terry Kennedy To: freebsd-stable@freebsd.org Message-id: <01LLMIICQ7RE004H9Z@tmk.com> MIME-version: 1.0 Content-type: TEXT/PLAIN; CHARSET=us-ascii Subject: Apparent interrupt routing problem in 5.4-PRERELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Mar 2005 03:04:21 -0000 I've communicated with a few people about this off-list, and it was sug- gested I give the issue some wider exposure on this list in the hope of having it addressed for 5.4-RELEASE. It may or not be related to the other interrupt storm problems some people are seeing. I have a number of systems running the latest 5-STABLE (as of 4 PM today or so). I've been seeing this issue for quite some time, though (5.3-RELEASE at least, though I don't remember it happening in 5.2.1-RELEASE). The first symptom is that at boot time, I see these messages: Interrupt storm detected on "irq16: uhci0"; throttling interrupt source Interrupt storm detected on "irq17: ichsmb0"; throttling interrupt source Next, during the whole time the system is up, a "systat -v" shows that my uhci0 and ichsmb0 devices have active interrupt counts (despite no activity on them) which happen to *exactly* correspond with "real" interrupt activity on other devices. The motherboards are Tyan S2721-533's. The rest should be apparent from the dmesg output. I'm attaching two consecutive screen captures of the systat -v output as well as the dmesg output. This happens on both SMP and UP (with a UP kernel) configs, and also with or without ACPI enabled (by option at boot time). Let me know if anyone needs further information to help diagnose this. I can also provide remote access to a test system if a developer needs it. Terry Kennedy http://www.tmk.com terry@tmk.com New York, NY USA 3 users Load 0.00 0.00 0.00 Mar 7 21:57 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 19276 3368 62804 4176 2873112 count 1 All 1108320 6276 3306404 8148 pages 1 zfod Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt cow 7244 total 35 15923 1 663810513 230 213256 wire 1: atkb 12032 act 6: fdc0 8.1%Sys 3.7%Intr 0.2%User 0.0%Nice 87.9%Idl 884388 inact 128 8: rtc | | | | | | | | | | cache 12: psm ====++ 2873112 free 13: npx daefr 15: ata Namei Name-cache Dir-cache prcfr 3175 16: uhc Calls hits % hits % react 332 17: ich pdwak 24: twa pdpgs 3175 48: em0 Disks da0 da1 sa0 pass0 pass1 pass2 intrn 332 72: twa KB/t 9.67 127 0.00 0.00 0.00 0.00 114464 buf 98: ahd tps 2 167 0 0 0 0 9 dirty 99: ahd MB/s 0.02 20.62 0.00 0.00 0.00 0.00 100000 desir 100e0: clk % busy 0 71 0 0 0 0 64552 numvnodes 14155 freevnodes 3 users Load 0.07 0.02 0.00 Mar 7 21:57 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 19276 3368 62804 4176 2552792 count All 1428640 6276 3821844 8148 pages zfod Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt cow 7610 total 35 16686 2 696811000 237 213324 wire 1: atkb 12040 act 6: fdc0 9.2%Sys 3.7%Intr 0.4%User 0.0%Nice 86.8%Idl 1204632 inact 128 8: rtc | | | | | | | | | | cache 12: psm =====+> 2552792 free 13: npx daefr 15: ata Namei Name-cache Dir-cache prcfr 3344 16: uhc Calls hits % hits % react 347 17: ich pdwak 24: twa pdpgs 3344 48: em0 Disks da0 da1 sa0 pass0 pass1 pass2 intrn 347 72: twa KB/t 0.00 127 0.00 0.00 0.00 0.00 114464 buf 98: ahd tps 0 174 0 0 0 0 9 dirty 99: ahd MB/s 0.00 21.69 0.00 0.00 0.00 0.00 100000 desir 100e0: clk % busy 0 69 0 0 0 0 64552 numvnodes 14150 freevnodes Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.4-PRERELEASE #0: Mon Mar 7 16:15:22 EST 2005 terry@rz1.tmk.com:/usr/obj/usr/src/sys/RAIDZILLA-SMP ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3065.81-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff real memory = 4160684032 (3967 MB) avail memory = 4073807872 (3885 MB) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard ioapic3 irqs 72-95 on motherboard ioapic4 irqs 96-119 on motherboard npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: on acpi0 acpi_throttle0: on cpu0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci5: on pcib1 pci5: at device 28.0 (no driver attached) pcib2: at device 29.0 on pci5 pci7: on pcib2 em0: port 0xd880-0xd8bf mem 0xfe9c0000-0xfe9dffff irq 48 at device 1.0 on pci7 em0: Ethernet address: 00:e0:81:28:94:d6 em0: Speed:N/A Duplex:N/A em1: port 0xdc00-0xdc3f mem 0xfe9e0000-0xfe9fffff irq 49 at device 1.1 on pci7 em1: Ethernet address: 00:e0:81:28:94:d7 em1: Speed:N/A Duplex:N/A pci5: at device 30.0 (no driver attached) pcib3: at device 31.0 on pci5 pci6: on pcib3 3ware device driver for 9000 series storage controllers, version: 2.50.02.012 twa0: <3ware 9000 series Storage Controller> port 0xc800-0xc8ff mem 0xfb800000-0xfbffffff,0xfe8ffc00-0xfe8ffcff irq 24 at device 3.0 on pci6 twa0: INFO: (0x04: 0x0055): Battery charging started: twa0: 8 ports, Firmware FE9X 2.04.00.005, BIOS BE9X 2.03.01.047 pcib4: at device 3.0 on pci0 pci2: on pcib4 pci2: at device 28.0 (no driver attached) pcib5: at device 29.0 on pci2 pci4: on pcib5 ahd0: port 0xa800-0xa8ff,0xb000-0xb0ff mem 0xfe6fc000-0xfe6fdfff irq 98 at device 2.0 on pci4 aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs ahd1: port 0xb400-0xb4ff,0xb800-0xb8ff mem 0xfe6fe000-0xfe6fffff irq 99 at device 2.1 on pci4 aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs pci2: at device 30.0 (no driver attached) pcib6: at device 31.0 on pci2 pci3: on pcib6 twa1: <3ware 9000 series Storage Controller> port 0x9800-0x98ff mem 0xfa000000-0xfa7fffff,0xfe3ffc00-0xfe3ffcff irq 72 at device 3.0 on pci3 twa1: INFO: (0x04: 0x0055): Battery charging started: twa1: INFO: (0x04: 0x0053): Battery capacity test is overdue: twa1: 8 ports, Firmware FE9X 2.04.00.005, BIOS BE9X 2.03.01.047 uhci0: port 0xec00-0xec1f irq 16 at device 29.0 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered pcib7: at device 30.0 on pci0 pci1: on pcib7 fxp0: port 0x8c00-0x8c3f mem 0xfe2a0000-0xfe2bffff,0xfe2fe000-0xfe2fefff irq 19 at device 1.0 on pci1 miibus0: on fxp0 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:e0:81:28:93:fb pci1: at device 2.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xffa0-0xffaf,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 ichsmb0: port 0x540-0x55f at device 31.3 on pci0 smbus0: on ichsmb0 smb0: on smbus0 acpi_button0: on acpi0 atkbdc0: port 0x64,0x60 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 psm0: irq 12 on atkbdc0 psm0: model Generic PS/2 mouse, device ID 0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port pmtimer0 on isa0 orm0: at iomem 0xcd000-0xce7ff,0xcb800-0xccfff,0xcb000-0xcb7ff,0xc9800-0xcafff,0xc8000-0xc97ff,0xc0000-0xc7fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 10.000 msec acd0: CDROM at ata1-master UDMA33 Waiting 15 seconds for SCSI devices to settle Interrupt storm detected on "irq16: uhci0"; throttling interrupt source Interrupt storm detected on "irq17: ichsmb0"; throttling interrupt source sa0 at ahd0 bus 0 target 6 lun 0 sa0: Removable Sequential Access SCSI-2 device sa0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit) SMP: AP CPU #1 Launched! da0 at twa0 bus 0 target 0 lun 0 da0: <3ware Logical Disk 00 1.00> Fixed Direct Access SCSI-0 device da0: 100.000MB/s transfers da0: 2288754MB (4687368192 512 byte sectors: 255H 63S/T 291775C) da1 at twa1 bus 0 target 0 lun 0 da1: <3ware Logical Disk 00 1.00> Fixed Direct Access SCSI-0 device da1: 100.000MB/s transfers da1: 2288754MB (4687368192 512 byte sectors: 255H 63S/T 291775C)