From owner-freebsd-questions@FreeBSD.ORG Tue Mar 13 16:04:45 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8C40106566B for ; Tue, 13 Mar 2012 16:04:45 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 999988FC08 for ; Tue, 13 Mar 2012 16:04:45 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.5/8.14.4) with ESMTP id q2DG4hL9042547 for ; Tue, 13 Mar 2012 12:04:43 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4F5F7016.1060406@sentex.net> Date: Tue, 13 Mar 2012 12:04:38 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-questions X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.71 on IPv6:2607:f3e0:0:1::12 Subject: Interrupt storm and Intel DQ67SW X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Mar 2012 16:04:46 -0000 In case anyone else runs into this problem, here it is for the archives. FreeBSD 8.3, AMD64 (Actually, RELENG_8 as of March 2012) I upgraded my home server to an Intel DQ67SW motherboard (hardware version AAG12527-309). I used a bge PCI-E nic in the 1x slot which caused an interrupt storm issue with the usb. Trying an old PCI fxp card showed the same problem. Even with just 2Mb of pppoe traffic crossing the bge or fxp, I would see storm issues. Running top, showed 10% of the system was spending its time servicing interrupts. vmstat -i showed a rate of close to 10k on irq16. I upgraded the BIOS to SWQ6710H.86A.0061.2012.0210.1130 from the Nov 2011 version and all is fine now. Everything about the box 'feels' more responsive and throughput on my wan connection is normal again. # vmstat -i interrupt total rate irq16: bge0 ehci0+ 40439 3 irq20: fwohci0 2 0 irq23: ehci1 22177 2 cpu0: timer 22062162 2000 irq256: em0 14825 1 irq257: ahci0 4874088 441 cpu3: timer 22054239 1999 cpu1: timer 22054150 1999 cpu2: timer 22054168 1999 Total 93176250 8446 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM) i5-2300 CPU @ 2.80GHz (2793.67-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 Features=0xbfebfbff Features2=0x179ae3bf AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant real memory = 17179869184 (16384 MB) avail memory = 16434216960 (15672 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ichwd module loaded cryptosoft0: on motherboard aesni0: on motherboard acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0xf000-0xf03f mem 0xfb400000-0xfb7fffff,0xc0000000-0xcfffffff irq 16 at device 2.0 on pci0 pci0: at device 22.0 (no driver attached) atapci0: port 0xf130-0xf137,0xf120-0xf123,0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0ff irq 18 at device 22.2 on pci0 atapci0: [ITHREAD] ata2: at channel 0 on atapci0 ata2: [ITHREAD] ata3: at channel 1 on atapci0 ata3: [ITHREAD] uart2: port 0xf0e0-0xf0e7 mem 0xfbc29000-0xfbc29fff irq 17 at device 22.3 on pci0 uart2: [FILTER] em0: port 0xf080-0xf09f mem 0xfbc00000-0xfbc1ffff,0xfbc28000-0xfbc28fff irq 20 at device 25.0 on pci0 em0: Using an MSI interrupt em0: [FILTER] em0: Ethernet address: 00:22:4d:52:5c:34 ehci0: mem 0xfbc27000-0xfbc273ff irq 16 at device 26.0 on pci0 ehci0: [ITHREAD] usbus0: EHCI version 1.0 usbus0: on ehci0 pci0: at device 27.0 (no driver attached) pcib2: irq 17 at device 28.0 on pci0 pci2: on pcib2 bge0: mem 0xfbb10000-0xfbb1ffff irq 16 at device 0.0 on pci2 bge0: CHIP ID 0x00004001; ASIC REV 0x04; CHIP REV 0x40; PCI-E miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:10:18:14:15:43 bge0: [ITHREAD] pcib3: irq 17 at device 28.4 on pci0 pci3: on pcib3 3ware device driver for 9000 series storage controllers, version: 3.80.06.003 twa0: <3ware 9000 series Storage Controller> port 0xe000-0xe0ff mem 0xd0000000-0xd1ffffff,0xfba20000-0xfba20fff irq 16 at device 0.0 on pci3 twa0: [ITHREAD] twa0: INFO: (0x15: 0x1300): Controller details:: Model 9650SE-2LP, 2 ports, Firmware FE9X 4.08.00.006, BIOS BE9X 4.08.00.001 pcib4: irq 18 at device 28.6 on pci0 pci4: on pcib4 xhci0: mem 0xfb900000-0xfb901fff irq 18 at device 0.0 on pci4 xhci0: [ITHREAD] xhci0: 32 byte context size. usbus1 on xhci0 ehci1: mem 0xfbc26000-0xfbc263ff irq 23 at device 29.0 on pci0 ehci1: [ITHREAD] usbus2: EHCI version 1.0 usbus2: on ehci1 pcib5: at device 30.0 on pci0 pci5: on pcib5 fwohci0: mem 0xfb800000-0xfb800fff irq 20 at device 3.0 on pci5 fwohci0: [ITHREAD] fwohci0: OHCI version 1.0 (ROM=0) fwohci0: No. of Isochronous channels is 8. fwohci0: EUI64 00:22:4d:ff:ff:52:8f:44 fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:22:4d:52:8f:44 fwe0: Ethernet address: 02:22:4d:52:8f:44 fwip0: on firewire0 fwip0: Firewire address: 00:22:4d:ff:ff:52:8f:44 @ 0xfffe00000000, S400, maxrec 2048 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x46ac000 fwohci0: Initiate bus reset fwohci0: fwohci_intr_core: BUS reset fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xfbc25000-0xfbc257ff irq 19 at device 31.2 on pci0 ahci0: [ITHREAD] ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich0: [ITHREAD] ahcich1: at channel 1 on ahci0 ahcich1: [ITHREAD] ahcich2: at channel 2 on ahci0 ahcich2: [ITHREAD] ahcich3: at channel 3 on ahci0 ahcich3: [ITHREAD] ahcich4: at channel 4 on ahci0 ahcich4: [ITHREAD] ahcich5: at channel 5 on ahci0 ahcich5: [ITHREAD] pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 atrtc0: port 0x70-0x71 irq 8 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] tpm0: iomem 0xfed40000-0xfed44fff on acpi0 tpm: WEC WPCT200 rev 0x46 ichwd0: on isa0 ichwd0: Intel Cougar Point watchdog timer (ICH10 or equivalent) orm0: at iomem 0xc0000-0xcdfff,0xce000-0xcefff,0xcf000-0xd0fff,0xd1000-0xd2fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] ppc0: cannot reserve I/O port range est0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 p4tcc1: on cpu1 est2: on cpu2 p4tcc2: on cpu2 est3: on cpu3 p4tcc3: on cpu3 ZFS filesystem version 5 ZFS storage pool version 28 Timecounters tick every 1.000 msec IPsec: Initialized Security Association Processing. firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) firewire0: bus manager 0 usbus0: 480Mbps High Speed USB v2.0 usbus1: 5.0Gbps Super Speed USB v3.0 usbus2: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: <0x1033> at usbus1 uhub1: <0x1033 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 uhub1: 4 ports with 4 removable, self powered uhub0: 3 ports with 3 removable, self powered uhub2: 3 ports with 3 removable, self powered ugen0.2: at usbus0 uhub3: on usbus0 ugen2.2: at usbus2 uhub4: on usbus2 uhub3: 6 ports with 6 removable, self powered uhub4: 8 ports with 8 removable, self powered ugen0.3: at usbus0 ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 2.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 2.x device ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) da0 at twa0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 100.000MB/s transfers da0: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C) SMP: AP CPU #3 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! ugen0.4: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 ums0: on usbus0 ums0: 5 buttons and [XYZ] coordinates ID=1 Trying to mount root from ufs:/dev/da0s1a ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/