Date: Tue, 16 Nov 2010 13:03:16 GMT From: Mykola Zubach <zuborg@gmail.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/152294: [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works Message-ID: <201011161303.oAGD3GHE072157@www.freebsd.org> Resent-Message-ID: <201011161310.oAGDA40n047415@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 152294 >Category: kern >Synopsis: [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Nov 16 13:10:04 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Mykola Zubach >Release: 8.1-RELEASE >Organization: AdvancedHosters.com >Environment: FreeBSD 23382 7.3-RELEASE-p2 FreeBSD 7.3-RELEASE-p2 #0: Fri Jul 16 14:22:06 UTC 2010 root@DS1105:/usr/obj/usr/src/sys/Z-AMD64 amd64 >Description: On 8.1-RELEASE (both custom and GENERIC kernels) after some period of quite high bandwidth (10-20 minutes of 600-800Mbit/s) bge0 device fail to receive any incoming packets. After such fail 'tcpdump' show no incoming traffic, ssh still provide some amount of outgoing data (like 'top' or 'nload' output), but keypresses don't reach the server and tcp session drops after timeout. These two commands restore working state # ifconfig bge0 down # ifconfig bge0 up Without 'polling' driver hangs up very quicky, in 1-2 minutes. Active 'polling' can provide 10-20 minutes of networking. Bandwidth is produced by 'nginx' webserver. Various configurations were tested, like 'aio sendfile' feature, or pure read/write operations without 'sendfile on' - in all cases bge fails. Kernel is compiled with ZERO_COPY_SOCKETS option, but GENERIC kernel fails too. 7.3-RELEASE works fine under same high load. # dmesg FreeBSD 8.1-RELEASE-p1 #0: Mon Oct 18 11:31:13 UTC 2010 root@DS1728:/usr/obj/usr/src/sys/Z-AMD64 amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU X3430 @ 2.40GHz (2394.00-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106e5 Family = 6 Model = 1e Stepping = 5 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x98e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant real memory = 17179869184 (16384 MB) avail memory = 16546603008 (15780 MB) ACPI APIC Table: <HP ProLiant> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 ioapic0 <Version 2.0> irqs 0-23 on motherboard netisr_init: forcing maxthreads to 1 and bindthreads to 0 for device polling kbd1 at kbdmux0 acpi0: <HP ProLiant> on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> irq 16 at device 3.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pci0: <base peripheral> at device 8.0 (no driver attached) pci0: <base peripheral> at device 8.1 (no driver attached) pci0: <base peripheral> at device 8.2 (no driver attached) pci0: <base peripheral> at device 8.3 (no driver attached) pci0: <base peripheral> at device 16.0 (no driver attached) pci0: <base peripheral> at device 16.1 (no driver attached) ehci0: <Intel PCH USB 2.0 controller USB-B> mem 0xdfd02000-0xdfd023ff irq 16 at device 26.0 on pci0 ehci0: [ITHREAD] usbus0: EHCI version 1.0 usbus0: <Intel PCH USB 2.0 controller USB-B> on ehci0 pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0 pci16: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0 pci32: <ACPI PCI bus> on pcib3 bge0: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem 0xdf900000-0xdf90ffff irq 16 at device 0.0 on pci32 miibus0: <MII bus> on bge0 brgphy0: <BCM5784 10/100/1000baseTX PHY> PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: Ethernet address: d4:85:64:38:ff:70 bge0: [FILTER] pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.5 on pci0 pci34: <ACPI PCI bus> on pcib4 bge1: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem 0xdfa00000-0xdfa0ffff irq 17 at device 0.0 on pci34 miibus1: <MII bus> on bge1 brgphy1: <BCM5784 10/100/1000baseTX PHY> PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge1: Ethernet address: d4:85:64:38:ff:71 bge1: [FILTER] pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.6 on pci0 pci36: <ACPI PCI bus> on pcib5 vgapci0: <VGA-compatible display> mem 0xde000000-0xdeffffff,0xdf800000-0xdf803fff,0xdf000000-0xdf7fffff irq 18 at device 0.0 on pci36 pcib6: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0 pci38: <ACPI PCI bus> on pcib6 ehci1: <Intel PCH USB 2.0 controller USB-A> mem 0xdfd02400-0xdfd027ff irq 23 at device 29.0 on pci0 ehci1: [ITHREAD] usbus1: EHCI version 1.0 usbus1: <Intel PCH USB 2.0 controller USB-A> on ehci1 pcib7: <PCI-PCI bridge> at device 30.0 on pci0 pci48: <PCI bus> on pcib7 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel PCH SATA300 controller> port 0x1830-0x1837,0x1824-0x1827,0x1828-0x182f,0x1820-0x1823,0x1800-0x181f mem 0xdfd01000-0xdfd017ff irq 18 at device 31.2 on pci0 atapci0: [ITHREAD] atapci0: AHCI called from vendor specific driver atapci0: AHCI v1.30 controller with 6 3Gbps ports, PM supported ata2: <ATA channel 0> on atapci0 ata2: [ITHREAD] ata3: <ATA channel 1> on atapci0 ata3: [ITHREAD] ata4: <ATA channel 2> on atapci0 ata4: [ITHREAD] ata5: <ATA channel 3> on atapci0 ata5: [ITHREAD] ata6: <ATA channel 4> on atapci0 ata6: [ITHREAD] ata7: <ATA channel 5> on atapci0 ata7: [ITHREAD] ichsmb0: <Intel PCH SMBus controller> port 0x1840-0x185f mem 0xdfd02800-0xdfd028ff irq 18 at device 31.3 on pci0 ichsmb0: [ITHREAD] smbus0: <System Management Bus> on ichsmb0 smb0: <SMBus generic I/O> on smbus0 acpi_button0: <Power Button> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdf000-0xdffff on isa0 atkbd: unable to set the command byte. sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: unable to set the command byte. coretemp0: <CPU On-Die Thermal Sensors> on cpu0 coretemp1: <CPU On-Die Thermal Sensors> on cpu1 coretemp2: <CPU On-Die Thermal Sensors> on cpu2 coretemp3: <CPU On-Die Thermal Sensors> on cpu3 Timecounter "TSC" frequency 2393999100 Hz quality 800 Timecounters tick every 0.976 msec ipfw2 initialized, divert loadable, nat loadable, rule-based forwarding disabled, default to accept, logging disabled usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 ad4: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata2-master UDMA100 SATA 3Gb/s ugen0.1: <Intel> at usbus0 uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0 ugen1.1: <Intel> at usbus1 uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 ad6: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata3-master UDMA100 SATA 3Gb/s GEOM: ad4s1: geometry does not match label (255h,63s != 16h,63s). GEOM: ad4s1: media size does not match label. ad8: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata4-master UDMA100 SATA 3Gb/s GEOM: ad4s2: geometry does not match label (255h,63s != 16h,63s). GEOM: ad4s2: media size does not match label. ad10: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata5-master UDMA100 SATA 3Gb/s SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! GEOM: ad6s1: geometry does not match label (255h,63s != 16h,63s). GEOM: ad6s1: media size does not match label. GEOM: ad6s2: geometry does not match label (255h,63s != 16h,63s). GEOM: ad6s2: media size does not match label. GEOM: ad8s1: geometry does not match label (255h,63s != 16h,63s). GEOM: ad8s1: media size does not match label. GEOM_MIRROR: Device mirror/gm0 launched (3/3). GEOM: ad8s2: geometry does not match label (255h,63s != 16h,63s). GEOM: ad8s2: media size does not match label. GEOM: ad10s1: geometry does not match label (255h,63s != 16h,63s). GEOM: ad10s1: media size does not match label. GEOM_MIRROR: Device mirror/gm2 launched (1/1). GEOM: ad10s2: geometry does not match label (255h,63s != 16h,63s). GEOM: ad10s2: media size does not match label. GEOM_MIRROR: Device mirror/gm1 launched (4/4). Root mount waiting for: usbus1 usbus0 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered Root mount waiting for: usbus1 usbus0 ugen0.2: <vendor 0x8087> at usbus0 uhub2: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on usbus0 ugen1.2: <vendor 0x8087> at usbus1 uhub3: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on usbus1 Root mount waiting for: usbus1 usbus0 uhub2: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered ugen1.3: <ServerEngines> at usbus1 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on usbus1 kbd2 at ukbd0 Trying to mount root from ufs:/dev/mirror/gm2a # pciconf -lv hostb0@pci0:0:0:0: class=0x060000 card=0x3318103c chip=0xd1308086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = bridge subclass = HOST-PCI pcib1@pci0:0:3:0: class=0x060400 card=0x3318103c chip=0xd1388086 rev=0x11 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI none0@pci0:0:8:0: class=0x088000 card=0x00000000 chip=0xd1558086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none1@pci0:0:8:1: class=0x088000 card=0x00000000 chip=0xd1568086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none2@pci0:0:8:2: class=0x088000 card=0x00000000 chip=0xd1578086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none3@pci0:0:8:3: class=0x088000 card=0x00000000 chip=0xd1588086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none4@pci0:0:16:0: class=0x088000 card=0x00000000 chip=0xd1508086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none5@pci0:0:16:1: class=0x088000 card=0x00000000 chip=0xd1518086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral ehci0@pci0:0:26:0: class=0x0c0320 card=0x3118103c chip=0x3b3c8086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' class = serial bus subclass = USB pcib2@pci0:0:28:0: class=0x060400 card=0x3318103c chip=0x3b428086 rev=0x05 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI pcib3@pci0:0:28:4: class=0x060400 card=0x3318103c chip=0x3b4a8086 rev=0x05 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI pcib4@pci0:0:28:5: class=0x060400 card=0x3318103c chip=0x3b4c8086 rev=0x05 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI pcib5@pci0:0:28:6: class=0x060400 card=0x3318103c chip=0x3b4e8086 rev=0x05 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI pcib6@pci0:0:28:7: class=0x060400 card=0x3318103c chip=0x3b508086 rev=0x05 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI ehci1@pci0:0:29:0: class=0x0c0320 card=0x3118103c chip=0x3b348086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' class = serial bus subclass = USB pcib7@pci0:0:30:0: class=0x060401 card=0x3318103c chip=0x244e8086 rev=0xa5 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:0:31:0: class=0x060100 card=0x3118103c chip=0x3b148086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' class = bridge subclass = PCI-ISA atapci0@pci0:0:31:2: class=0x010601 card=0x3118103c chip=0x3b228086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = 'IBEX AHCI Controller(6Port)' class = mass storage subclass = SATA ichsmb0@pci0:0:31:3: class=0x0c0500 card=0x3318103c chip=0x3b308086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' class = serial bus subclass = SMBus bge0@pci0:32:0:0: class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10 hdr=0x00 vendor = 'Broadcom Corporation' class = network subclass = ethernet bge1@pci0:34:0:0: class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10 hdr=0x00 vendor = 'Broadcom Corporation' class = network subclass = ethernet vgapci0@pci0:36:0:0: class=0x030000 card=0x31fa103c chip=0x0522102b rev=0x02 hdr=0x00 vendor = 'Matrox Electronic Systems Ltd.' device = 'Matrox G200e (ServerEngines) - English (G200e)' class = display subclass = VGA Server model: ProLiant DL120 G6 >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201011161303.oAGD3GHE072157>