From owner-freebsd-current@FreeBSD.ORG Sat May 8 10:08:47 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AE0A31065673 for ; Sat, 8 May 2010 10:08:47 +0000 (UTC) (envelope-from joe@hostedcontent.com) Received: from mail01.fasti.net (mail01.fasti.net [216.105.91.156]) by mx1.freebsd.org (Postfix) with ESMTP id 861638FC13 for ; Sat, 8 May 2010 10:08:47 +0000 (UTC) Received: from mail01.fasti.net (localhost [127.0.0.2]) by mail01.fasti.net (Postfix) with ESMTP id 1B82394B1031; Sat, 8 May 2010 06:08:47 -0400 (EDT) X-Virus-Scanned: amavisd-new at fasti.net Received: from mail01.fasti.net ([127.0.0.2]) by mail01.fasti.net (mail01.fasti.net [127.0.0.2]) (amavisd-new, port 10024) with ESMTP id GBLi76RDWuck; Sat, 8 May 2010 06:08:45 -0400 (EDT) Received: from [192.168.1.137] (69-165-175-27.dsl.teksavvy.com [69.165.175.27]) by mail01.fasti.net (Postfix) with ESMTPA id 53F6894B1030; Sat, 8 May 2010 06:08:45 -0400 (EDT) Message-ID: <4BE5303B.8050409@hostedcontent.com> Date: Sat, 08 May 2010 05:34:51 -0400 From: joe User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Fabien Thomas , freebsd-current@freebsd.org References: <4BE529FF.5000008@hostedcontent.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2010 10:08:47 -0000 On 05/08/2010 05:54 AM, Fabien Thomas wrote: > Have you tried to disable TSO / LRO? > > Fabien > > >> I have 3 boxes, each with two nics. One nic for the private network and one for the public network. >> The private network is all on the same vlan. All 6 nics are on the same switch. All connections are 1000tx Full Duplex. >> >> I will call the servers Box A, Box B, and Box C. >> >> When i FTP data between Box A& B i get abou 25MB/sec. >> When i FTP data from Box C to Box A or B, i get about 20MB/sec. >> When i FTP data from Box A to C i get 10MB/sec >> When i FTP data from Box B to C i get 200KB/sec... >> >> Can anyone suggest why i might only be getting 200KB when transfering data from Box B to C but not when transferring data from Box A to C? >> >> I tried installing ubuntu to see if the problem was still there and it is not. I can FTP data from Box A or B to box C at 10MB/sec, still not the gigabit speeds i should be seeing but not the 200KB/sec i am getting with freebsd. >> >> I logged into my switch and it reports that the ports are 1000tx full duplex, the same as what freebsd is reporting. The switch& freebsd also report no errors. >> >> I really dont know what to do. Nothing anywhere reports showing a problem but obviously there is a problem somewhere. I've tried drivers 1.9.5, 1.8.4, and 1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec transferring data from box B whereas if i use 1.9.5 i get 200-300KB sec. I should be getting like 50MB/sec ;( >> >> Any help would be grateful. Or, maybe someone could put me in touch with the person responsible for the intel nic drivers and i can work with them on resolving this issue? >> >> Thanks in advance. >> >> >> Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll >> igb0 1500 00:30:48:9f:11:04 25853 0 0 18340 0 0 >> igb0 1500 216.105.91.14 216.105.91.145 19040 - - 18343 - - >> igb1* 1500 00:30:48:9f:11:05 0 0 0 0 0 0 >> lo0 16384 72 0 0 72 0 0 >> lo0 16384 127.0.0.0/8 127.0.0.1 72 - - 72 - - >> >> Sysctl info for igb0/1 >> dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4 >> dev.igb.0.%driver: igb >> dev.igb.0.%location: slot=0 function=0 >> dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 subdevice=0x0100 class=0x020000 >> dev.igb.0.%parent: pci1 >> dev.igb.0.debug: -1 >> dev.igb.0.stats: -1 >> dev.igb.0.flow_control: 3 >> dev.igb.0.enable_aim: 1 >> dev.igb.0.low_latency: 128 >> dev.igb.0.ave_latency: 450 >> dev.igb.0.bulk_latency: 1200 >> dev.igb.0.rx_processing_limit: 100 >> >> Debug info for igb0 >> May 8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295 >> May 8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106 >> May 8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0 >> May 8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353 >> May 8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450 >> May 8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687 >> May 8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0 >> May 8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335 >> May 8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354 >> May 8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424 >> May 8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809 >> May 8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262 >> May 8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129 >> May 8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428 >> May 8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601 >> May 8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966 >> May 8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699 >> May 8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700 >> May 8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937 >> May 8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691 >> May 8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888 >> May 8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362 >> May 8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351 >> May 8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32 >> May 8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177 >> May 8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442 >> May 8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498 >> May 8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258 >> May 8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794 >> May 8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673 >> May 8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417 >> May 8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418 >> May 8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939 >> May 8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710 >> May 8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399 >> May 8 09:17:57 debbie kernel: igb0: RX(3) LRO Queued= 1102 >> May 8 09:17:57 debbie kernel: igb0: RX(3) LRO Flushed= 882 >> May 8 09:17:57 debbie kernel: igb0: LINK MSIX IRQ Handled = 2 >> May 8 09:17:57 debbie kernel: igb0: Mbuf defrag failed = 0 >> May 8 09:17:57 debbie kernel: igb0: Std mbuf header failed = 0 >> May 8 09:17:57 debbie kernel: igb0: Std mbuf packet failed = 0 >> May 8 09:17:57 debbie kernel: igb0: Driver dropped packets = 0 >> May 8 09:17:57 debbie kernel: igb0: Driver tx dma failure in xmit = 0 >> >> >> sysctl status info: >> May 8 09:19:08 debbie kernel: igb0: Excessive collisions = 0 >> May 8 09:19:08 debbie kernel: igb0: Sequence errors = 0 >> May 8 09:19:08 debbie kernel: igb0: Defer count = 0 >> May 8 09:19:08 debbie kernel: igb0: Missed Packets = 0 >> May 8 09:19:08 debbie kernel: igb0: Receive No Buffers = 0 >> May 8 09:19:08 debbie kernel: igb0: Receive Length Errors = 0 >> May 8 09:19:08 debbie kernel: igb0: Receive errors = 0 >> May 8 09:19:08 debbie kernel: igb0: Crc errors = 0 >> May 8 09:19:08 debbie kernel: igb0: Alignment errors = 0 >> May 8 09:19:08 debbie kernel: igb0: Collision/Carrier extension errors = 0 >> May 8 09:19:08 debbie kernel: igb0: RX overruns = 0 >> May 8 09:19:08 debbie kernel: igb0: watchdog timeouts = 0 >> May 8 09:19:08 debbie kernel: igb0: XON Rcvd = 17 >> May 8 09:19:08 debbie kernel: igb0: XON Xmtd = 0 >> May 8 09:19:08 debbie kernel: igb0: XOFF Rcvd = 17 >> May 8 09:19:08 debbie kernel: igb0: XOFF Xmtd = 0 >> May 8 09:19:08 debbie kernel: igb0: Good Packets Rcvd = 26159 >> May 8 09:19:08 debbie kernel: igb0: Good Packets Xmtd = 21564 >> May 8 09:19:08 debbie kernel: igb0: TSO Contexts Xmtd = 1687 >> May 8 09:19:08 debbie kernel: igb0: TSO Contexts Failed = 0 >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> > I have just tried your suggeston and it has no effect for me ;( igb0: flags=8843 metric 0 mtu 1500 options=3b ether 00:30:48:9f:11:04 inet 216.105.91.145 netmask 0xfffffff0 broadcast 216.105.91.159 media: Ethernet autoselect (1000baseT ) status: active