From owner-freebsd-net@FreeBSD.ORG Tue Oct 4 14:27:43 2005 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C35BF16A41F for ; Tue, 4 Oct 2005 14:27:43 +0000 (GMT) (envelope-from ferdinand.goldmann@jku.at) Received: from mail2.edvz.uni-linz.ac.at (mail2.edvz.uni-linz.ac.at [140.78.3.69]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42DD343D46 for ; Tue, 4 Oct 2005 14:27:42 +0000 (GMT) (envelope-from ferdinand.goldmann@jku.at) Received: from emailsecure.uni-linz.ac.at (emailsecure.uni-linz.ac.at [140.78.3.66]) by mail2.edvz.uni-linz.ac.at (8.13.3/8.13.3) with ESMTP id j94ERWpK093090 for ; Tue, 4 Oct 2005 16:27:32 +0200 (CEST) (envelope-from ferdinand.goldmann@jku.at) Received: from [140.78.164.13] (jku006048.edvz.uni-linz.ac.at [140.78.6.48]) by emailsecure.uni-linz.ac.at (Postfix) with ESMTP id 272A6228019 for ; Tue, 4 Oct 2005 16:27:32 +0200 (CEST) Message-ID: <43429157.90606@jku.at> Date: Tue, 04 Oct 2005 16:27:35 +0200 From: Ferdinand Goldmann Organization: Johannes Kepler University User-Agent: Thunderbird 1.4 (Macintosh/20050908) MIME-Version: 1.0 To: net@freebsd.org References: <4341089F.7010504@jku.at> <20051003104548.GB70355@cell.sick.ru> <4341242F.9060602@jku.at> <20051003123210.GF70355@cell.sick.ru> <43426EF3.3020404@jku.at> <9CD8C672-1EF2-42FE-A61E-83DC684C893D@dragondata.com> In-Reply-To: <9CD8C672-1EF2-42FE-A61E-83DC684C893D@dragondata.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.44 Cc: Subject: Re: dummynet, em driver, device polling issues :-(( X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ferdinand.goldmann@jku.at List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2005 14:27:44 -0000 Kevin Day wrote: > This is pretty odd. We've got dozens of servers using various versions > of 5.x, and many different em cards, and have no problem, even when > shoving near line rate speeds out of them. Maximum transfer rates we see in MRTG were around 320Mbit/s (with polling disabled) > em0: port > 0xecc0-0xecff mem 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6 em0: port 0x2280-0x22bf mem 0xeffc0000-0xeffdffff irq 20 at device 5.0 on pci1 Pretty much the same here, even the driver version. em0@pci1:5:0: class=0x020000 card=0x10028086 chip=0x10268086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '82545GM Gigabit Ethernet Controller' > After you experience your problems, can you do "sysctl -w > hw.em0.stats=1" and "sysctl -w hw.em0.debug_info=1" and post what gets > dumped to your syslog/dmesg output? em0: Excessive collisions = 0 em0: Symbol errors = 0 em0: Sequence errors = 0 em0: Defer count = 11 em0: Missed Packets = 0 em0: Receive No Buffers = 0 em0: Receive length errors = 0 em0: Receive errors = 0 em0: Crc errors = 0 em0: Alignment errors = 0 em0: Carrier extension errors = 0 em0: XON Rcvd = 11 em0: XON Xmtd = 0 em0: XOFF Rcvd = 11 em0: XOFF Xmtd = 0 em0: Good Packets Rcvd = 283923273 em0: Good Packets Xmtd = 272613648 em0: Adapter hardware address = 0xc12cfb48 em0:CTRL = 0x58f00249 em0:RCTL = 0x8002 PS=(0x8402) em0:tx_int_delay = 66, tx_abs_int_delay = 66 em0:rx_int_delay = 0, rx_abs_int_delay = 66 em0: fifo workaround = 0, fifo_reset = 0 em0: hw tdh = 173, hw tdt = 173 em0: Num Tx descriptors avail = 256 em0: Tx Descriptors not avail1 = 0 em0: Tx Descriptors not avail2 = 0 em0: Std mbuf failed = 0 em0: Std mbuf cluster failed = 0 em0: Driver dropped packets = 0 > We're using polling on nearly all the servers, and don't see ierrs at > all. Hm. That's strange. The above values were gathered with polling disabled. As soon as I enable polling, ierrs on the em0 interface are rising: em0: Excessive collisions = 0 em0: Symbol errors = 0 em0: Sequence errors = 0 em0: Defer count = 11 em0: Missed Packets = 39 em0: Receive No Buffers = 2458 em0: Receive length errors = 0 em0: Receive errors = 0 em0: Crc errors = 0 em0: Alignment errors = 0 em0: Carrier extension errors = 0 em0: XON Rcvd = 11 em0: XON Xmtd = 4 em0: XOFF Rcvd = 11 em0: XOFF Xmtd = 43 em0: Good Packets Rcvd = 315880003 em0: Good Packets Xmtd = 303985941 em0: Adapter hardware address = 0xc12cfb48 em0:CTRL = 0x58f00249 em0:RCTL = 0x8002 PS=(0x8402) em0:tx_int_delay = 66, tx_abs_int_delay = 66 em0:rx_int_delay = 0, rx_abs_int_delay = 66 em0: fifo workaround = 0, fifo_reset = 0 em0: hw tdh = 57, hw tdt = 57 em0: Num Tx descriptors avail = 249 em0: Tx Descriptors not avail1 = 0 em0: Tx Descriptors not avail2 = 0 em0: Std mbuf failed = 0 em0: Std mbuf cluster failed = 0 em0: Driver dropped packets = 0 Can you tell me what settings you are using for polling? I have set it to HZ=1000 and burst_max=300. I have now noticed another thing which might indicate one of the possible causes for the problem - this box until now ran FreeBSD 4.x and did not support ipfw tables to lock out whole lists of IP adresses. So there were quite a few inefficient rules for this. I now put all the locked IP addresses in a table which is referenced by only one rule. Since I did this, the ierrs seem to rise slower with polling enabled. > Have you tried contacting Intel directly about this? > freebsdnic@mailbox.intel.com has been pretty helpful with em specific > problems in the past. Not yet, thank you for the hint. -- >> Ferdinand Goldmann //// | | >> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at |--00 | UNIX | >> Tel. : +43/732/2468/9398 Fax. : +43/732/2468/9397 C ^ | | >> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at \ ~/ ~~~|~~~~~~~~ >> PGP D4CF 8AA4 4B2A 7B88 65CA 5EDC 0A9B FA9A 13EA B993| |-----3