From owner-freebsd-current@FreeBSD.ORG Fri Feb 9 17:30:47 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4882A16A400 for ; Fri, 9 Feb 2007 17:30:47 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id CF9E513C48D for ; Fri, 9 Feb 2007 17:30:46 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1HFZXc-0008Bp-Ez for freebsd-current@freebsd.org; Fri, 09 Feb 2007 18:28:11 +0100 Received: from mulderlab.f5.com ([205.229.151.151]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 09 Feb 2007 18:28:08 +0100 Received: from atkin901 by mulderlab.f5.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 09 Feb 2007 18:28:08 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Mark Atkinson Date: Fri, 09 Feb 2007 09:23:41 -0800 Lines: 105 Message-ID: References: <20070110120731.GA1515@shark.localdomain> <200701100910.13167.jhb@freebsd.org> <20070110155331.GA2762@shark.localdomain> <20070111004044.GA33964@cdnetworks.co.kr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7Bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: mulderlab.f5.com User-Agent: KNode/0.10.4 Sender: news Subject: Re: nve related LOR triggered by lots of small packets, and a hard hang X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Feb 2007 17:30:47 -0000 Mark Atkinson wrote: > Pyun YongHyeon wrote: > >> On Wed, Jan 10, 2007 at 06:53:31PM +0300, Sergey Zaharchenko wrote: >> > Hello John! >> > >> > Wed, Jan 10, 2007 at 09:10:12AM -0500 you wrote: >> > [snip] >> > > Have you tried using nfe(4)? :) >> > >> > Now I have, and it works just fine, thanks (I somehow thought nfe was >> > specific to some platform). Why isn't it the default? Smaller range of >> > hardware supported? >> > >> >> AFAIK, nfe(4) supports more hardwares than that of nve(4). >> Try overhauled nfe(4) in the following URL. >> >> http://people.freebsd.org/~yongari/nfe/if_nfe.c >> http://people.freebsd.org/~yongari/nfe/if_nfereg.h >> http://people.freebsd.org/~yongari/nfe/if_nfevar.h >> >> The patch fixed serveral bugs in nfe(4) and it should perform better >> than nve(4). The following hardware features are supported. >> o TSO >> o Tx/Rx IP/TCP/UDP checksum offload >> o VLAN hardware tag insertion/stripping >> o Jumbo frame(up to 9100 bytes) >> >> It seems that the hardware supports MSI/MSI-X too but I don't have >> nForce hardwares that supports MSI/MSI-X so it's hard to implement/ >> experiment it. Accoring to the Shigeaki Tagashira, the author of >> FreeBSD nfe(4), his hardware claims to support 8 messages. I've >> checked Linux forcedeth driver to get hardware information for >> MSI/MSI-X but it I cound't understand the details. :-( >> > > I've been running into this hardlock LOR a lot recently on a TYAN 2895 > (K8WE) based box. So I tried your patch to nfe on today's -current. I > tried a couple of small packet ping floods to a lan neighbor under nfe and > it survived. Did fine with some large NFS over TCP transfers as well. > However, I'll leave it up and running to see if it keels over in the > future. > > pci128: on pcib6 > pci128: physical bus=128 > found-> vendor=0x10de, dev=0x005e, revid=0xa3 > bus=128, slot=0, func=0 > class=05-80-00, hdrtype=0x00, mfdev=0 > cmdreg=0x0006, statreg=0x00b0, cachelnsz=0 (dwords) > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > found-> vendor=0x10de, dev=0x00d3, revid=0xa3 > bus=128, slot=1, func=0 > class=05-80-00, hdrtype=0x00, mfdev=1 > cmdreg=0x000f, statreg=0x00a0, cachelnsz=0 (dwords) > lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) > map[14]: type 1, range 32, base 0xd8400000, size 12, enabled > found-> vendor=0x10de, dev=0x0057, revid=0xa3 > bus=128, slot=10, func=0 > class=06-80-00, hdrtype=0x00, mfdev=0 > cmdreg=0x0007, statreg=0x00b0, cachelnsz=0 (dwords) > lattimer=0x00 (0 ns), mingnt=0x01 (250 ns), maxlat=0x14 (5000 ns) > intpin=a, irq=5 > powerspec 2 supports D0 D1 D2 D3 current D0 > map[10]: type 1, range 32, base 0xd8401000, size 12, enabled > map[14]: type 4, range 32, base 0x3000, size 3, enabled > pcib6: matched entry for 128.10.INTA (src \\_SB_.PCI1.LMAC:0) > pci_link22: Picked IRQ 52 with weight 0 > ioapic3: Changing polarity for pin 20 to high > pcib6: slot 10 INTA routed to irq 52 via \\_SB_.PCI1.LMAC > found-> vendor=0x10de, dev=0x005d, revid=0xa3 > bus=128, slot=14, func=0 > class=06-04-00, hdrtype=0x01, mfdev=0 > cmdreg=0x0107, statreg=0x0010, cachelnsz=16 (dwords) > lattimer=0x00 (0 ns), mingnt=0x04 (1000 ns), maxlat=0x00 (0 ns) > powerspec 2 supports D0 D3 current D0 > MSI supports 2 messages, 64 bit > pci128: at device 0.0 (no driver attached) > pci128: at device 1.0 (no driver attached) > nfe1: port 0x3000-0x3007 > mem 0xd8 > 401000-0xd8401fff irq 52 at device 10.0 on pci128 > nfe1: Reserved 0x1000 bytes for rid 0x10 type 3 at 0xd8401000 > nfe1: bpf attached > e1: Ethernet address: 00:e0:81:57:d9:af > miibus1: on nfe1 > e1000phy1: PHY 1 on miibus1 > e1000phy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, > 1000baseTX-FDX, auto > ioapic3: routing intpin 20 (PCI IRQ 52) to vector 57 > nfe1: [MPSAFE] > nfe1: [FAST] After a day of running this, it became obvious the nfe driver patch has some sort of issue, at least with -current and this board. Although NFS speeds seemed reasonable, transfers over TCP from a webserver suffered some sort of very noticeable pause/send/pause/send... type problem that reduced transfers to about 6Kbyte/s. This problem went away when putting nve back into the kernel and retrying the same scenerio. -- Mark Atkinson atkin901@yahoo.com (!wired)?(coffee++):(wired);