From owner-freebsd-net@FreeBSD.ORG Tue May 25 15:58:50 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 054E01065745 for ; Tue, 25 May 2010 15:58:50 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ww0-f54.google.com (mail-ww0-f54.google.com [74.125.82.54]) by mx1.freebsd.org (Postfix) with ESMTP id 78A228FC19 for ; Tue, 25 May 2010 15:58:46 +0000 (UTC) Received: by wwd20 with SMTP id 20so214666wwd.13 for ; Tue, 25 May 2010 08:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=EEg2tnYXoxe58RStedWbZSuGuLdy/ZWJt1JkBnU1HJ0=; b=hb7X8bpUgAVjNlOpdhX3PyaCh/4/A7A6e7jO1WQYhC+9or7jxiTIoaPfQiRrJ8i0kV Qc4Ux+zIdxUzsoIZ9g8Mkkn2ByAX1s3dNJHzYdkkEp8D8tB3iFKMTEX0+z1lwano+8uz gv9OYqPG1VKZw/exG4OAdMkr1zRuv2f+OUxag= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=RBr5C+t0J+vGyXTnDWvJdwqZvCAF/fGGFNCQXa557uXvjq0B3LRsrkoJQmysPBF0y0 UeTjcUYeI1pqjL9wcx1ShT3IrcPv7ewohsduSRkEkr4N2OCa/nTLFw8VfkFlLwzZoOPK IFA+b+YZuxgSExnsb8A8zgz3nMdYRRiyPPSIc= MIME-Version: 1.0 Received: by 10.216.87.204 with SMTP id y54mr4659217wee.142.1274803125763; Tue, 25 May 2010 08:58:45 -0700 (PDT) Received: by 10.216.29.129 with HTTP; Tue, 25 May 2010 08:58:44 -0700 (PDT) In-Reply-To: <20100525154906.GA298@rvdp.org> References: <20100524111655.GG22200@rvdp.org> <20100525154906.GA298@rvdp.org> Date: Tue, 25 May 2010 08:58:44 -0700 Message-ID: From: Jack Vogel To: Ronald van der Pol Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: 8.1-PRERELEASE 10GE ix driver panics when setting mtu > 1500 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 15:58:50 -0000 Hmmm, the driver was tested before the MFC and was performing great, what exactly is your hardware: pciconf -l Jack On Tue, May 25, 2010 at 8:49 AM, Ronald van der Pol < Ronald.vanderPol@rvdp.org> wrote: > On Mon, May 24, 2010 at 10:22:25 -0700, Jack Vogel wrote: > > > Try the driver in HEAD, I am going to MFC that shortly anyway. > > We are using 8.1-PRERELEASE now with the 2.2.0 ixgbe driver and > have several problems. The setup is: > > FreeBSD 8.1-PRERELEASE ixgbe --- UTP -- Arista 7120T -- UTP -- 10G Linux > <========================================== > > receiver: iperf -s -u -B 10.0.1.1 -l 63K -w200K > sender: iperf -u -i1 -t 120 -c 10.0.1.1 -l 63K -w200K -b 10000M > > MTU is 9000 on all interfaces. Flow control is on. > > This is /var/log/messages on the receiving FreeBSD 8.1-PRERELEASE server: > > May 25 15:06:19 flits kernel: ix0: Total Packets Rcvd = 17705828 > May 25 15:06:19 flits kernel: ix0: Good Packets Rcvd = 4816 > May 25 15:06:19 flits kernel: ix0: Good Packets Xmtd = 64 > May 25 15:06:19 flits kernel: ix0: TSO Transmissions = 0 > May 25 15:06:20 flits kernel: ix0: Std Mbuf Failed = 0 > May 25 15:06:20 flits kernel: ix0: Missed Packets = 17880814 > May 25 15:06:20 flits kernel: ix0: Receive length errors = 0 > May 25 15:06:20 flits kernel: ix0: Crc errors = 0 > May 25 15:06:20 flits kernel: ix0: Driver dropped packets = 0 > May 25 15:06:20 flits kernel: ix0: watchdog timeouts = 0 > May 25 15:06:20 flits kernel: ix0: XON Rcvd = 0 > May 25 15:06:20 flits kernel: ix0: XON Xmtd = 0 > May 25 15:06:20 flits kernel: ix0: XOFF Rcvd = 0 > May 25 15:06:20 flits kernel: ix0: XOFF Xmtd = 0 > May 25 15:06:20 flits kernel: ix0: Total Packets Rcvd = 17888305 > May 25 15:06:20 flits kernel: ix0: Good Packets Rcvd = 4817 > May 25 15:06:20 flits kernel: ix0: Good Packets Xmtd = 64 > May 25 15:06:20 flits kernel: ix0: TSO Transmissions = 0 > May 25 15:06:22 flits kernel: ix0: Std Mbuf Failed = 0 > May 25 15:06:22 flits kernel: ix0: Missed Packets = 18063137 > May 25 15:06:22 flits kernel: ix0: Receive length errors = 0 > May 25 15:06:22 flits kernel: ix0: Crc errors = 0 > May 25 15:06:22 flits kernel: ix0: Driver dropped packets = 0 > May 25 15:06:22 flits kernel: ix0: watchdog timeouts = 0 > May 25 15:06:22 flits kernel: ix0: XON Rcvd = 0 > May 25 15:06:22 flits kernel: ix0: XON Xmtd = 0 > May 25 15:06:22 flits kernel: ix0: XOFF Rcvd = 0 > May 25 15:06:22 flits kernel: ix0: XOFF Xmtd = 0 > May 25 15:06:22 flits kernel: ix0: Total Packets Rcvd = 18070630 > May 25 15:06:22 flits kernel: ix0: Good Packets Rcvd = 4818 > May 25 15:06:22 flits kernel: ix0: Good Packets Xmtd = 64 > May 25 15:06:22 flits kernel: ix0: TSO Transmissions = 0 > May 25 15:06:23 flits kernel: ix0: Std Mbuf Failed = 0 > May 25 15:06:23 flits kernel: ix0: Missed Packets = 18245600 > May 25 15:06:23 flits kernel: ix0: Receive length errors = 0 > May 25 15:06:23 flits kernel: ix0: Crc errors = 0 > May 25 15:06:23 flits kernel: ix0: Driver dropped packets = 0 > May 25 15:06:23 flits kernel: ix0: watchdog timeouts = 0 > May 25 15:06:23 flits kernel: ix0: XON Rcvd = 0 > May 25 15:06:23 flits kernel: ix0: XON Xmtd = 0 > > So, it's receiving 182477 packets per second, only 1 good packet and > 182323 missed packets (btw, this does not count up correctly). > > What could be a reason for missed packets? It seems to be a counter in > the NIC? > > After half a minute or so, the ix0 interface stops responding. > ifconfig ix0 down/ifconfig ix0 up makes it reachable again. > > How can I debug this further? Any other statistics? > > I set ixgbe_display_debug_stats to 1 in ixgbe.c and recompiled and > installed the kernel. Is there a better way to get these statistics? > > The server is a Dell PowerEdge T610. > > May 25 13:41:30 flits kernel: ix0: Driver, Version - 2.2.0> port 0xece0-0xecff mem > 0xdf1a0000-0xdf1bffff,0xdf1c0000-0xdf > 1fffff,0xdf19c000-0xdf19ffff irq 34 at device 0.0 on pci5 > May 25 13:41:30 flits kernel: ix0: Using MSIX interrupts with 9 vectors > May 25 13:41:30 flits kernel: ix0: [ITHREAD] > May 25 13:41:30 flits last message repeated 8 times > May 25 13:41:30 flits kernel: ix0: Ethernet address: 00:1b:21:49:56:61 > May 25 13:41:30 flits kernel: ix0: PCI Express Bus: Speed 2.5Gb/s Width x8 > > ix0: flags=8843 metric 0 mtu 9000 > > options=5bb > ether 00:1b:21:49:56:61 > inet 10.0.1.1 netmask 0xffffff00 broadcast 10.0.1.255 > media: Ethernet autoselect (10Gbase-SR ) > status: active > > rvdp >