From owner-freebsd-net@FreeBSD.ORG Tue May 25 15:49:08 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B9841065673 for ; Tue, 25 May 2010 15:49:08 +0000 (UTC) (envelope-from rvdp@rvdp.org) Received: from kirk.rvdp.org (kirk.rvdp.org [IPv6:2001:610:108:87::2]) by mx1.freebsd.org (Postfix) with ESMTP id CA9808FC20 for ; Tue, 25 May 2010 15:49:07 +0000 (UTC) Received: from kirk.rvdp.org (localhost [127.0.0.1]) by kirk.rvdp.org (8.14.3/8.14.3) with ESMTP id o4PFn6wx003281; Tue, 25 May 2010 17:49:06 +0200 (CEST) (envelope-from rvdp@kirk.rvdp.org) Received: (from rvdp@localhost) by kirk.rvdp.org (8.14.3/8.14.3/Submit) id o4PFn6iO003280; Tue, 25 May 2010 17:49:06 +0200 (CEST) (envelope-from rvdp) Date: Tue, 25 May 2010 17:49:06 +0200 From: Ronald van der Pol To: Jack Vogel Message-ID: <20100525154906.GA298@rvdp.org> References: <20100524111655.GG22200@rvdp.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-net@freebsd.org Subject: Re: 8.1-PRERELEASE 10GE ix driver panics when setting mtu > 1500 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2010 15:49:08 -0000 On Mon, May 24, 2010 at 10:22:25 -0700, Jack Vogel wrote: > Try the driver in HEAD, I am going to MFC that shortly anyway. We are using 8.1-PRERELEASE now with the 2.2.0 ixgbe driver and have several problems. The setup is: FreeBSD 8.1-PRERELEASE ixgbe --- UTP -- Arista 7120T -- UTP -- 10G Linux <========================================== receiver: iperf -s -u -B 10.0.1.1 -l 63K -w200K sender: iperf -u -i1 -t 120 -c 10.0.1.1 -l 63K -w200K -b 10000M MTU is 9000 on all interfaces. Flow control is on. This is /var/log/messages on the receiving FreeBSD 8.1-PRERELEASE server: May 25 15:06:19 flits kernel: ix0: Total Packets Rcvd = 17705828 May 25 15:06:19 flits kernel: ix0: Good Packets Rcvd = 4816 May 25 15:06:19 flits kernel: ix0: Good Packets Xmtd = 64 May 25 15:06:19 flits kernel: ix0: TSO Transmissions = 0 May 25 15:06:20 flits kernel: ix0: Std Mbuf Failed = 0 May 25 15:06:20 flits kernel: ix0: Missed Packets = 17880814 May 25 15:06:20 flits kernel: ix0: Receive length errors = 0 May 25 15:06:20 flits kernel: ix0: Crc errors = 0 May 25 15:06:20 flits kernel: ix0: Driver dropped packets = 0 May 25 15:06:20 flits kernel: ix0: watchdog timeouts = 0 May 25 15:06:20 flits kernel: ix0: XON Rcvd = 0 May 25 15:06:20 flits kernel: ix0: XON Xmtd = 0 May 25 15:06:20 flits kernel: ix0: XOFF Rcvd = 0 May 25 15:06:20 flits kernel: ix0: XOFF Xmtd = 0 May 25 15:06:20 flits kernel: ix0: Total Packets Rcvd = 17888305 May 25 15:06:20 flits kernel: ix0: Good Packets Rcvd = 4817 May 25 15:06:20 flits kernel: ix0: Good Packets Xmtd = 64 May 25 15:06:20 flits kernel: ix0: TSO Transmissions = 0 May 25 15:06:22 flits kernel: ix0: Std Mbuf Failed = 0 May 25 15:06:22 flits kernel: ix0: Missed Packets = 18063137 May 25 15:06:22 flits kernel: ix0: Receive length errors = 0 May 25 15:06:22 flits kernel: ix0: Crc errors = 0 May 25 15:06:22 flits kernel: ix0: Driver dropped packets = 0 May 25 15:06:22 flits kernel: ix0: watchdog timeouts = 0 May 25 15:06:22 flits kernel: ix0: XON Rcvd = 0 May 25 15:06:22 flits kernel: ix0: XON Xmtd = 0 May 25 15:06:22 flits kernel: ix0: XOFF Rcvd = 0 May 25 15:06:22 flits kernel: ix0: XOFF Xmtd = 0 May 25 15:06:22 flits kernel: ix0: Total Packets Rcvd = 18070630 May 25 15:06:22 flits kernel: ix0: Good Packets Rcvd = 4818 May 25 15:06:22 flits kernel: ix0: Good Packets Xmtd = 64 May 25 15:06:22 flits kernel: ix0: TSO Transmissions = 0 May 25 15:06:23 flits kernel: ix0: Std Mbuf Failed = 0 May 25 15:06:23 flits kernel: ix0: Missed Packets = 18245600 May 25 15:06:23 flits kernel: ix0: Receive length errors = 0 May 25 15:06:23 flits kernel: ix0: Crc errors = 0 May 25 15:06:23 flits kernel: ix0: Driver dropped packets = 0 May 25 15:06:23 flits kernel: ix0: watchdog timeouts = 0 May 25 15:06:23 flits kernel: ix0: XON Rcvd = 0 May 25 15:06:23 flits kernel: ix0: XON Xmtd = 0 So, it's receiving 182477 packets per second, only 1 good packet and 182323 missed packets (btw, this does not count up correctly). What could be a reason for missed packets? It seems to be a counter in the NIC? After half a minute or so, the ix0 interface stops responding. ifconfig ix0 down/ifconfig ix0 up makes it reachable again. How can I debug this further? Any other statistics? I set ixgbe_display_debug_stats to 1 in ixgbe.c and recompiled and installed the kernel. Is there a better way to get these statistics? The server is a Dell PowerEdge T610. May 25 13:41:30 flits kernel: ix0: port 0xece0-0xecff mem 0xdf1a0000-0xdf1bffff,0xdf1c0000-0xdf 1fffff,0xdf19c000-0xdf19ffff irq 34 at device 0.0 on pci5 May 25 13:41:30 flits kernel: ix0: Using MSIX interrupts with 9 vectors May 25 13:41:30 flits kernel: ix0: [ITHREAD] May 25 13:41:30 flits last message repeated 8 times May 25 13:41:30 flits kernel: ix0: Ethernet address: 00:1b:21:49:56:61 May 25 13:41:30 flits kernel: ix0: PCI Express Bus: Speed 2.5Gb/s Width x8 ix0: flags=8843 metric 0 mtu 9000 options=5bb ether 00:1b:21:49:56:61 inet 10.0.1.1 netmask 0xffffff00 broadcast 10.0.1.255 media: Ethernet autoselect (10Gbase-SR ) status: active rvdp