From owner-freebsd-net@FreeBSD.ORG Tue Mar 25 23:21:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35CF04CD; Tue, 25 Mar 2014 23:21:45 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BA0CC23A; Tue, 25 Mar 2014 23:21:44 +0000 (UTC) Received: from 46-127-132-15.dynamic.hispeed.ch ([46.127.132.15]:55203 helo=[172.16.1.156]) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WSaf6-000BLB-I7; Wed, 26 Mar 2014 00:21:40 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Wed, 26 Mar 2014 00:21:38 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 23:21:45 -0000 On 26.03.2014, at 00:06, Christopher Forgeron = wrote: > Update: >=20 > I'm changing my mind, and I believe Rick's TSO patch is fixing things > (sorry). In looking at my notes, it's possible I had lagg on for those > tests. lagg does seem to negate the TSO patch in my case. I=92m glad to hear you could check that scenario again. In the other = email I just sent, I just asked you to redo this test. Now it makes = perfect sense why you saw oversized packets despite Rick=92s = if_hw_tsomax patch. > kernel.10stable_basicTSO_65535/ >=20 > - IP_MAXPACKET =3D 65535; > - manually forced (no if statement) ifp->if_hw_tsomax =3D IP_MAXPACKET = - > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > - Verified on boot via printf that ifp->if_hw_tsomax =3D 65517 Is 65517 correct? With Ricks patch, I get this: dev.ix.0.hw_tsomax: 65518 Also the dtrace command you used excludes 65518... > - Boot in a NON LAGG environment. ix0 only. >=20 > ixgbe's printf is showing packets up to 65530. Haven't run long enough = yet > to see if anything will go over 65535 >=20 > I have this tcpdump running to check packet size. > tcpdump -ennvvXS -i ix0 greater 65518 >=20 > I do expect to get packets over 65518, but I was just curious to see = if any > of them would go over 65535. Time will tell. >=20 > In a separate test, If I enable lagg, we have LOTS of oversized packet > problems. It looks like tsomax is definitely not making it through in > if_lagg.c - Any recommendations there? I will eventually need lagg, as = I'm > sure will others. I think somebody has to invent a way to propagate if_hw_maxtso to = interfaces on top of each other. > With dtrace, it's showing t_tsomax >=3D 65518. Shouldn't that not be > happening? Looks like these all come from bce interfaces (bce_intr in the stack = trace), which probably have another value for if_hw_tsomax. Markus > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && > args[0]->t_tsomax >=3D 65518 / { printf("unexpected tp->t_tsomax: = %i\n", > args[0]->t_tsomax); stack(); }' >=20 >=20 > 6 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e >=20 > 3 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e >=20 > 6 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e >=20 > 1 31403 tcp_output:entry unexpected tp->t_tsomax: = 65535 >=20 > kernel`tcp_do_segment+0x2c99 > kernel`tcp_input+0x11a2 > kernel`ip_input+0xa2 > kernel`netisr_dispatch_src+0x5e > kernel`ether_demux+0x12a > kernel`ether_nh_input+0x35f > kernel`netisr_dispatch_src+0x5e > kernel`bce_intr+0x765 > kernel`intr_event_execute_handlers+0xab > kernel`ithread_loop+0x96 > kernel`fork_exit+0x9a > kernel`0xffffffff80c75b2e > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20