From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 14:14:06 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A214C2FD; Thu, 27 Mar 2014 14:14:06 +0000 (UTC) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5C4E169C; Thu, 27 Mar 2014 14:14:06 +0000 (UTC) Received: from [2001:1620:2013:1:2c52:e467:d98f:82e8] (port=56683) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WTB4E-000H5F-2O; Thu, 27 Mar 2014 15:14:02 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: 9.2 ixgbe tx queue hang From: Markus Gebert In-Reply-To: Date: Thu, 27 Mar 2014 15:13:57 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5C2A730F-47B6-4E1D-8DA3-276E48DEA810@hostpoint.ch> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> To: Christopher Forgeron X-Mailer: Apple Mail (2.1874) Cc: FreeBSD Net , Rick Macklem , Garrett Wollman , Jack Vogel X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 14:14:06 -0000 On 26.03.2014, at 03:33, Christopher Forgeron = wrote: > On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert > wrote: >=20 >>=20 >>=20 >> Is 65517 correct? With Ricks patch, I get this: >>=20 >> dev.ix.0.hw_tsomax: 65518 >>=20 >=20 > Perhaps a difference between 9.2 and 10 for one of the macros? My = code is: >=20 > ifp->if_hw_tsomax =3D IP_MAXPACKET - (ETHER_HDR_LEN + = ETHER_VLAN_ENCAP_LEN); > printf("CSF - 3 Init, ifp->if_hw_tsomax =3D %d\n", = ifp->if_hw_tsomax); Hm, I=92m using Rick=92s patch: if ((adapter->num_segs * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN)) < IP_MAXPACKET) ifp->if_hw_tsomax =3D adapter->num_segs * = MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); > (BTW, you should submit the hw_tsomax sysctl patch, that's useful to = others) My patch added a sysctl that is writable, but if I got this right = if_hw_tsomax is not expected to change after the interface is attached. = That=92s why I didn=92t post it. But here=92s a read-only version: --- sys/dev/ixgbe/ixgbe.c 2013-12-19 14:24:10.624279412 +0100 +++ sys/dev/ixgbe/ixgbe.c 2014-03-27 15:00:59.503424634 +0100 @@ -577,6 +582,12 @@ if (ixgbe_setup_interface(dev, adapter) !=3D 0) goto err_late; =20 + /* add interface to hw_tsomax */ + SYSCTL_ADD_INT(device_get_sysctl_ctx(dev), + SYSCTL_CHILDREN(device_get_sysctl_tree(dev)), + OID_AUTO, "hw_tsomax", CTLTYPE_INT|CTLFLAG_RD, + &adapter->ifp->if_hw_tsomax, 1, "hardware TSO limit"); + /* Initialize statistics */ ixgbe_update_stats_counters(adapter); =20 >> Also the dtrace command you used excludes 65518... >>=20 >=20 > Oh, I thought it was giving every packet that is greater than or equal = to > 65518 - Could you show me the proper command? That's the third time = I've > used dtrace, so I'm making this up as I go. :-) No, what looks like a comment (between slashes) are conditions in = dtrace: dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && = args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: %i\n", = args[0]->t_tsomax); stack(); }=92 You have to read the above like this: - fbt::tcp_output:entry -> Add a probe to the beginning of the kernel = function tcp_output() - / args[0]->t_tsomax !=3D 0 && args[0]->t_tsomax !=3D 65518 / -> only = match if t_tsomax is neither 0 nor 65518 (args[0] is struct tcpcb in = case of tcp_output()) - { printf("unexpected tp->t_tsomax: %i\n", args[0]->t_tsomax); stack(); = } -> this is only executed if the probe matched and the condition were = true. It that case a t_tsomax gets printed and a stack trace is = generated In your case, you stated that your if_hw_tsomax is 65517. Since my = version of the dtrace one-liner does _not_ ignore 65517, you should have = seen a lot of output, which you didn=92t mention (you=92ve just posted = dtrace output that was generated from bce interfaces). That=92s why I = thought 65517 was a typo on your part, and I wanted to clarify that. Markus