Date: Thu, 27 Mar 2014 15:13:57 +0100 From: Markus Gebert <markus.gebert@hostpoint.ch> To: Christopher Forgeron <csforgeron@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <5C2A730F-47B6-4E1D-8DA3-276E48DEA810@hostpoint.ch> In-Reply-To: <CAB2_NwA-zLnJh7Teqx4eQ-Tkc8bjx2qL0JMZeZqvSzJZ4mx4cg@mail.gmail.com> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <CAB2_NwBrdp3XsVrgkWgf7zGsrCBq1%2BC7FgsYvV28yjcUe4qZ=Q@mail.gmail.com> <CAB2_NwA2zp9VaBm4ZRR1udghbSpSDs_x8bK6CPHCaBuiBKD=KA@mail.gmail.com> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> <CAB2_NwA-zLnJh7Teqx4eQ-Tkc8bjx2qL0JMZeZqvSzJZ4mx4cg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 26.03.2014, at 03:33, Christopher Forgeron <csforgeron@gmail.com> =
wrote:
> On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert
> <markus.gebert@hostpoint.ch>wrote:
>=20
>>=20
>>=20
>> Is 65517 correct? With Ricks patch, I get this:
>>=20
>> dev.ix.0.hw_tsomax: 65518
>>=20
>=20
> Perhaps a difference between 9.2 and 10 for one of the macros? My =
code is:
>=20
> ifp->if_hw_tsomax =3D IP_MAXPACKET - (ETHER_HDR_LEN + =
ETHER_VLAN_ENCAP_LEN);
> printf("CSF - 3 Init, ifp->if_hw_tsomax =3D %d\n", =
ifp->if_hw_tsomax);
Hm, I=92m using Rick=92s patch:
if ((adapter->num_segs * MCLBYTES - (ETHER_HDR_LEN +
ETHER_VLAN_ENCAP_LEN)) < IP_MAXPACKET)
ifp->if_hw_tsomax =3D adapter->num_segs * =
MCLBYTES -
(ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
> (BTW, you should submit the hw_tsomax sysctl patch, that's useful to =
others)
My patch added a sysctl that is writable, but if I got this right =
if_hw_tsomax is not expected to change after the interface is attached. =
That=92s why I didn=92t post it. But here=92s a read-only version:
--- sys/dev/ixgbe/ixgbe.c 2013-12-19 14:24:10.624279412 +0100
+++ sys/dev/ixgbe/ixgbe.c 2014-03-27 15:00:59.503424634 +0100
@@ -577,6 +582,12 @@
if (ixgbe_setup_interface(dev, adapter) !=3D 0)
goto err_late;
=20
+ /* add interface to hw_tsomax */
+ SYSCTL_ADD_INT(device_get_sysctl_ctx(dev),
+ SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
+ OID_AUTO, "hw_tsomax", CTLTYPE_INT|CTLFLAG_RD,
+ &adapter->ifp->if_hw_tsomax, 1, "hardware TSO limit");
+
/* Initialize statistics */
ixgbe_update_stats_counters(adapter);
=20
>> Also the dtrace command you used excludes 65518...
>>=20
>=20
> Oh, I thought it was giving every packet that is greater than or equal =
to
> 65518 - Could you show me the proper command? That's the third time =
I've
> used dtrace, so I'm making this up as I go. :-)
No, what looks like a comment (between slashes) are conditions in =
dtrace:
dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax !=3D 0 && =
args[0]->t_tsomax !=3D 65518 / { printf("unexpected tp->t_tsomax: %i\n", =
args[0]->t_tsomax); stack(); }=92
You have to read the above like this:
- fbt::tcp_output:entry -> Add a probe to the beginning of the kernel =
function tcp_output()
- / args[0]->t_tsomax !=3D 0 && args[0]->t_tsomax !=3D 65518 / -> only =
match if t_tsomax is neither 0 nor 65518 (args[0] is struct tcpcb in =
case of tcp_output())
- { printf("unexpected tp->t_tsomax: %i\n", args[0]->t_tsomax); stack(); =
} -> this is only executed if the probe matched and the condition were =
true. It that case a t_tsomax gets printed and a stack trace is =
generated
In your case, you stated that your if_hw_tsomax is 65517. Since my =
version of the dtrace one-liner does _not_ ignore 65517, you should have =
seen a lot of output, which you didn=92t mention (you=92ve just posted =
dtrace output that was generated from bce interfaces). That=92s why I =
thought 65517 was a typo on your part, and I wanted to clarify that.
Markus
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5C2A730F-47B6-4E1D-8DA3-276E48DEA810>