Date: Thu, 27 Mar 2014 15:13:57 +0100 From: Markus Gebert <markus.gebert@hostpoint.ch> To: Christopher Forgeron <csforgeron@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <5C2A730F-47B6-4E1D-8DA3-276E48DEA810@hostpoint.ch> In-Reply-To: <CAB2_NwA-zLnJh7Teqx4eQ-Tkc8bjx2qL0JMZeZqvSzJZ4mx4cg@mail.gmail.com> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <CAB2_NwBrdp3XsVrgkWgf7zGsrCBq1%2BC7FgsYvV28yjcUe4qZ=Q@mail.gmail.com> <CAB2_NwA2zp9VaBm4ZRR1udghbSpSDs_x8bK6CPHCaBuiBKD=KA@mail.gmail.com> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> <CAB2_NwA-zLnJh7Teqx4eQ-Tkc8bjx2qL0JMZeZqvSzJZ4mx4cg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 26.03.2014, at 03:33, Christopher Forgeron <csforgeron@gmail.com> wrote:
> On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert
> <markus.gebert@hostpoint.ch>wrote:
>
>>
>>
>> Is 65517 correct? With Ricks patch, I get this:
>>
>> dev.ix.0.hw_tsomax: 65518
>>
>
> Perhaps a difference between 9.2 and 10 for one of the macros? My code is:
>
> ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
> printf("CSF - 3 Init, ifp->if_hw_tsomax = %d\n", ifp->if_hw_tsomax);
Hm, I’m using Rick’s patch:
if ((adapter->num_segs * MCLBYTES - (ETHER_HDR_LEN +
ETHER_VLAN_ENCAP_LEN)) < IP_MAXPACKET)
ifp->if_hw_tsomax = adapter->num_segs * MCLBYTES -
(ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
> (BTW, you should submit the hw_tsomax sysctl patch, that's useful to others)
My patch added a sysctl that is writable, but if I got this right if_hw_tsomax is not expected to change after the interface is attached. That’s why I didn’t post it. But here’s a read-only version:
--- sys/dev/ixgbe/ixgbe.c 2013-12-19 14:24:10.624279412 +0100
+++ sys/dev/ixgbe/ixgbe.c 2014-03-27 15:00:59.503424634 +0100
@@ -577,6 +582,12 @@
if (ixgbe_setup_interface(dev, adapter) != 0)
goto err_late;
+ /* add interface to hw_tsomax */
+ SYSCTL_ADD_INT(device_get_sysctl_ctx(dev),
+ SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
+ OID_AUTO, "hw_tsomax", CTLTYPE_INT|CTLFLAG_RD,
+ &adapter->ifp->if_hw_tsomax, 1, "hardware TSO limit");
+
/* Initialize statistics */
ixgbe_update_stats_counters(adapter);
>> Also the dtrace command you used excludes 65518...
>>
>
> Oh, I thought it was giving every packet that is greater than or equal to
> 65518 - Could you show me the proper command? That's the third time I've
> used dtrace, so I'm making this up as I go. :-)
No, what looks like a comment (between slashes) are conditions in dtrace:
dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 && args[0]->t_tsomax != 65518 / { printf("unexpected tp->t_tsomax: %i\n", args[0]->t_tsomax); stack(); }’
You have to read the above like this:
- fbt::tcp_output:entry -> Add a probe to the beginning of the kernel function tcp_output()
- / args[0]->t_tsomax != 0 && args[0]->t_tsomax != 65518 / -> only match if t_tsomax is neither 0 nor 65518 (args[0] is struct tcpcb in case of tcp_output())
- { printf("unexpected tp->t_tsomax: %i\n", args[0]->t_tsomax); stack(); } -> this is only executed if the probe matched and the condition were true. It that case a t_tsomax gets printed and a stack trace is generated
In your case, you stated that your if_hw_tsomax is 65517. Since my version of the dtrace one-liner does _not_ ignore 65517, you should have seen a lot of output, which you didn’t mention (you’ve just posted dtrace output that was generated from bce interfaces). That’s why I thought 65517 was a typo on your part, and I wanted to clarify that.
Markus
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5C2A730F-47B6-4E1D-8DA3-276E48DEA810>