Date: Wed, 26 Mar 2014 14:44:37 -0300 From: Christopher Forgeron <csforgeron@gmail.com> To: Markus Gebert <markus.gebert@hostpoint.ch> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <CAB2_NwDCLAwR8TXMbgoL366GaNFGrrQShTs%2B3hBpvj8Fy=iT3g@mail.gmail.com> In-Reply-To: <CAB2_NwA-zLnJh7Teqx4eQ-Tkc8bjx2qL0JMZeZqvSzJZ4mx4cg@mail.gmail.com> References: <0BC10908-2081-45AC-A1C8-14220D81EC0A@hostpoint.ch> <1236110257.2510701.1395709458870.JavaMail.root@uoguelph.ca> <CAB2_NwBrdp3XsVrgkWgf7zGsrCBq1%2BC7FgsYvV28yjcUe4qZ=Q@mail.gmail.com> <CAB2_NwA2zp9VaBm4ZRR1udghbSpSDs_x8bK6CPHCaBuiBKD=KA@mail.gmail.com> <1197F2E5-F20C-43E4-B8C8-8732F45457C2@hostpoint.ch> <CAB2_NwA-zLnJh7Teqx4eQ-Tkc8bjx2qL0JMZeZqvSzJZ4mx4cg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Up for almost 19 hours under load without a single error. I would say the
TSO patch does work, now I'm going to run lagg tests.
The more I think of it, the more I wonder if setting tsomax in if.c at line
660 isn't the better idea, like below.
660: if (ifp->if_hw_tsomax == 0)
661: ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN +
ETHER_VLAN_ENCAP_LEN);
I know there are concerns about the impact on various cards, but right now
if.c will set if_hw_tssomax to IP_MAXPACKET, which we know is bad for
ixgbe, and I believe bad for lagg (tests will show) - If the driver isn't
specifically setting it to a different setting, is there harm in limiting
all if's to a default of IP_MAXPACKET - (ETHER_HDR_LEN +
ETHER_VLAN_ENCAP_LEN) if not specified otherwise? When is a TSO of 65535
going to be useful?
I can confirm that with just the TSO patch in ixgbe, and lagg enabled, the
problem still exists. Last night's tests never went above a packet of
65530. Now with lagg enabled, I'm seeing packets of 65543 within 5 minutes,
so we're already breaking.
On Tue, Mar 25, 2014 at 11:33 PM, Christopher Forgeron <csforgeron@gmail.com
> wrote:
>
>
> On Tue, Mar 25, 2014 at 8:21 PM, Markus Gebert <markus.gebert@hostpoint.ch
> > wrote:
>
>>
>>
>> Is 65517 correct? With Ricks patch, I get this:
>>
>> dev.ix.0.hw_tsomax: 65518
>>
>
> Perhaps a difference between 9.2 and 10 for one of the macros? My code is:
>
>
> ifp->if_hw_tsomax = IP_MAXPACKET - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
> printf("CSF - 3 Init, ifp->if_hw_tsomax = %d\n",
> ifp->if_hw_tsomax);
>
> (BTW, you should submit the hw_tsomax sysctl patch, that's useful to
> others)
>
>
>> Also the dtrace command you used excludes 65518...
>>
>
> Oh, I thought it was giving every packet that is greater than or equal to
> 65518 - Could you show me the proper command? That's the third time I've
> used dtrace, so I'm making this up as I go. :-)
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB2_NwDCLAwR8TXMbgoL366GaNFGrrQShTs%2B3hBpvj8Fy=iT3g>