Date: Mon, 24 Mar 2014 12:21:26 -0300 From: Christopher Forgeron <csforgeron@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com>, Markus Gebert <markus.gebert@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <CAB2_NwDKkgTfNuapm2gA5xhuBgVK6jE2uHwb2Nu-vsRvw_NwKQ@mail.gmail.com> In-Reply-To: <CAB2_NwCHM9D1HZSMsuQQ-dYNAt-t2721jKqfO=2h3M4qdumY7w@mail.gmail.com> References: <CAB2_NwAcDPM6YKNLQMC0=YSp%2Bn9nBpXGJQR9ajbgbfcQFoWYPw@mail.gmail.com> <1164414873.1690348.1395622026185.JavaMail.root@uoguelph.ca> <CAB2_NwAbHzFqa8RM5pwV7Yy5t=96JwzaF%2BSdjJN9kK3uhKKn_w@mail.gmail.com> <CAB2_NwCHM9D1HZSMsuQQ-dYNAt-t2721jKqfO=2h3M4qdumY7w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is regarding the TSO patch that Rick suggested earlier. (With many
thanks for his time and suggestion)
As I mentioned earlier, it did not fix the issue on a 10.0 system. It did
make it less of a problem on 9.2, but either way, I think it's not needed,
and shouldn't be considered as a patch for testing/etc.
Patching TSO to anything other than a max value (and by default the code
gives it IP_MAXPACKET) is confusing the matter, as the packet length
ultimately needs to be adjusted for many things on the fly like TCP
Options, etc. Using static header sizes won't be a good idea.
Additionally, it seems that setting nic TSO will/may be ignored by code
like this in sys/netinet/tcp_output.c:
10.0 Code:
780 if (len > tp->t_tsomax - hdrlen)
{ !!
781 len = tp->t_tsomax -
hdrlen; !!
782 sendalot =
1;
783 }
I've put debugging here, set the nic's max TSO as per Rick's patch ( set to
say 32k), and have seen that tp->t_tsomax == IP_MAXPACKET. It's being set
someplace else, and thus our attempts to set TSO on the nic may be in vain.
It may have mattered more in 9.2, as I see the code doesn't use
tp->t_tsomax in some locations, and may actually default to what the nic is
set to.
The NIC may still win, I didn't walk through the code to confirm, it was
enough to suggest to me that setting TSO wouldn't fix this issue.
However, this is still a TSO related issue, it's just not one related to
the setting of TSO's max size.
A 10.0-STABLE system with tso disabled on ix0 doesn't have a single packet
over IP_MAXPACKET in 1 hour of runtime. I'll let it go a bit longer to
increase confidence in this assertion, but I don't want to waste time on
this when I could be logging problem packets on a system with TSO enabled.
Comments are very welcome..
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB2_NwDKkgTfNuapm2gA5xhuBgVK6jE2uHwb2Nu-vsRvw_NwKQ>