Date: Sun, 23 Mar 2014 17:28:02 -0300 From: Christopher Forgeron <csforgeron@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com>, Markus Gebert <markus.gebert@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <CAB2_NwAcDPM6YKNLQMC0=YSp%2Bn9nBpXGJQR9ajbgbfcQFoWYPw@mail.gmail.com> In-Reply-To: <CAB2_NwBBvhQOkNFwj9RdwHTB5DT5OTmczR8UgsciS0UP2LkrdQ@mail.gmail.com> References: <CAB2_NwADUfs%2BbKV9QE_C4B1vchnzGWr1TK4C7wP8Fh8m94_mHA@mail.gmail.com> <1532337187.1452820.1395543491196.JavaMail.root@uoguelph.ca> <CAB2_NwBBvhQOkNFwj9RdwHTB5DT5OTmczR8UgsciS0UP2LkrdQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Update: For giggles, I set IP_MAXPACKET = 32768. Over a hour of runtime, and no issues. This is better than with the TSO patch and the 9.2 ixgbe, as that was just a drastic reduction in errors. Still have an 'angry' netstat -m on boot, and I'm still incrementing denied netbuf calls, so something else is wrong. I'm going to modify Rick's prinft in ixgbe to also output when we're over 32768. I'm sure it's still happening, but with an extra 32k of space, we're not busting like we did before. I notice a few interesting ip->ip_len changes since 9.2 - Like here, at line 720 http://fxr.watson.org/fxr/diff/netinet/ip_output.c?v=FREEBSD10;im=kwqeqdhhvovqn;diffval=FREEBSD92;diffvar=v Looks like older code didn't byteswap with ntohs - I see that often in tcp_output.c, and in tcp_options.c. I'm also curious about this:Line 524 http://fxr.watson.org/fxr/diff/netinet/ip_options.c?v=FREEBSD10;diffval=FREEBSD92;diffvar=v New 10 code: ip <http://fxr.watson.org/fxr/ident?v=FREEBSD10;im=kwqeqdhhvovqn;i=ip>->ip_len = htons <http://fxr.watson.org/fxr/ident?v=FREEBSD10;im=kwqeqdhhvovqn;i=htons>(ntohs <http://fxr.watson.org/fxr/ident?v=FREEBSD10;im=kwqeqdhhvovqn;i=ntohs>(ip <http://fxr.watson.org/fxr/ident?v=FREEBSD10;im=kwqeqdhhvovqn;i=ip>->ip_len) + optlen); Old 9.2 Code: ip <http://fxr.watson.org/fxr/ident?v=FREEBSD92;i=ip>->ip_len += optlen; I wonder if there are any unexpected consequences of these changes, or perhaps a line someplace that doesn't make the change. Is there a dtrace command I could use to watch these functions and compare the new ip_len with ip->ip_len or other variables? On Sun, Mar 23, 2014 at 12:25 PM, Christopher Forgeron <csforgeron@gmail.com > wrote: > > On Sat, Mar 22, 2014 at 11:58 PM, Rick Macklem <rmacklem@uoguelph.ca>wrote: > >> Christopher Forgeron wrote: >> > >> >> > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax to >> > make sure VLANs fit? >> > >> I took a look and, yes, this does seem to be needed. It will only be >> needed for the case where a vlan is in use and hwtagging is disabled, >> if I read the code correctly. >> > > Yes, or in the rare care where you configure your switch to pass the v_lan > header through to the NIC. > >> >> Do you use vlans? >> > > (Answered in above email) > > >> >> I've attached an updated patch. >> >> It might be nice to have the printf() patch in the driver too, so >> we can see how big the ones that are too big are? >> >> > Yes, I'm going to leave those in until I know we have this fixed.. will > probably leave it in a while longer as it should only have a minor > performance impact to iter-loop like that, and I'd like to see what the > story is a few months down the road. > > Thanks for the patches, will have to start giving them code-names so we > can keep them straight. :-) I guess we have printf, tsomax, and this one. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB2_NwAcDPM6YKNLQMC0=YSp%2Bn9nBpXGJQR9ajbgbfcQFoWYPw>