Date: Tue, 25 Mar 2014 23:27:33 -0300 From: Christopher Forgeron <csforgeron@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Andre Oppermann <andre@freebsd.org>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com>, Markus Gebert <markus.gebert@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <CAB2_NwBZjVYVoPMh4HgNtGoULBuy0gA2=K7K=Bn=zCkjAu=Paw@mail.gmail.com> In-Reply-To: <1985661701.540906.1395789389694.JavaMail.root@uoguelph.ca> References: <CAB2_NwA2zp9VaBm4ZRR1udghbSpSDs_x8bK6CPHCaBuiBKD=KA@mail.gmail.com> <1985661701.540906.1395789389694.JavaMail.root@uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
That's interesting. I see here in the r251296 commit Andre says :
Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to
change the limit.
I wonder if we add your same TSO patch to if_lagg.c before line 356's
ether_ifattach() will fix it.
Ultimately, it will need to load the if_hw_tsomax from the if below it -
but then again, if the calculation for ixgbe is good enough for that
driver, why wouldn't it be good enough for lagg?
Unless people think I'm crazy, I'll compile that in at line 356 in
if_lagg.c and give it a test run tomorrow.
This may need to go into vlan and carp as well, I'm not sure yet.
On Tue, Mar 25, 2014 at 8:16 PM, Rick Macklem <rmacklem@uoguelph.ca> wrote:
> Christopher Forgeron wrote:
> > Update:
> >
> > I'm changing my mind, and I believe Rick's TSO patch is fixing
> > things
> > (sorry). In looking at my notes, it's possible I had lagg on for
> > those
> > tests. lagg does seem to negate the TSO patch in my case.
> >
> Ok, that's useful information. It implies that r251296 doesn't quite
> work and needs to be fixed for "stacked" network interface drivers
> before it can be used. I've cc'd Andre who is the author of that
> patch, in case he knows how to fix it.
>
> Thanks for checking this, rick
>
> > kernel.10stable_basicTSO_65535/
> >
> > - IP_MAXPACKET = 65535;
> > - manually forced (no if statement) ifp->if_hw_tsomax = IP_MAXPACKET
> > -
> > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
> > - Verified on boot via printf that ifp->if_hw_tsomax = 65517
> > - Boot in a NON LAGG environment. ix0 only.
> >
> > ixgbe's printf is showing packets up to 65530. Haven't run long
> > enough yet
> > to see if anything will go over 65535
> >
> > I have this tcpdump running to check packet size.
> > tcpdump -ennvvXS -i ix0 greater 65518
> >
> > I do expect to get packets over 65518, but I was just curious to see
> > if any
> > of them would go over 65535. Time will tell.
> >
> > In a separate test, If I enable lagg, we have LOTS of oversized
> > packet
> > problems. It looks like tsomax is definitely not making it through in
> > if_lagg.c - Any recommendations there? I will eventually need lagg,
> > as I'm
> > sure will others.
> >
> > With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not be
> > happening?
> >
> >
> > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 &&
> > args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax:
> > %i\n",
> > args[0]->t_tsomax); stack(); }'
> >
> >
> > 6 31403 tcp_output:entry unexpected tp->t_tsomax:
> > 65535
> >
> > kernel`tcp_do_segment+0x2c99
> > kernel`tcp_input+0x11a2
> > kernel`ip_input+0xa2
> > kernel`netisr_dispatch_src+0x5e
> > kernel`ether_demux+0x12a
> > kernel`ether_nh_input+0x35f
> > kernel`netisr_dispatch_src+0x5e
> > kernel`bce_intr+0x765
> > kernel`intr_event_execute_handlers+0xab
> > kernel`ithread_loop+0x96
> > kernel`fork_exit+0x9a
> > kernel`0xffffffff80c75b2e
> >
> > 3 31403 tcp_output:entry unexpected tp->t_tsomax:
> > 65535
> >
> > kernel`tcp_do_segment+0x2c99
> > kernel`tcp_input+0x11a2
> > kernel`ip_input+0xa2
> > kernel`netisr_dispatch_src+0x5e
> > kernel`ether_demux+0x12a
> > kernel`ether_nh_input+0x35f
> > kernel`netisr_dispatch_src+0x5e
> > kernel`bce_intr+0x765
> > kernel`intr_event_execute_handlers+0xab
> > kernel`ithread_loop+0x96
> > kernel`fork_exit+0x9a
> > kernel`0xffffffff80c75b2e
> >
> > 6 31403 tcp_output:entry unexpected tp->t_tsomax:
> > 65535
> >
> > kernel`tcp_do_segment+0x2c99
> > kernel`tcp_input+0x11a2
> > kernel`ip_input+0xa2
> > kernel`netisr_dispatch_src+0x5e
> > kernel`ether_demux+0x12a
> > kernel`ether_nh_input+0x35f
> > kernel`netisr_dispatch_src+0x5e
> > kernel`bce_intr+0x765
> > kernel`intr_event_execute_handlers+0xab
> > kernel`ithread_loop+0x96
> > kernel`fork_exit+0x9a
> > kernel`0xffffffff80c75b2e
> >
> > 1 31403 tcp_output:entry unexpected tp->t_tsomax:
> > 65535
> >
> > kernel`tcp_do_segment+0x2c99
> > kernel`tcp_input+0x11a2
> > kernel`ip_input+0xa2
> > kernel`netisr_dispatch_src+0x5e
> > kernel`ether_demux+0x12a
> > kernel`ether_nh_input+0x35f
> > kernel`netisr_dispatch_src+0x5e
> > kernel`bce_intr+0x765
> > kernel`intr_event_execute_handlers+0xab
> > kernel`ithread_loop+0x96
> > kernel`fork_exit+0x9a
> > kernel`0xffffffff80c75b2e
> > _______________________________________________
> > freebsd-net@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to
> > "freebsd-net-unsubscribe@freebsd.org"
> >
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAB2_NwBZjVYVoPMh4HgNtGoULBuy0gA2=K7K=Bn=zCkjAu=Paw>