Date: Mon, 27 Jul 2020 08:36:25 -0400 From: Joe Clarke <jclarke@marcuscom.com> To: Eugene Grosbein <eugen@grosbein.net> Cc: freebsd-stable@freebsd.org Subject: Re: Traffic "corruption" in 12-stable Message-ID: <1C16B9F2-73E2-4B0B-8A69-1A7E060E8D1B@marcuscom.com> In-Reply-To: <9d6dc414-2866-e6c8-6b66-22af23efc728@grosbein.net> References: <9FAE54DE-F409-4A53-B91E-59AE52A86513@marcuscom.com> <9d6dc414-2866-e6c8-6b66-22af23efc728@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Jul 27, 2020, at 01:00, Eugene Grosbein <eugen@grosbein.net> wrote: >=20 > 27.07.2020 5:16, Joe Clarke wrote: >=20 >> About two weeks ago, I upgraded from the latest 11-stable to the = latest 12-stable. After that, I periodically see the network throughput = come to a near standstill. This FreeBSD machine is an ESXi VM with two = interfaces. It acts as a router. It uses vmxnet3 interfaces for both = LAN and WAN. It runs ipfw with in-kernel NAT. The LAN side uses a = bridge with vmx0 and a tap0 L2 VPN interface. My LAN side uses an MTU = of 9000, and my vmx1 (WAN side) uses the default 1500. >>=20 >> Besides seeing massive packet loss and huge latency (~ 200 ms for = on-LAN ping times), I know the problem has occurred because my lldpd = reports: >>=20 >> Jul 26 15:47:03 namale lldpd[1126]: frame too short for tlv received = on bridge0 >>=20 >> And if I turn on ipfw verbose messages, I see tons of: >>=20 >> Jul 26 16:02:23 namale kernel: ipfw: pullup failed >>=20 >> This leads to me to believe packets are being corrupted on ingress. = I=E2=80=99ve applied all the recent iflib changes, but the problem = persists. What causes it, I don=E2=80=99t know. >>=20 >> The only thing that changed (and yes, it=E2=80=99s a big one) is I = upgraded to 12-stable. Meaning, the rest of the network infra and = topology has remained the same. This did not happen at all in = 11-stable. >>=20 >> I=E2=80=99m open to suggestions. >=20 > First, try: ifconfig $ifname -rxcsum -txcsum Thanks for the suggestion. I should have mentioned I=E2=80=99ve been = initializing these two interfaces since 11-stable with: ifconfig_vmx0=3D"up mtu 9000 -tso -lro -vlanhwtso -rxcsum -txcsum = -rxcsum6 -txcsum6 -tso4 -tso6 -vlanhwcsum=E2=80=9D ifconfig_vmx1=3D"DHCP -tso -lro -vlanhwtso -rxcsum -txcsum -rxcsum6 = -txcsum6 -tso4 -tso6 -vlanhwcsum=E2=80=9D And I=E2=80=99m running: FreeBSD namale.marcuscom.com 12.1-STABLE FreeBSD 12.1-STABLE NAMALE = amd64 1201520 1201520 I most recently built this yesterday, but the previous kernel that = exhibited the problem was built about a week ago. It had the fragment = fixes for iflib.c. Joe >=20 --- PGP Key : http://www.marcuscom.com/pgp.asc
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1C16B9F2-73E2-4B0B-8A69-1A7E060E8D1B>