From owner-freebsd-stable@freebsd.org Mon Jul 27 07:15:36 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 76772378F36 for ; Mon, 27 Jul 2020 07:15:36 +0000 (UTC) (envelope-from graham@menhennitt.com.au) Received: from bumble.birch.relay.mailchannels.net (bumble.birch.relay.mailchannels.net [23.83.209.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BFWKz0gRpz4Z4d for ; Mon, 27 Jul 2020 07:15:34 +0000 (UTC) (envelope-from graham@menhennitt.com.au) X-Sender-Id: dreamhost|x-authsender|graham@menhennitt.com.au Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 54DD440198E for ; Mon, 27 Jul 2020 07:15:32 +0000 (UTC) Received: from pdx1-sub0-mail-a39.g.dreamhost.com (100-96-8-55.trex.outbound.svc.cluster.local [100.96.8.55]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id D40DE401961 for ; Mon, 27 Jul 2020 07:15:29 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|graham@menhennitt.com.au Received: from pdx1-sub0-mail-a39.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.18.8); Mon, 27 Jul 2020 07:15:32 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|graham@menhennitt.com.au X-MailChannels-Auth-Id: dreamhost X-Whistle-Lonely: 5404dbca6348b52a_1595834132142_2547389992 X-MC-Loop-Signature: 1595834132142:515953178 X-MC-Ingress-Time: 1595834132142 Received: from pdx1-sub0-mail-a39.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a39.g.dreamhost.com (Postfix) with ESMTP id 94D55A7F75 for ; Mon, 27 Jul 2020 00:15:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=menhennitt.com.au; h= subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s= menhennitt.com.au; bh=oriZVdfX61zOp1mIwUplkGzkK50=; b=glw6Md0ouF 6LijSeKK88NeklVGkKoUDYycBasYgGa6YV+NTvZtgYEk97Ez50MVMlUQxp7F6SMM Y2eVaVp88Xk7fCk6okssNv718f4czviWRpWW4/O4DSQlCO52GvrfWIUkjLJhzKRu EdRlifd6DNUSRD+w+7TqIxHQszdgl587E= Received: from [203.2.73.68] (194-223-62-221.tpgi.com.au [194.223.62.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: graham@menhennitt.com.au) by pdx1-sub0-mail-a39.g.dreamhost.com (Postfix) with ESMTPSA id 0DC99A9823 for ; Mon, 27 Jul 2020 00:15:28 -0700 (PDT) Subject: Re: Traffic "corruption" in 12-stable To: freebsd-stable@freebsd.org References: <9FAE54DE-F409-4A53-B91E-59AE52A86513@marcuscom.com> <9d6dc414-2866-e6c8-6b66-22af23efc728@grosbein.net> X-DH-BACKEND: pdx1-sub0-mail-a39 From: Graham Menhennitt Message-ID: Date: Mon, 27 Jul 2020 17:15:26 +1000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <9d6dc414-2866-e6c8-6b66-22af23efc728@grosbein.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-VR-OUT-STATUS: OK X-VR-OUT-SCORE: 0 X-VR-OUT-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduiedrheelgdeigecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucggtfgfnhhsuhgsshgtrhhisggvpdfftffgtefojffquffvnecuuegrihhlohhuthemuceftddtnecunecujfgurhepuffvfhfhkffffgggjggtgfesthekredttdefjeenucfhrhhomhepifhrrghhrghmucfovghnhhgvnhhnihhtthcuoehgrhgrhhgrmhesmhgvnhhhvghnnhhithhtrdgtohhmrdgruheqnecuggftrfgrthhtvghrnhepgeevheekjeeiudeghedvgeekveelveevvdeiffejhfetveeujedvhfduheefgeevnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecukfhppeduleegrddvvdefrdeivddrvddvudenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhhouggvpehsmhhtphdphhgvlhhopeglvddtfedrvddrjeefrdeikegnpdhinhgvthepudelgedrvddvfedriedvrddvvddupdhrvghtuhhrnhdqphgrthhhpefirhgrhhgrmhcuofgvnhhhvghnnhhithhtuceoghhrrghhrghmsehmvghnhhgvnhhnihhtthdrtghomhdrrghuqedpmhgrihhlfhhrohhmpehgrhgrhhgrmhesmhgvnhhhvghnnhhithhtrdgtohhmrdgruhdpnhhrtghpthhtohepfhhrvggvsghsugdqshhtrggslhgvsehfrhgvvggsshgurdhorhhg Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4BFWKz0gRpz4Z4d X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=menhennitt.com.au header.s=menhennitt.com.au header.b=glw6Md0o; dmarc=none; spf=pass (mx1.freebsd.org: domain of graham@menhennitt.com.au designates 23.83.209.25 as permitted sender) smtp.mailfrom=graham@menhennitt.com.au X-Spamd-Result: default: False [-3.08 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[194.223.62.221:received]; FROM_HAS_DN(0.00)[]; R_DKIM_ALLOW(-0.20)[menhennitt.com.au:s=menhennitt.com.au]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:23.83.208.1/20]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.02)[-1.020]; DMARC_NA(0.00)[menhennitt.com.au]; DKIM_TRACE(0.00)[menhennitt.com.au:+]; NEURAL_HAM_SHORT(-0.56)[-0.555]; RCVD_IN_DNSWL_NONE(0.00)[23.83.209.25:from]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36483, ipnet:23.83.208.0/21, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[23.83.209.25:from] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jul 2020 07:15:36 -0000 On 27/7/20 3:00 pm, Eugene Grosbein wrote: > 27.07.2020 5:16, Joe Clarke wrote: > >> About two weeks ago, I upgraded from the latest 11-stable to the lates= t 12-stable. After that, I periodically see the network throughput come = to a near standstill. This FreeBSD machine is an ESXi VM with two interf= aces. It acts as a router. It uses vmxnet3 interfaces for both LAN and = WAN. It runs ipfw with in-kernel NAT. The LAN side uses a bridge with v= mx0 and a tap0 L2 VPN interface. My LAN side uses an MTU of 9000, and my= vmx1 (WAN side) uses the default 1500. >> >> Besides seeing massive packet loss and huge latency (~ 200 ms for on-L= AN ping times), I know the problem has occurred because my lldpd reports: >> >> Jul 26 15:47:03 namale lldpd[1126]: frame too short for tlv received o= n bridge0 >> >> And if I turn on ipfw verbose messages, I see tons of: >> >> Jul 26 16:02:23 namale kernel: ipfw: pullup failed >> >> This leads to me to believe packets are being corrupted on ingress. I= =E2=80=99ve applied all the recent iflib changes, but the problem persist= s. What causes it, I don=E2=80=99t know. >> >> The only thing that changed (and yes, it=E2=80=99s a big one) is I upg= raded to 12-stable. Meaning, the rest of the network infra and topology = has remained the same. This did not happen at all in 11-stable. >> >> I=E2=80=99m open to suggestions. > First, try: ifconfig $ifname -rxcsum -txcsum > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" And possibly " -vlanhwtso -tso4" as well. Graham