From nobody Tue Apr 12 20:08:29 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 52A321AFE83C for ; Tue, 12 Apr 2022 20:08:47 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KdGy63M5xz3pWg; Tue, 12 Apr 2022 20:08:46 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-2ebd70a4cf5so328207b3.3; Tue, 12 Apr 2022 13:08:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BW1Gli9jgYX2XlnIPhgIBbJ8zBZ1GO1Ywin9TpB5CmE=; b=dvNIo38Cg8z6uGq8N4ucNut7jn6zvULFrbRmQaHRuW4fc1Me9LfM1O4XXSgczan0Zm 0EL/pyRzqhs6lQDRPmHpeh9QqJ2EVW1xR5O8vBCjxhfjVl2DF0JpK/hgG6L6SsaPafip OwbyDrii7f1StONVH4Zu/PZlu7IeRb04FhPysmWLZVIEWzshGVD7ipZRhgZ+w7YG4YP/ lcgKJWHFEa3w509ilFJFLjmuINH+vzdTiZq0KmY9MkK+r9Ff889LMBVXxmEKJm7YiyOe mrmu+u+L4vZryiTjaqX9kZ3K3pbV4gXH4LuIDhKI7PZMCAuu002JBlRA3uu771F77opk zEJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BW1Gli9jgYX2XlnIPhgIBbJ8zBZ1GO1Ywin9TpB5CmE=; b=TNNthkvYTMNQF27dpZ8NaqbvrGCQg+8TKe/zxmniQ33sDxj7JmeScQhiun+IBRRX9H qbiudx05D8JqPaiah1MNGZR9B8aTuveACW1MksEcoizICmirqtbtdB1Y0+0owaIOSZ+0 gBhz2AoqGTvi24vfrHYt8NnFv463Wa9aKHX5MbrlT9xG4DlpEuM146K5mCKmQVeqQdxU GfGP4pRJW0mDj86a3BkhsHGnGIsyuEl2uVKdssr9HIjJEvR6KaX0KvRgFSk1e31VnfCD UEr5dOxYyfKZihhGo6Cs5ux6T1S8fJMMnpv6eKr5T8/SkfiugwEYVbIklU4G9XE5VDgV YLpQ== X-Gm-Message-State: AOAM530yQYRVDDRkstIfMVljHBx5K4eAgAKVc2sWNvIUqcK6SU8u94qo U9EqamNM1Qg/nDCSp4WMShiK3R4UHOWWRLq0lMo= X-Google-Smtp-Source: ABdhPJzf6XWpKP/hCMjzn5RtX2ra9wqCZdi5KLLyMIse0Qw7g2Hh8gCeJtfywVkt/dgJhMBUd+Z1XO6g7GBBlVMPuCE= X-Received: by 2002:a81:57ca:0:b0:2ef:3a80:2288 with SMTP id l193-20020a8157ca000000b002ef3a802288mr2167816ywb.270.1649794120232; Tue, 12 Apr 2022 13:08:40 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <0FE1F488-EEA5-4010-9926-2D9567E8461F@FreeBSD.org> <5A9B449D-BC3C-4D89-8AE8-7CC680B2F41E@bway.net> <322649DF-446E-4BAE-876D-D4FC47FE84B0@FreeBSD.org> <745890A5-983D-41BA-9592-D283EE800AD6@punkt.de> In-Reply-To: <745890A5-983D-41BA-9592-D283EE800AD6@punkt.de> From: Matt Garber Date: Tue, 12 Apr 2022 16:08:29 -0400 Message-ID: Subject: Re: vtnet rxcsum broken for forwarding RELENG_13 ? To: "Patrick M. Hausen" Cc: Charles Sprickman , FreeBSD-STABLE Mailing List , Kristof Provost , mike tancsa Content-Type: multipart/alternative; boundary="0000000000007d340c05dc7aa05c" X-Rspamd-Queue-Id: 4KdGy63M5xz3pWg X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=dvNIo38C; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mattgarber@gmail.com designates 2607:f8b0:4864:20::1136 as permitted sender) smtp.mailfrom=mattgarber@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCPT_COUNT_FIVE(0.00)[5]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1136:from]; MLMMJ_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --0000000000007d340c05dc7aa05c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Apr 12, 2022 at 4:01 PM Patrick M. Hausen wrote: > Hi Kristof, hi all, > > > Am 12.04.2022 um 21:48 schrieb Kristof Provost : > > That PF checksum issue was fixed > c110fc49da2995d10d60d908af0838ecb4be9bee, back in 2015. > > I still have abysmal performance with pf NAT in a DigitalOcean droplet > running 13.1-RC2 unless I configure: > > ifconfig_vtnet0=3D"-rxcsum -txcsum -rxcsum6 -txcsum6" > > I can give you SSH access, if needed. > > Kind regards, > Patrick Same for me, on 12.x RELEASEs, and I=E2=80=99d previously tested on Digital= Ocean and Google Compute Platform infrastructure. While I don=E2=80=99t doubt tha= t some issues with TCP checksums have potentially been resolved, there are still unresolved performance problems using the vtnet driver (VirtIO, KVM host) unless checksums are disabled. This might only be specific to NAT/forwarding at this point, as I realized my setup also involved PF NAT=E2=80=99ing on cloned loopback interfaces for non-VNET jailed services. Also note that other KVM setups using drivers other than vtnet don=E2=80=99= t seem to have the same problem, at least based on the alternatives I tested =E2= =80=94 e.g., AWS Nitro KVM instances are unaffected as they=E2=80=99re using the E= lastic Network Adapters. Thanks, =E2=80=94Matt --0000000000007d340c05dc7aa05c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, Apr 12, 2022 at 4:01 PM Patrick M. Hausen <hausen@punkt.de> wrote:
Hi Kristof, hi all,

> Am 12.04.2022 um 21:48 schrieb Kristof Provost <kp@FreeBSD.org>:=
> That PF checksum issue was fixed c110fc49da2995d10d60d908af0838ecb4be9= bee, back in 2015.

I still have abysmal performance with pf NAT in a DigitalOcean droplet
running 13.1-RC2 unless I configure:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 ifconfig_vtnet0=3D"-rxcsum -txcsum -rxcsum= 6 -txcsum6"

I can give you SSH access, if needed.

Kind regards,
Patrick

Same for = me, on 12.x RELEASEs, and I=E2=80=99d previously tested on Digital Ocean an= d Google Compute Platform infrastructure. While I don=E2=80=99t doubt that = some issues with TCP checksums have potentially been resolved, there are st= ill unresolved performance problems using the vtnet driver (VirtIO, KVM hos= t) unless checksums are disabled.

This might only be specific to NAT/forwarding at this point, as I= realized my setup also involved PF NAT=E2=80=99ing on cloned loopback inte= rfaces for non-VNET jailed services.

Also note that other KVM setups using drivers other than vtnet= don=E2=80=99t seem to have the same problem, at least based on the alterna= tives I tested =E2=80=94 e.g., AWS Nitro KVM instances are unaffected as th= ey=E2=80=99re using the Elastic Network Adapters.

Thanks,
=E2=80=94Matt

=

--0000000000007d340c05dc7aa05c--