From nobody Tue Apr 12 19:48:53 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 618E21AF9C31 for ; Tue, 12 Apr 2022 19:48:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KdGWG2FcFz3kCr; Tue, 12 Apr 2022 19:48:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1649792938; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WgSl8RQnP3cTjkCr+hLm0+N2z/gppQKiRduaAdBOHL4=; b=EO+DvmONyE4UvrXairyveVeuhRnAUVurDwGPKT2zgaUalC5BSHpaQHlYhyf3z1O+MKhMNO MQzbNi2QcS9YexhN553VRJZBI68rnkkh2xX67DP88S1iUOPTbnabbKM2HV7FXPPhM5rNEs 0pQZKIAlgQypNmL90IVaQQtkOpYc5frBbKfnE1BaBd2TkTMnQbGxv71fHzFdDMiiLmpLSy 3vqpV2kITqm3EtqGD8H+tyQhyeDpctMRb5s0PNZDmiDbwvhndqLlW8LQC3nO4dPJDZ5IA0 cUuNh7MOyCE1F6JQ3tjkO+IX5XW36HPa+ItfEzdAJJ0FIXvGEtyl7SOemq+qcA== Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id E57E922506; Tue, 12 Apr 2022 19:48:57 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id B33D644D56; Tue, 12 Apr 2022 21:48:54 +0200 (CEST) From: Kristof Provost To: Charles Sprickman Cc: Matt Garber , mike tancsa , FreeBSD-STABLE Mailing List Subject: Re: vtnet rxcsum broken for forwarding RELENG_13 ? Date: Tue, 12 Apr 2022 21:48:53 +0200 X-Mailer: MailMate (1.14r5852) Message-ID: <322649DF-446E-4BAE-876D-D4FC47FE84B0@FreeBSD.org> In-Reply-To: <5A9B449D-BC3C-4D89-8AE8-7CC680B2F41E@bway.net> References: <0FE1F488-EEA5-4010-9926-2D9567E8461F@FreeBSD.org> <5A9B449D-BC3C-4D89-8AE8-7CC680B2F41E@bway.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_=" Content-Transfer-Encoding: 8bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1649792938; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WgSl8RQnP3cTjkCr+hLm0+N2z/gppQKiRduaAdBOHL4=; b=u+SJl3tg+hR7u3HEWzgmJBxbj1aPM1rvCqA8DOIookOWTMwMRBj2rjOnLjbHL9jGeucP+X rg4XhW02T/YgSea6zURGRrwxEjFn+dwjkG9VoCKQ+9HVHLIIQkfORYW0t3sHdhl/GmHBIm ApZ8FGdR4AZj/L9boTHh9ibSmh6SI/JbnsJ54Mn0DEa6Nkeg01j7bG0mxGuRR4/45/rSTA EW4aCdP2SEjCbSHskifSj8XeUF5+595YftxNr0cxP8lVjfaCi5PItSuEPbXXDX+7ZDgEbC ACyevLQysK9iP9qEwT8akj/l3bwm2+dux+A5D6Q+pA0m4upSghItDtTxQSMOrw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1649792938; a=rsa-sha256; cv=none; b=hzq7/t2IZkFqjTGPdo+3OzFZ0RM2CTE7eggBs2TmYkDiTb03rz+fwHygC0WbUWvHWr23A0 MGtb91Ho62G4BQZDG1Xdzoqk2wVYH0bp1+j3UDxb8MvqI3nImrRdyrcr+pmsDF+FGALYmN 0ZXEpKEkhP8ZIYXaNo/SmUC8t8rubCp5yOxn1uKyhhDIImg5rKp90cCEqFQ0obLstWmvai IH0Q411jOUTjJRCq7/9Kl/+iQ5S4OEteEiiLKP9gC6es9W7JVUVriTq7Ejuo66cAeBdgXw yfsCzzpSMvPWNfdyysCNtkmbopJCOos8Z4PhGwOC4zxOyCFewvndOLWm+JE85g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N --=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_= Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 12 Apr 2022, at 21:40, Charles Sprickman wrote: >> On Apr 12, 2022, at 6:43 AM, Kristof Provost wrote: >> >> On 12 Apr 2022, at 2:07, Matt Garber wrote: >>> On Mon, Apr 11, 2022 at 7:15 PM mike tancsa wrote: >>> >>>> I was setting up a VM pf firewall and noticed I was not able to nat >>>> out >>>> for some reason. Looking at the pcap, it seems when the vm is in >>>> forwarding mode, I get tcp checksum errors. If I do a >>>> >>>> ifconfig vtnet1 -rxcsum >>>> >>>> ifconfig vtnet0 -rxcsum >>>> >>>> nat then seems to work fine >>>> >>>> The setup is a simple VM with the hypervisor libvirt/KVM ubuntu 20 >>>> LTS. >>>> Guest is RELENG_13 from Apr 11/2022. If I change to em nics in the >>>> VM, >>>> all is fine out of the box. >>>> >>>> >>>> I opened up >>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263229 >>> >>> >>> >>> Unless someone knows otherwise, I’ve been under the impression >>> that PF — or >>> potentially any of the other FreeBSD firewalls (?), but I use PF — >>> has been >>> “broken” in that regard on Linux KVM-based FreeBSD guests for >>> years. As >>> such I’ve always needed to use csum_disable flags on the vtnet >>> interfaces >>> or suffer *extremely* poor network performance, even for servers not >>> doing >>> NAT forwarding. >>> >> That PF checksum issue was fixed >> c110fc49da2995d10d60d908af0838ecb4be9bee, back in 2015. > > Do you have a bug ID that references this issue/fix? > commit c110fc49da2995d10d60d908af0838ecb4be9bee Author: Kristof Provost Date: Wed Oct 14 16:21:41 2015 +0000 pf: Fix TSO issues In certain configurations (mostly but not exclusively as a VM on Xen) pf produced packets with an invalid TCP checksum. The problem was that pf could only handle packets with a full checksum. The FreeBSD IP stack produces TCP packets with a pseudo-header checksum (only addresses, length and protocol). Certain network interfaces expect to see the pseudo-header checksum, so they end up producing packets with invalid checksums. To fix this stop calculating the full checksum and teach pf to only update TCP checksums if TSO is disabled or the change affects the pseudo-header checksum. PR: 154428, 193579, 198868 Reviewed by: sbruno MFC after: 1 week Relnotes: yes Sponsored by: RootBSD Differential Revision: https://reviews.freebsd.org/D3779 Kristof --=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_= Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On 12 Apr 2022, at 21:40, Charles Sprickman wrote:

On Apr 12, 2022, at 6:43 AM, Kristof Provost &l= t;kp@FreeBSD.org> wrote:

On 12 Apr 2022, at 2:07, Matt Garber wrote:

On Mon, Apr 11, 2022 at 7:15 PM mike tancsa <mike@sentex.net> w= rote:

I was setting up a VM pf firewall and noticed I was not able to nat o= ut
for some reason. Looking at the pcap, it seems when the vm is in
forwarding mode, I get tcp checksum errors. If I do a

ifconfig vtnet1 -rxcsum

ifconfig vtnet0 -rxcsum

nat then seems to work fine

The setup is a simple VM with the hypervisor libvirt/KVM = ubuntu 20 LTS.
Guest is RELENG_13 from Apr 11/2022. If I change to em nics in the VM,
all is fine out of the box.

I opened up https://bugs.freebsd.org/bugzilla/show_bug.cgi?= id=3D263229

Unless someone knows otherwise, I=E2=80=99ve= been under the impression that PF =E2=80=94 or
potentially any of the other FreeBSD firewalls (?), but I use PF =E2=80=94= has been
=E2=80=9Cbroken=E2=80=9D in that regard on Linux KVM-based FreeBSD guests= for years. As
such I=E2=80=99ve always needed to use csum_disable flags on the vtnet in= terfaces
or suffer *extremely* poor network performance, even for servers not doin= g
NAT forwarding.

That PF checksum issue was fixed c110fc49da2= 995d10d60d908af0838ecb4be9bee, back in 2015.

Do you have a bug ID that references this is= sue/fix?


commit c110fc49da2995d10d60d908af0838ecb4be9bee
Author: Kristof Provost <kp@FreeBSD.org>
Date:   Wed Oct 14 16:21:41 2015 +0000

    pf: Fix TSO issues

    In certain configurations (mostly but not exclusively as a VM on Xen)=
 pf
    produced packets with an invalid TCP checksum.

    The problem was that pf could only handle packets with a full checksu=
m. The
    FreeBSD IP stack produces TCP packets with a pseudo-header checksum (=
only
    addresses, length and protocol).
    Certain network interfaces expect to see the pseudo-header checksum, =
so they
    end up producing packets with invalid checksums.

    To fix this stop calculating the full checksum and teach pf to only u=
pdate TCP
    checksums if TSO is disabled or the change affects the pseudo-header =
checksum.

    PR:             154428, 193579, 198868
    Reviewed by:    sbruno
    MFC after:      1 week
    Relnotes:       yes
    Sponsored by:   RootBSD
    Differential Revision:  https://reviews.freebsd.org/D3779

Kristof

--=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_=--