Date: Tue, 12 Apr 2022 21:48:53 +0200 From: Kristof Provost <kp@FreeBSD.org> To: Charles Sprickman <spork@bway.net> Cc: Matt Garber <matt.garber@gmail.com>, mike tancsa <mike@sentex.net>, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Subject: Re: vtnet rxcsum broken for forwarding RELENG_13 ? Message-ID: <322649DF-446E-4BAE-876D-D4FC47FE84B0@FreeBSD.org> In-Reply-To: <5A9B449D-BC3C-4D89-8AE8-7CC680B2F41E@bway.net> References: <d30a54ad-6b93-456e-64fc-75d1b09b2fb3@sentex.net> <CANwXMPPUEYWOoYLcYGhzMpP=MOd-oNrT4S7NJUy8AE52cPRvEg@mail.gmail.com> <0FE1F488-EEA5-4010-9926-2D9567E8461F@FreeBSD.org> <5A9B449D-BC3C-4D89-8AE8-7CC680B2F41E@bway.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_= Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 12 Apr 2022, at 21:40, Charles Sprickman wrote: >> On Apr 12, 2022, at 6:43 AM, Kristof Provost <kp@FreeBSD.org> wrote: >> >> On 12 Apr 2022, at 2:07, Matt Garber wrote: >>> On Mon, Apr 11, 2022 at 7:15 PM mike tancsa <mike@sentex.net> wrote: >>> >>>> I was setting up a VM pf firewall and noticed I was not able to nat >>>> out >>>> for some reason. Looking at the pcap, it seems when the vm is in >>>> forwarding mode, I get tcp checksum errors. If I do a >>>> >>>> ifconfig vtnet1 -rxcsum >>>> >>>> ifconfig vtnet0 -rxcsum >>>> >>>> nat then seems to work fine >>>> >>>> The setup is a simple VM with the hypervisor libvirt/KVM ubuntu 20 >>>> LTS. >>>> Guest is RELENG_13 from Apr 11/2022. If I change to em nics in the >>>> VM, >>>> all is fine out of the box. >>>> >>>> >>>> I opened up >>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263229 >>> >>> >>> >>> Unless someone knows otherwise, I’ve been under the impression >>> that PF — or >>> potentially any of the other FreeBSD firewalls (?), but I use PF — >>> has been >>> “broken” in that regard on Linux KVM-based FreeBSD guests for >>> years. As >>> such I’ve always needed to use csum_disable flags on the vtnet >>> interfaces >>> or suffer *extremely* poor network performance, even for servers not >>> doing >>> NAT forwarding. >>> >> That PF checksum issue was fixed >> c110fc49da2995d10d60d908af0838ecb4be9bee, back in 2015. > > Do you have a bug ID that references this issue/fix? > commit c110fc49da2995d10d60d908af0838ecb4be9bee Author: Kristof Provost <kp@FreeBSD.org> Date: Wed Oct 14 16:21:41 2015 +0000 pf: Fix TSO issues In certain configurations (mostly but not exclusively as a VM on Xen) pf produced packets with an invalid TCP checksum. The problem was that pf could only handle packets with a full checksum. The FreeBSD IP stack produces TCP packets with a pseudo-header checksum (only addresses, length and protocol). Certain network interfaces expect to see the pseudo-header checksum, so they end up producing packets with invalid checksums. To fix this stop calculating the full checksum and teach pf to only update TCP checksums if TSO is disabled or the change affects the pseudo-header checksum. PR: 154428, 193579, 198868 Reviewed by: sbruno MFC after: 1 week Relnotes: yes Sponsored by: RootBSD Differential Revision: https://reviews.freebsd.org/D3779 Kristof --=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_= Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <!DOCTYPE html> <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/xhtml; charset=3Dutf-8"= > </head> <body><div style=3D"font-family: sans-serif;"><div class=3D"markdown" sty= le=3D"white-space: normal;"> <p dir=3D"auto">On 12 Apr 2022, at 21:40, Charles Sprickman wrote:</p> </div><div class=3D"plaintext" style=3D"white-space: normal;"><blockquote= style=3D"margin: 0 0 5px; padding-left: 5px; border-left: 2px solid #136= BCE; color: #136BCE;"><blockquote style=3D"margin: 0 0 5px; padding-left:= 5px; border-left: 2px solid #136BCE; border-left-color: #4B89CF; color: = #4B89CF;"><p dir=3D"auto">On Apr 12, 2022, at 6:43 AM, Kristof Provost &l= t;kp@FreeBSD.org> wrote:</p> <p dir=3D"auto">On 12 Apr 2022, at 2:07, Matt Garber wrote:</p> <blockquote style=3D"margin: 0 0 5px; padding-left: 5px; border-left: 2px= solid #136BCE; border-left-color: #4B89CF; color: #4B89CF;"><p dir=3D"au= to">On Mon, Apr 11, 2022 at 7:15 PM mike tancsa <mike@sentex.net> w= rote:</p> <blockquote style=3D"margin: 0 0 5px; padding-left: 5px; border-left: 2px= solid #136BCE; border-left-color: #4B89CF; color: #4B89CF;"><p dir=3D"au= to">I was setting up a VM pf firewall and noticed I was not able to nat o= ut <br> for some reason. Looking at the pcap, it seems when the vm is in <br> forwarding mode, I get tcp checksum errors. If I do a</p> <p dir=3D"auto">ifconfig vtnet1 -rxcsum</p> <p dir=3D"auto">ifconfig vtnet0 -rxcsum</p> <p dir=3D"auto">nat then seems to work fine</p> <p dir=3D"auto">The setup is a simple VM with the hypervisor libvirt/KVM = ubuntu 20 LTS. <br> Guest is RELENG_13 from Apr 11/2022. If I change to em nics in the VM, <br> all is fine out of the box.</p> <p dir=3D"auto">I opened up <a href=3D"https://bugs.freebsd.org/bugzilla/= show_bug.cgi?id=3D263229">https://bugs.freebsd.org/bugzilla/show_bug.cgi?= id=3D263229</a></p> </blockquote><p dir=3D"auto">Unless someone knows otherwise, I=E2=80=99ve= been under the impression that PF =E2=80=94 or <br> potentially any of the other FreeBSD firewalls (?), but I use PF =E2=80=94= has been <br> =E2=80=9Cbroken=E2=80=9D in that regard on Linux KVM-based FreeBSD guests= for years. As <br> such I=E2=80=99ve always needed to use csum_disable flags on the vtnet in= terfaces <br> or suffer *extremely* poor network performance, even for servers not doin= g <br> NAT forwarding.</p> </blockquote><p dir=3D"auto">That PF checksum issue was fixed c110fc49da2= 995d10d60d908af0838ecb4be9bee, back in 2015.</p> </blockquote><p dir=3D"auto">Do you have a bug ID that references this is= sue/fix?</p> <br></blockquote></div> <div class=3D"markdown" style=3D"white-space: normal;"> <pre style=3D"margin-left: 15px; margin-right: 15px; padding: 5px; border= : thin solid gray; overflow-x: auto; max-width: 90vw; background-color: #= E4E4E4;"><code>commit c110fc49da2995d10d60d908af0838ecb4be9bee Author: Kristof Provost <kp@FreeBSD.org> Date: Wed Oct 14 16:21:41 2015 +0000 pf: Fix TSO issues In certain configurations (mostly but not exclusively as a VM on Xen)= pf produced packets with an invalid TCP checksum. The problem was that pf could only handle packets with a full checksu= m. The FreeBSD IP stack produces TCP packets with a pseudo-header checksum (= only addresses, length and protocol). Certain network interfaces expect to see the pseudo-header checksum, = so they end up producing packets with invalid checksums. To fix this stop calculating the full checksum and teach pf to only u= pdate TCP checksums if TSO is disabled or the change affects the pseudo-header = checksum. PR: 154428, 193579, 198868 Reviewed by: sbruno MFC after: 1 week Relnotes: yes Sponsored by: RootBSD Differential Revision: https://reviews.freebsd.org/D3779 </code></pre> <p dir=3D"auto">Kristof</p> </div></div></body> </html> --=_MailMate_A689D751-50AB-4466-ADDF-6771A0C1147A_=--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?322649DF-446E-4BAE-876D-D4FC47FE84B0>