Date: Fri, 26 Sep 2025 10:08:56 +0200 (CEST) From: Ronald Klop <ronald-lists@klop.ws> To: Michael Tuexen <michael.tuexen@lurchi.franken.de> Cc: np@freebsd.org, net@freebsd.org, virtualization@freebsd.org, John Nielsen <lists@jnielsen.net> Subject: Re: Chelsio (cxgbe) VF on FreeBSD guest under KVM requires -rxcsum Message-ID: <1500781860.3093.1758874136857@localhost> In-Reply-To: <CF7D677F-2D3A-41B5-A3F1-0FCA844D206F@lurchi.franken.de> References: <5D2A2499-4B43-4FE6-AA5A-AD334BC4E872@jnielsen.net> <CF7D677F-2D3A-41B5-A3F1-0FCA844D206F@lurchi.franken.de>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Hi, This sounds very similar to the scenario that is fixed in 16-CURRENT and 15 with delayed checksum in a virtual environment. These are some of the commits around this, but there are more commits involved. https://cgit.freebsd.org/src/commit/?id=39d4094173f9a49ff52f5f4408e4dbd5d6ef0409 https://cgit.freebsd.org/src/commit/?id=bcb298fa9e23c1192c5707086a67d3b396186abc I also remember bug reports about this case, but don't have the link availabie. I hope this is a pointer to the solution for you. Regards, Ronald. Van: Michael Tuexen <michael.tuexen@lurchi.franken.de> Datum: vrijdag, 26 september 2025 09:45 Aan: John Nielsen <lists@jnielsen.net> CC: net@freebsd.org, virtualization@freebsd.org, np@freebsd.org Onderwerp: Re: Chelsio (cxgbe) VF on FreeBSD guest under KVM requires -rxcsum > > > On 26. Sep 2025, at 02:58, John Nielsen <lists@jnielsen.net> wrote: > > > > Hi- > > > > Not sure if this is a known issue or even an issue on the FreeBSD side but decided to ask here first. I’m happy to put in a bug report if appropriate. > > > > I have a hypervisor machine running Arch Linux with KVM, Qemu and libvirtd. The machine has a Chelsio T520-CR adapter. I recently began passing through virtual functions of the NIC to several of the guests I run on the hypervisor. One of the guests runs Windows 11, and the change was seamless. Two of the guests are running FreeBSD (14.3 or so). On each of them the VFs were readily identified and configured (using DHCP in one case), and ping and ARP appeared to work fine. However, TCP and UDP traffic to the guests never received a response. After some head-scratching and troubleshooting I discovered that running “ifconfig cxlv0 -rxcsum” immediately allowed traffic to flow as expected. > Hi John, > > I don't have access to such a network card. Just to be clear: you are running the “ifconfig cxlv0 -rxcsum” command inside the guest running FreeBSD, right? > What is the peer, when you mention TCP and UDP do not work? Is it the host running Linux? Is it another VM? Is it some external host? > > > > As workarounds go that isn’t terrible, but I was surprised that the FreeBSD guests required this adjustment while the Windows one did not. > > > > As a point of reference I have another hypervisor machine that runs FreeBSD 14-STABLE. It also has a > So the host runs FreeBSD and you use bhyve to run some VM? > > Chelsio T520-CR adapter. FreeBSD guests on that machine do not require “-rxcsum” in order to use the Chelsio virtual functions. > The guests run FreeBSD-STABLE or Freebsd 14.3? > What is the peer, when you mention TCP and UDP do not work? Is it the host running Linux? Is it another VM? Is it some external host? > > > > Let me know if anyone can shed some light on this. I’m happy to provide additional details or run further tests if needed. Thank you! > Maybe I can, but I need to understand the scenario... > > Best regards > Michael > > > > -John > > > > > > > > > [-- Attachment #2 --] <html><head></head><body>Hi,<br> <br> This sounds very similar to the scenario that is fixed in 16-CURRENT and 15 with delayed checksum in a virtual environment.<br> <br> These are some of the commits around this, but there are more commits involved.<br> <a href="https://cgit.freebsd.org/src/commit/?id=39d4094173f9a49ff52f5f4408e4dbd5d6ef0409">https://cgit.freebsd.org/src/commit/?id=39d4094173f9a49ff52f5f4408e4dbd5d6ef0409</a><br>; <a href="https://cgit.freebsd.org/src/commit/?id=bcb298fa9e23c1192c5707086a67d3b396186abc">https://cgit.freebsd.org/src/commit/?id=bcb298fa9e23c1192c5707086a67d3b396186abc</a><br>; <br> I also remember bug reports about this case, but don't have the link availabie.<br> <br> I hope this is a pointer to the solution for you.<br> <br> Regards,<br> Ronald.<br> <br> <p><strong>Van:</strong> Michael Tuexen <michael.tuexen@lurchi.franken.de><br> <strong>Datum:</strong> vrijdag, 26 september 2025 09:45<br> <strong>Aan:</strong> John Nielsen <lists@jnielsen.net><br> <strong>CC:</strong> net@freebsd.org, virtualization@freebsd.org, np@freebsd.org<br> <strong>Onderwerp:</strong> Re: Chelsio (cxgbe) VF on FreeBSD guest under KVM requires -rxcsum</p> <blockquote style="padding-right: 0px; padding-left: 5px; margin-left: 5px; border-left: #000000 2px solid; margin-right: 0px"> <div class="MessageRFC822Viewer" id="P"> <div class="TextPlainViewer" id="P.P">> On 26. Sep 2025, at 02:58, John Nielsen <lists@jnielsen.net> wrote:<br> ><br> > Hi-<br> ><br> > Not sure if this is a known issue or even an issue on the FreeBSD side but decided to ask here first. I’m happy to put in a bug report if appropriate.<br> ><br> > I have a hypervisor machine running Arch Linux with KVM, Qemu and libvirtd. The machine has a Chelsio T520-CR adapter. I recently began passing through virtual functions of the NIC to several of the guests I run on the hypervisor. One of the guests runs Windows 11, and the change was seamless. Two of the guests are running FreeBSD (14.3 or so). On each of them the VFs were readily identified and configured (using DHCP in one case), and ping and ARP appeared to work fine. However, TCP and UDP traffic to the guests never received a response. After some head-scratching and troubleshooting I discovered that running “ifconfig cxlv0 -rxcsum” immediately allowed traffic to flow as expected.<br> Hi John,<br> <br> I don't have access to such a network card. Just to be clear: you are running the “ifconfig cxlv0 -rxcsum” command inside the guest running FreeBSD, right?<br> What is the peer, when you mention TCP and UDP do not work? Is it the host running Linux? Is it another VM? Is it some external host?<br> ><br> > As workarounds go that isn’t terrible, but I was surprised that the FreeBSD guests required this adjustment while the Windows one did not.<br> ><br> > As a point of reference I have another hypervisor machine that runs FreeBSD 14-STABLE. It also has a<br> So the host runs FreeBSD and you use bhyve to run some VM?<br> > Chelsio T520-CR adapter. FreeBSD guests on that machine do not require “-rxcsum” in order to use the Chelsio virtual functions.<br> The guests run FreeBSD-STABLE or Freebsd 14.3?<br> What is the peer, when you mention TCP and UDP do not work? Is it the host running Linux? Is it another VM? Is it some external host?<br> ><br> > Let me know if anyone can shed some light on this. I’m happy to provide additional details or run further tests if needed. Thank you!<br> Maybe I can, but I need to understand the scenario...<br> <br> Best regards<br> Michael<br> ><br> > -John<br> ><br> ><br> <br> </div> <hr></div> </blockquote> <br> </body></html>help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1500781860.3093.1758874136857>