Date: Sat, 27 Sep 2025 20:37:24 +0200 From: Michael Tuexen <michael.tuexen@lurchi.franken.de> To: John Nielsen <lists@jnielsen.net> Cc: net@freebsd.org, virtualization@freebsd.org, np@freebsd.org Subject: Re: Chelsio (cxgbe) VF on FreeBSD guest under KVM requires -rxcsum Message-ID: <4855A0F6-DD05-42A1-9119-398DC4124FDF@lurchi.franken.de> In-Reply-To: <CDFB2AB5-36DD-4B33-9658-9559EEF9BB47@jnielsen.net> References: <CF7D677F-2D3A-41B5-A3F1-0FCA844D206F@lurchi.franken.de> <B648CE12-8D42-4978-A50B-DE5E7F39A512@jnielsen.net> <FA555678-F299-43BB-BF65-5A429CDF495E@lurchi.franken.de> <CDFB2AB5-36DD-4B33-9658-9559EEF9BB47@jnielsen.net>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 27. Sep 2025, at 18:26, John Nielsen <lists@jnielsen.net> wrote: >=20 >> On Sep 26, 2025, at 3:44=E2=80=AFPM, Michael Tuexen = <michael.tuexen@lurchi.franken.de> wrote: >>=20 >>> On 26. Sep 2025, at 20:52, John Nielsen <lists@jnielsen.net> wrote: >>>=20 >>>> On Sep 26, 2025, at 1:46=E2=80=AFAM, Michael Tuexen = <michael.tuexen@lurchi.franken.de> wrote: >>>> =EF=BB=BF >>>>>=20 >>>>> On 26. Sep 2025, at 02:58, John Nielsen <lists@jnielsen.net> = wrote: >>>>>=20 >>>>> Not sure if this is a known issue or even an issue on the FreeBSD = side but decided to ask here first. I=E2=80=99m happy to put in a bug = report if appropriate. >>>>>=20 >>>>> I have a hypervisor machine running Arch Linux with KVM, Qemu and = libvirtd. The machine has a Chelsio T520-CR adapter. I recently began = passing through virtual functions of the NIC to several of the guests I = run on the hypervisor. One of the guests runs Windows 11, and the change = was seamless. Two of the guests are running FreeBSD (14.3 or so). On = each of them the VFs were readily identified and configured (using DHCP = in one case), and ping and ARP appeared to work fine. However, TCP and = UDP traffic to the guests never received a response. After some = head-scratching and troubleshooting I discovered that running = =E2=80=9Cifconfig cxlv0 -rxcsum=E2=80=9D immediately allowed traffic to = flow as expected. >>>>=20 >>>> I don't have access to such a network card. Just to be clear: you = are running the =E2=80=9Cifconfig cxlv0 -rxcsum=E2=80=9D command inside = the guest running FreeBSD, right? >>>=20 >>> Yes. >>>=20 >>>> What is the peer, when you mention TCP and UDP do not work? Is it = the host running Linux? Is it another VM? Is it some external host? >>>=20 >>> My laptop on the same subnet primarily, but. Also tested from = another physical machine running FreeBSD. >> OK. That does not seem to be related to what I initially thought. >>=20 >> Could you run >> tcpdump -i outgoing_interface -w laptop.pcap >> on your laptop and >> tcpdump -i cxlv0 -w vm.pcap >> on your vm at the same time and try to do some TCP based = communication. >> Maybe two times, one time with ifconfig cxlv0 rxcsum and one time = with ifconfig cxlv0 -rxcsum. >>=20 >> If you are fine with doing the measurements, you can send the .pcap = files >> to tuexen@freebsd.org <mailto:tuexen@freebsd.org>. >>=20 >> At least I would like to understand what is going on. >=20 > Thank you, I sent you those packet captures under separate cover. Thanks I got the email and sent a response.=20 >=20 > =46rom what I can tell so far the VM does receive incoming packets = even when rxcsum is enabled, and the checksums appear correct at least = according to tcpdump. But something prevents the packets from being = processed or sent up the stack. The VM=E2=80=99s sshd never generates a = SYN/ACK in response to a connection attempt. A DNS query from the VM = looks like it gets a response on the wire but the response doesn=E2=80=99t= make it to the process doing the query. Yes, I agree. For whatever reason the packets received, but are not = delivered to the transport stack. I would like to figure out where the packets are = dropped or stored and not processed further. I sent you some suggestions to try. I hope we can figure out where the = problem is related. In addition to what I suggested, you might want to provide the output of netstat -s from before and after running the experiment. Best regards Michael >=20 > -John >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4855A0F6-DD05-42A1-9119-398DC4124FDF>