Date: Tue, 30 Sep 2025 12:48:47 -0600 From: John Nielsen <lists@jnielsen.net> To: Michael Tuexen <michael.tuexen@lurchi.franken.de> Cc: net@freebsd.org, virtualization@freebsd.org, np@freebsd.org Subject: Re: Chelsio (cxgbe) VF on FreeBSD guest under KVM requires -rxcsum Message-ID: <07254AC1-40B9-43B5-A902-08261C4CE2F0@jnielsen.net> In-Reply-To: <4855A0F6-DD05-42A1-9119-398DC4124FDF@lurchi.franken.de> References: <CF7D677F-2D3A-41B5-A3F1-0FCA844D206F@lurchi.franken.de> <B648CE12-8D42-4978-A50B-DE5E7F39A512@jnielsen.net> <FA555678-F299-43BB-BF65-5A429CDF495E@lurchi.franken.de> <CDFB2AB5-36DD-4B33-9658-9559EEF9BB47@jnielsen.net> <4855A0F6-DD05-42A1-9119-398DC4124FDF@lurchi.franken.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sep 27, 2025, at 12:37=E2=80=AFPM, Michael Tuexen = <michael.tuexen@lurchi.franken.de> wrote: >=20 >> On 27. Sep 2025, at 18:26, John Nielsen <lists@jnielsen.net> wrote: >>=20 >>> On Sep 26, 2025, at 3:44=E2=80=AFPM, Michael Tuexen = <michael.tuexen@lurchi.franken.de> wrote: >>>=20 >>>> On 26. Sep 2025, at 20:52, John Nielsen <lists@jnielsen.net> wrote: >>>>=20 >>>>> On Sep 26, 2025, at 1:46=E2=80=AFAM, Michael Tuexen = <michael.tuexen@lurchi.franken.de> wrote: >>>>> =EF=BB=BF >>>>>>=20 >>>>>> On 26. Sep 2025, at 02:58, John Nielsen <lists@jnielsen.net> = wrote: >>>>>>=20 >>>>>> Not sure if this is a known issue or even an issue on the FreeBSD = side but decided to ask here first. I=E2=80=99m happy to put in a bug = report if appropriate. >>>>>>=20 >>>>>> I have a hypervisor machine running Arch Linux with KVM, Qemu and = libvirtd. The machine has a Chelsio T520-CR adapter. I recently began = passing through virtual functions of the NIC to several of the guests I = run on the hypervisor. One of the guests runs Windows 11, and the change = was seamless. Two of the guests are running FreeBSD (14.3 or so). On = each of them the VFs were readily identified and configured (using DHCP = in one case), and ping and ARP appeared to work fine. However, TCP and = UDP traffic to the guests never received a response. After some = head-scratching and troubleshooting I discovered that running = =E2=80=9Cifconfig cxlv0 -rxcsum=E2=80=9D immediately allowed traffic to = flow as expected. >>>>>=20 >>>>> I don't have access to such a network card. Just to be clear: you = are running the =E2=80=9Cifconfig cxlv0 -rxcsum=E2=80=9D command inside = the guest running FreeBSD, right? >>>>=20 >>>> Yes. >>>>=20 >>>>> What is the peer, when you mention TCP and UDP do not work? Is it = the host running Linux? Is it another VM? Is it some external host? >>>>=20 >>>> My laptop on the same subnet primarily, but. Also tested from = another physical machine running FreeBSD. >>> OK. That does not seem to be related to what I initially thought. >>>=20 >>> Could you run >>> tcpdump -i outgoing_interface -w laptop.pcap >>> on your laptop and >>> tcpdump -i cxlv0 -w vm.pcap >>> on your vm at the same time and try to do some TCP based = communication. >>> Maybe two times, one time with ifconfig cxlv0 rxcsum and one time = with ifconfig cxlv0 -rxcsum. >>>=20 >>> If you are fine with doing the measurements, you can send the .pcap = files >>> to tuexen@freebsd.org <mailto:tuexen@freebsd.org>. >>>=20 >>> At least I would like to understand what is going on. >>=20 >> Thank you, I sent you those packet captures under separate cover. > Thanks I got the email and sent a response. >>=20 >> =46rom what I can tell so far the VM does receive incoming packets = even when rxcsum is enabled, and the checksums appear correct at least = according to tcpdump. But something prevents the packets from being = processed or sent up the stack. The VM=E2=80=99s sshd never generates a = SYN/ACK in response to a connection attempt. A DNS query from the VM = looks like it gets a response on the wire but the response doesn=E2=80=99t= make it to the process doing the query. > Yes, I agree. For whatever reason the packets received, but are not = delivered to > the transport stack. I would like to figure out where the packets are = dropped or > stored and not processed further. I > sent you some suggestions to try. I hope we can figure out where the = problem is > related. > In addition to what I suggested, you might want to provide the output = of > netstat -s > from before and after running the experiment. Thanks again Michael for your help pinpointing the issue. For the = archives, this is now being tracked under = https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D289907 . JN
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?07254AC1-40B9-43B5-A902-08261C4CE2F0>