From nobody Mon Apr 11 23:14:00 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2F68D11FFF03 for ; Mon, 11 Apr 2022 23:14:08 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kcl6R0m9Kz4gD8 for ; Mon, 11 Apr 2022 23:14:07 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.16.1/8.16.1) with ESMTPS id 23BNE0qQ034886 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Mon, 11 Apr 2022 19:14:00 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:d955:181b:888c:769f] ([IPv6:2607:f3e0:0:4:d955:181b:888c:769f]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 23BNE0ic092455 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Mon, 11 Apr 2022 19:14:00 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: Date: Mon, 11 Apr 2022 19:14:00 -0400 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Content-Language: en-US To: FreeBSD-STABLE Mailing List From: mike tancsa Subject: vtnet rxcsum broken for forwarding RELENG_13 ? Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 X-Rspamd-Queue-Id: 4Kcl6R0m9Kz4gD8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:1::12 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-2.39 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[sentex.net]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-stable]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[199.212.134.19:received] X-ThisMailContainsUnwantedMimeParts: N I was setting up a VM pf firewall and noticed I was not able to nat out for some reason. Looking at the pcap, it seems when the vm is in forwarding mode, I get tcp checksum errors. If I do a ifconfig vtnet1 -rxcsum ifconfig vtnet0 -rxcsum nat then seems to work fine The setup is a simple VM with the hypervisor libvirt/KVM ubuntu 20 LTS.  Guest is RELENG_13 from Apr 11/2022. If I change to em nics in the VM, all is fine out of the box.  tcpdump -s0 -vnei vtnet1 port 443 tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes 18:35:05.937364 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 64547, offset 0, flags [DF], proto TCP (6), length 60)     192.168.199.100.56996 > 172.217.1.4.443: Flags [S], cksum 0x3619 (incorrect -> 0x4755), seq 2994289493, win 64240, options [mss 1460,sackOK,TS val 1606656477 ecr 0,nop,wscale 7], length 0 18:35:06.939305 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 64548, offset 0, flags [DF], proto TCP (6), length 60)     192.168.199.100.56996 > 172.217.1.4.443: Flags [S], cksum 0x3619 (incorrect -> 0x436b), seq 2994289493, win 64240, options [mss 1460,sackOK,TS val 1606657479 ecr 0,nop,wscale 7], length 0 18:35:21.040936 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 45298, offset 0, flags [DF], proto TCP (6), length 60)     192.168.199.100.59876 > 192.168.199.7.443: Flags [S], cksum 0x0fec (incorrect -> 0xf8f6), seq 1122263205, win 64240, options [mss 1460,sackOK,TS val 2124691085 ecr 0,nop,wscale 7], length 0 18:35:21.040993 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)     192.168.199.7.443 > 192.168.199.100.59876: Flags [S.], cksum 0x0fec (incorrect -> 0x5be5), seq 3948792593, ack 1122263206, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2270714410 ecr 2124691085], length 0 18:35:21.041224 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 45299, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.100.59876 > 192.168.199.7.443: Flags [.], cksum 0x0fe4 (incorrect -> 0x88b9), ack 1, win 502, options [nop,nop,TS val 2124691086 ecr 2270714410], length 0 18:35:22.843871 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 45300, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.100.59876 > 192.168.199.7.443: Flags [F.], cksum 0x0fe4 (incorrect -> 0x81ae), seq 1, ack 1, win 502, options [nop,nop,TS val 2124692888 ecr 2270714410], length 0 18:35:22.843910 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.7.443 > 192.168.199.100.59876: Flags [.], cksum 0x0fe4 (incorrect -> 0x7895), ack 2, win 1027, options [nop,nop,TS val 2270716214 ecr 2124692888], length 0 18:35:22.843980 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.7.443 > 192.168.199.100.59876: Flags [F.], cksum 0x0fe4 (incorrect -> 0x7894), seq 1, ack 2, win 1027, options [nop,nop,TS val 2270716214 ecr 2124692888], length 0 18:35:22.844159 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.100.59876 > 192.168.199.7.443: Flags [.], cksum 0x7aa0 (correct), ack 2, win 502, options [nop,nop,TS val 2124692889 ecr 2270716214], length 0 e.g. after disabling rxcsum tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes 18:39:57.425940 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 53398, offset 0, flags [DF], proto TCP (6), length 60)     192.168.199.101.55810 > 142.251.32.68.443: Flags [S], cksum 0x5467 (correct), seq 2136333436, win 64240, options [mss 1460,sackOK,TS val 2746738194 ecr 0,nop,wscale 7], length 0 18:39:57.432020 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 123, id 58394, offset 0, flags [none], proto TCP (6), length 60)     142.251.32.68.443 > 192.168.199.101.55810: Flags [S.], cksum 0xa929 (correct), seq 292800283, ack 2136333437, win 65535, options [mss 1430,sackOK,TS val 460829236 ecr 2746738194,nop,wscale 8], length 0 18:39:57.432265 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 53399, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.101.55810 > 142.251.32.68.443: Flags [.], cksum 0xd5dc (correct), ack 1, win 502, options [nop,nop,TS val 2746738200 ecr 460829236], length 0 18:39:58.605990 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 53400, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.101.55810 > 142.251.32.68.443: Flags [F.], cksum 0xd145 (correct), seq 1, ack 1, win 502, options [nop,nop,TS val 2746739374 ecr 460829236], length 0 18:39:58.612128 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 123, id 58984, offset 0, flags [none], proto TCP (6), length 52)     142.251.32.68.443 > 192.168.199.101.55810: Flags [F.], cksum 0xcd9e (correct), seq 1, ack 2, win 256, options [nop,nop,TS val 460830416 ecr 2746739374], length 0 18:39:58.612402 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl 64, id 53401, offset 0, flags [DF], proto TCP (6), length 52)     192.168.199.101.55810 > 142.251.32.68.443: Flags [.], cksum 0xcca2 (correct), ack 2, win 502, options [nop,nop,TS val 2746739380 ecr 460830416], length 0 I opened up https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263229