From owner-freebsd-arm@freebsd.org Sun Nov 1 18:48:48 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6BC84562AC; Sun, 1 Nov 2020 18:48:48 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CPQ745KXpz4S1n; Sun, 1 Nov 2020 18:48:48 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 81081263AE; Sun, 1 Nov 2020 18:48:48 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 59CCB3F557; Sun, 1 Nov 2020 19:48:46 +0100 (CET) From: "Kristof Provost" To: "Carsten =?utf-8?q?B=C3=A4cker?=" Cc: "John-Mark Gurney" , freebsd-hackers@freebsd.org, freebsd-arm@freebsd.org Subject: Re: Problem with checksum offloading on RPi3 (PF + Jails involved) Date: Sun, 01 Nov 2020 19:48:45 +0100 X-Mailer: MailMate (1.13.2r5673) Message-ID: In-Reply-To: References: <748edc3d-4ef7-c4de-291f-7c0b460a6052@gmx.de> <5130ee46-5832-d4df-d774-c6bd32e10b30@gmx.de> <20201029213622.GM31099@funkthat.com> <55713894-A896-4F12-ABB9-93DFEB2F16B9@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 18:48:48 -0000 On 31 Oct 2020, at 16:06, Carsten Bäcker wrote: > Am 29.10.2020 um 22:39 schrieb Kristof Provost: >> >> On 29 Oct 2020, at 22:36, John-Mark Gurney wrote: >> >> Kristof Provost wrote this message on Thu, Oct 29, 2020 at 21:30 >> +0100: >> >> On 29 Oct 2020, at 16:30, Carsten Bäcker wrote: >> >> Sure, i am willing to help. >> >> Device is a Raspberry Pi 3B (not +), using the >> onboard-ethernet. >> I attached a bunch of information. >> >> Configuration is stripped down to the minimum required to >> reproduce >> the >> problem. >> >> Okay, so that???s an SMC2 LAN9514_ETH device. >> That???s the dev/usb/net/if_smsc.c driver. >> >> However, before we dig into that driver we should make sure >> that we???re >> really looking at a checksum problem. >> It???s entirely normal for TX checksums to be incorrect when >> logged on >> the sending host itself (if the hardware does checksum >> offloading the >> checksum in the packet sent to the MAC is incorrect, and left >> to the >> hardware to fix). >> >> So, can you confirm that the `"[bad udp cksum 0xe58a -> >> 0x482d!]` you >> reported was on an inbound packet? And let???s be safe: try >> to >> capture >> packets on a different machine. That???ll give us the true >> packet, after >> the hardware has done checksum calculations. >> >> One interesting point is that the smsc driver claims to not do TX >> offload, >> and a brief check shows that it doesn't allow a user to set the >> TXCSUM. >> >> It does seem to do RX offload, and the comments in the driver suggest >> some .. ahem, creative hardware behaviour: >> >> |/* The checksum appears to be simplistically calculated * over the >> udp/tcp header and data up to the end of the * eth frame. Which means >> if the eth frame is padded * the csum calculation is incorrectly >> performed over * the padding bytes as well. Therefore to be safe we * >> ignore the H/W csum on frames less than or equal to * 64 bytes. * * >> Ignore H/W csum for non-IPv4 packets. */ | >> >> It’s not impossible that there’s some more issues like that in >> the >> hardware, or even that there are different chip revisions out there. >> >> That also matches up with |ifconfig ue0 -rxcsum| fixing things. >> >> Best regards, >> Kristof >> > Hello again, > > just gave it another try, using "ping heise.de" (twice) from within > the > jail. > > This is the output of tcpdump: > # tcpdump -i pflog0 -vv > tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), > capture size 262144 bytes > 14:06:04.293996 IP (tos 0x0, ttl 64, id 19576, offset 0, flags [none], > proto UDP (17), length 54) >     PC-192-168-178-3.fritz.box.52421 > fritz.box.domain: [bad udp > cksum > 0xe589 -> 0x0a2c!] 64654+ A? heise.de. (26) > > Netstat: > # netstat -s -p udp | grep checksum >         4 with bad checksum >         0 with no checksum > > Packet-capture from the router is attached. > Okay… but that doesn’t actually demonstrate anything at all. It shows an outbound packet with an incorrect checksum. See what I’ve mentioned before about checksums potentially being calculated by hardware. Given that there’s been a reply to that DNS query the checksum clearly got set correctly after the packet was captured. Best, Kristof