From owner-freebsd-pf@FreeBSD.ORG Sat Oct 13 14:25:38 2012 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9481FF00 for ; Sat, 13 Oct 2012 14:25:38 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id 1EB648FC16 for ; Sat, 13 Oct 2012 14:25:38 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3Xf7Vn3WXNzGMpZ for ; Sat, 13 Oct 2012 16:25:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:organization:user-agent:date:date:subject:subject :from:from:received:received:received:vbr-info; s=jakla2; t= 1350138335; x=1352730336; bh=PUa9yDeFR6r8eLH78xT8vplparkH2LTjB/E cGNelqgk=; b=S0jVEtmGu6/AjT0CSb5dR1NMNDn5KHynBjkqksGz7GIKx6dMxtP iiUQdeIXaClMtWzi4W6U0VP8TMkRYq66Jpmaf+G7mrh+Cl8c3WReYmPakmrOQAZw n3VRVinY6GpFy924XIrIz91znh/Jr9NlWQgSgSIiS43iAsg7K/nIvyaE= VBR-Info: md=ijs.si; mc=all; mv=dwl.spamhaus.org; X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id LqsgoKcScytr for ; Sat, 13 Oct 2012 16:25:35 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP for ; Sat, 13 Oct 2012 16:25:35 +0200 (CEST) Received: from sleepy.ijs.si (sleepy.ijs.si [IPv6:2001:1470:ff80:e001::1:1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id 4AE3A74A for ; Sat, 13 Oct 2012 16:25:35 +0200 (CEST) From: Mark Martinec To: freebsd-pf@freebsd.org Subject: Re: (was: Regression with jails/IPv6/pf) 'scrub reassemble tcp' breaks IPv6 packet checksum on SYN ACK Date: Sat, 13 Oct 2012 16:25:34 +0200 User-Agent: KMail/1.13.7 (FreeBSD/9.1-PRERELEASE; KDE/4.8.4; amd64; ; ) Organization: J. Stefan Institute MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201210131625.34871.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2012 14:25:38 -0000 Bjoern A. Zeeb wrote on 2012-08-01: > Any of you who are expereincing problems with packets dropped due to > invalid checksums with IPv6 and pf after the recent merges, can you > report back if you also see this without "modulate state" in your > pf.conf (if you have 'modulate' in there, can you try changing it to > 'keep' and see if that fixes the problem)? Indeed, invalid checksums with IPv6 and pf after the recent merges. I've opened a PR (before finding about this thread): http://www.freebsd.org/cgi/query-pr.cgi?pr=172648 pf(4): 'scrub reassemble tcp' breaks IPv6 packet checksum on SYN ACK When pf (packet filter) is enabled and configured with 'scrub reassemble tcp', IPv6 TCP connections take 9 seconds to establish. Packet capture shows checksum errors on SYN ACK packets but not on other packets. A TCP connection establishment (SYN) on IPv6 is (re-)tried four times, with a 3 second delay between each attempt, while the TCP options are being simplified each time by the kernel (dropping ECN, CWR, window scaling, and dropping a timestamp options). Only the fourth attempt is successful, with no other options but SACK, and this TCP session then proceeds normally. Disabling 'scrub reassemble tcp' in the pf avoids the problem. Similarly, turning off net.inet.tcp.rfc1323 on either end also avoids the problem, even with 'reassemble tcp' enabled. The problem does not occur on IPv4 sessions, only on IPv6. The problem is not associated with interface checksum offloading, it is repeatable on gif, em, and re interfaces. Also a packet capture (wireshark) shows packet checksum errors on SYN ACK packets (but not on the SYN packet) in the first couple of failed attempts, and no checksum errors on other packets (e.g. after a successfully established session). My guess is that the TCP timestamp option triggers a pf bug, which then miscalculates a packet checksum on SYN ACK. How-To-Repeat Use the following trivial pf config file: scrub all reassemble tcp pass all Then try to establish any TCP session to any IPv6 address. Any client will do (telnet, ssh, curl, web browser). Try for example: curl -6 -L http://tools.ietf.org/rfc/rfc3021.txt | wc -l The connection will 'hang' for 9 seconds (until a sufficiently dumbed-down SYN options are tried), then it proceeds normally. Fix No known fix. Two workarounds: - don't use 'scrub reassemble tcp' in PF, or disable PF - sysctl net.inet.tcp.rfc1323=0 Mark