From owner-freebsd-stable@FreeBSD.ORG Wed Dec 13 06:10:28 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43D7716A403 for ; Wed, 13 Dec 2006 06:10:28 +0000 (UTC) (envelope-from spork@bway.net) Received: from xena.bway.net (xena.bway.net [216.220.96.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id D11D843C9D for ; Wed, 13 Dec 2006 06:09:00 +0000 (GMT) (envelope-from spork@bway.net) Received: (qmail 76163 invoked by uid 0); 13 Dec 2006 06:10:26 -0000 Received: from unknown (HELO white.nat.fasttrackmonkey.com) (spork@bway.net@216.220.116.154) by smtp.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 13 Dec 2006 06:10:26 -0000 Date: Wed, 13 Dec 2006 01:10:25 -0500 (EST) From: Charles Sprickman X-X-Sender: spork@white.nat.fasttrackmonkey.com To: freebsd-stable@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: pf killing NFS X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2006 06:10:28 -0000 Hi all, I'm running a 6.2-RC1 box (cvsup'd today) that has two broadcom nics. One is an internal network (nfs) and the other is external. PF has this rule for all traffic on the private net: [root@archive /home/jails]# pfctl -sr|grep bge1 pass in quick on bge1 inet from 192.168.1.0/24 to any pass out quick on bge1 inet from any to 192.168.1.0/24 No state since these are "quick" and symmetrical. Doing something like "ls /usr/ports" will just hang until interrupted. Using tcp for nfs makes it workable, but very slow. If I disable pf (pfctl -d), both types of mounts work, and speed is excellent. I also just found that if I remove the "scrub in all" statement and change it to "scrub in on bge0", things are fine. Any idea what's going on? The tcpdump output confuses me (see "bad cksum!"), so I'm posting some snippets here. Looking at tcpdump, things look a bit odd. 192.168.1.111 is the nfs client (6.2-RC1), 192.168.1.100 is the nfs server (4.11): [root@archive /home/spork]# tcpdump -i bge1 -v tcpdump: listening on bge1, link-type EN10MB (Ethernet), capture size 96 bytes 00:59:16.269659 IP (tos 0x0, ttl 64, id 5395, offset 0, flags [none], proto: UDP (17), length: 132, bad cksum 0 (->e132)!) 192.168.1.111.1861387036 > 192.168.1.100.nfs: 104 access [|nfs] bad checksum before even hitting the wire?? 00:59:16.269920 IP (tos 0x0, ttl 64, id 46705, offset 0, flags [none], proto: UDP (17), length: 148) 192.168.1.100.nfs > 192.168.1.111.1861387036: reply ok 120 access attr: DIR 755 ids 0/0 [|nfs] We get a reply (dir is mode 755) 00:59:16.270010 IP (tos 0x0, ttl 64, id 5396, offset 0, flags [none], proto: UDP (17), length: 132, bad cksum 0 (->e131)!) 192.168.1.111.1861387037 > 192.168.1.100.nfs: 104 access [|nfs] Again, bad checksum FROM nfs client to server... 00:59:16.270211 IP (tos 0x0, ttl 64, id 58236, offset 0, flags [none], proto: UDP (17), length: 148) 192.168.1.100.nfs > 192.168.1.111.1861387037: reply ok 120 access attr: DIR 755 ids 0/0 [|nfs] 00:59:16.270306 IP (tos 0x0, ttl 64, id 5397, offset 0, flags [none], proto: UDP (17), length: 132, bad cksum 0 (->e130)!) 192.168.1.111.1861387038 > 192.168.1.100.nfs: 104 access [|nfs] Now to confuse things further, if I disable pf (pfctl -d), speeds are great, but I still get these bad checksum errors: 01:04:21.498293 IP (tos 0x0, ttl 64, id 5482, offset 0, flags [none], proto: UDP (17), length: 132, bad cksum 0 (->e0db)!) 192.168.1.111.1861387048 > 192.168.1.100.nfs: 104 access [|nfs] 01:04:21.498607 IP (tos 0x0, ttl 64, id 16228, offset 0, flags [none], proto: UDP (17), length: 148) 192.168.1.100.nfs > 192.168.1.111.1861387048: reply ok 120 access attr: DIR 755 ids 0/0 [|nfs] 01:04:21.498675 IP (tos 0x0, ttl 64, id 5483, offset 0, flags [none], proto: UDP (17), length: 132, bad cksum 0 (->e0da)!) 192.168.1.111.1861387049 > 192.168.1.100.nfs: 104 access [|nfs] 01:04:21.498900 IP (tos 0x0, ttl 64, id 13349, offset 0, flags [none], proto: UDP (17), length: 148) 192.168.1.100.nfs > 192.168.1.111.1861387049: reply ok 120 access attr: DIR 755 ids 0/0 [|nfs] 01:04:21.498924 IP (tos 0x0, ttl 64, id 5484, offset 0, flags [none], proto: UDP (17), length: 132, bad cksum 0 (->e0d9)!) 192.168.1.111.1861387050 > 192.168.1.100.nfs: 104 access [|nfs] 01:04:21.499195 IP (tos 0x0, ttl 64, id 34907, offset 0, flags [none], proto: UDP (17), length: 148) 192.168.1.100.nfs > 192.168.1.111.1861387050: reply ok 120 access attr: DIR 755 ids 0/0 [|nfs]