From owner-freebsd-pf@FreeBSD.ORG Sat Jul 2 15:40:43 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5286106566B for ; Sat, 2 Jul 2011 15:40:43 +0000 (UTC) (envelope-from pierre@userid.org) Received: from mail.storm.ca (unknown [IPv6:2607:f0b0:0:6:209:87:239:66]) by mx1.freebsd.org (Postfix) with ESMTP id 901388FC14 for ; Sat, 2 Jul 2011 15:40:43 +0000 (UTC) Received: from mail.userid.org (pandora.userid.org [216.106.102.33]) by mail.storm.ca (8.14.2+Sun/8.14.2) with ESMTP id p62FXoeK020983 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 2 Jul 2011 11:33:57 -0400 (EDT) Received: from [IPv6:2607:f0b0:1:3800:7caf:75d:96c1:fd07] (unknown [IPv6:2607:f0b0:1:3800:7caf:75d:96c1:fd07]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: pierre) by mail.userid.org (Postfix) with ESMTP id 426C52C77B4; Sat, 2 Jul 2011 11:33:13 -0400 (EDT) Message-ID: <4E0F3A2D.60409@userid.org> Date: Sat, 02 Jul 2011 11:33:01 -0400 From: Pierre Lamy User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11 MIME-Version: 1.0 To: Fabian Keil References: <201106281157.p5SBvP5g048097@svn.freebsd.org> <20110629192224.2283efc8@fabiankeil.de> In-Reply-To: <20110629192224.2283efc8@fabiankeil.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-userid-MailScanner-Information: Please contact the ISP for more information X-userid-MailScanner-ID: 426C52C77B4.A44A0 X-userid-MailScanner: Found to be clean X-userid-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=0.599, required 6, J_CHICKENPOX_33 0.60, NO_RELAYS -0.00) X-userid-MailScanner-From: pierre@userid.org X-Spam-Status: No Cc: freebsd-pf@freebsd.org Subject: Re: svn commit: r223637 - in head: . contrib/pf/authpf contrib/pf/ftp-proxy contrib/pf/man contrib/pf/pfctl contrib/pf/pflogd sbin/pflogd sys/conf sys/contrib/altq/altq sys/contrib/pf/net sys/modules s... X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2011 15:40:43 -0000 On 6/29/2011 1:22 PM, Fabian Keil wrote: > "Bjoern A. Zeeb" wrote: > >> Begin forwarded message: >> >>> From: "Bjoern A. Zeeb" >>> Date: June 28, 2011 11:57:25 AM GMT+00:00 >>> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org >>> Subject: svn commit: r223637 - in head: . contrib/pf/authpf contrib/pf/ftp-proxy contrib/pf/man contrib/pf/pfctl contrib/pf/pflogd sbin/pflogd sys/conf sys/contrib/altq/altq sys/contrib/pf/net sys/modules s... >>> >>> Author: bz >>> Date: Tue Jun 28 11:57:25 2011 >>> New Revision: 223637 >>> URL: http://svn.freebsd.org/changeset/base/223637 >>> >>> Log: >>> Update packet filter (pf) code to OpenBSD 4.5. > Thanks! > >> In short; please test! > I didn't experience any real problems yet, but running > Privoxy-Regression-Test, I reproducible got this log message > for one of the tests: > > Jun 29 18:26:19 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo1, stored af=2, a0: 10.0.0.1:50722, a1: 10.0.0.1:12345, proto=6, found af=2, a0: 10.0.0.1:50722, a1: 10.0.0.1:12345, proto=6. > > This didn't happen with the previous pf version. > > I tracked it down to a test that does a connect() > to a local unbound port. > > It's also reproducible for every address on the system with: > > ifconfig -a | awk '/inet / {system("telnet "$2" 12345")}' > > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo0, stored af=2, a0: 192.168.5.49:61512, a1: 192.168.5.49:12345, proto=6, found af=2, a0: 192.168.5.49:61512, a1: 192.168.5.49:12345, proto=6. > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo0, stored af=2, a0: 127.0.0.1:44717, a1: 127.0.0.1:12345, proto=6, found af=2, a0: 127.0.0.1:44717, a1: 127.0.0.1:12345, proto=6. > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo1, stored af=2, a0: 192.168.6.100:31600, a1: 192.168.6.100:12345, proto=6, found af=2, a0: 192.168.6.100:31600, a1: 192.168.6.100:12345, proto=6. > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo1, stored af=2, a0: 10.0.0.1:20126, a1: 10.0.0.1:12345, proto=6, found af=2, a0: 10.0.0.1:20126, a1: 10.0.0.1:12345, proto=6. > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo1, stored af=2, a0: 10.0.0.1:10895, a1: 10.0.0.2:12345, proto=6, found af=2, a0: 10.0.0.1:10895, a1: 10.0.0.2:12345, proto=6. > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo1, stored af=2, a0: 10.0.0.1:25081, a1: 10.0.0.3:12345, proto=6, found af=2, a0: 10.0.0.1:25081, a1: 10.0.0.3:12345, proto=6. > Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=OUT, if=lo0, stored af=2, a0: 192.168.0.106:32448, a1: 192.168.0.106:12345, proto=6, found af=2, a0: 192.168.0.106:32448, a1: 192.168.0.106:12345, proto=6. > > 12345 can be replaced with any unbound port it seems. > > I'm additionally occasionally seeing the message for successfully > established connections (both internal and outgoing) but don't > know how to reproduce it. > > Fabian I also get the state key mismatch problem, it seems that pf is leaking states (I assume this is the same problem). I also see a strange NAT issue, internal IPs leak somewhat on the outside int. Eventually the system runs out of state entry slots and connectivity is lost. This is on a -current kernel from ~Jun 30, after the 4.5 import. tun0: flags=8151 metric 0 mtu 1492 options=80000 inet6 fe80::290:bff:fe1a:a674%tun0 prefixlen 64 scopeid 0xf inet6 2607:f0b0:0:1:290:bff:fe1a:a674 prefixlen 64 autoconf inet 216.106.102.33 --> 209.87.255.1 netmask 0xffffffff nd6 options=23 Opened by PID 3446 em0 is on the 192.168.3/24 network [/var/preserve/root] # tcpdump -i tun0 net 192.168.3.0 mask 255.255.255.0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tun0, link-type NULL (BSD loopback), capture size 65535 bytes 11:22:37.030244 IP 192.168.3.99 > 190.252.34.186: ICMP pandora.userid.org udp port 16881 unreachable, length 134 11:24:03.137016 IP 192.168.3.99 > 190.252.34.186: ICMP pandora.userid.org udp port 16881 unreachable, length 98 Relevant pf.conf lines: int_if = "em0" ext_if = "tun0" # NAT nat on $ext_if from $int_if:network to any -> ($ext_if) Here is the info about states leaking: State Table Total Rate current entries 108488 [/var/preserve/root] # pfctl -F states 1003 states cleared [/var/preserve/root] # pfctl -s info Status: Enabled for 0 days 02:21:18 Debug: Urgent Interface Stats for tun0 IPv4 IPv6 Bytes In 1252327614 1907903 Bytes Out 373783492 1429003 Packets In Passed 1341017 12360 Blocked 45437 831 Packets Out Passed 1186359 13441 Blocked 1641 3724 State Table Total Rate current entries 125127 States aren't getting cleared properly. Below is a sample of the state key linking mismatch problem: Jul 2 11:28:17 pyr7535 kernel: pf: state key linking mismatch! dir=OUT, if=em0, stored af=2, a0: Jul 2 11:28:17 pyr7535 kernel: 192.168.3.238:55590, a1: 216.106.102.33 Jul 2 11:28:18 pyr7535 kernel: :18825, proto=6 Jul 2 11:28:18 pyr7535 kernel: , found af=2, a0: 192.168.3.238 Jul 2 11:28:18 pyr7535 kernel: :55590, a1: Jul 2 11:28:18 pyr7535 kernel: 216.106.102.33:18825 Jul 2 11:28:18 pyr7535 kernel: , proto=6. Jul 2 11:28:18 pyr7535 kernel: pf: state key linking mismatch! dir=OUT, if=em0, stored af=2, a0: 192.168.3.238:55590, a1: 216.106.102.33:18825, proto=6, found af=2, a0: 192.168.3.238:55590, a1: 216.106.102.33:18825, proto=6. Jul 2 11:28:19 pyr7535 kernel: pf: state key linking mismatch! dir=OUT, if=em0, stored af=2, a0: 192.168.3.238 Jul 2 11:28:19 pyr7535 kernel: :55590, a1: Jul 2 11:28:19 pyr7535 kernel: 216.106.102.33:18825 Jul 2 11:28:19 pyr7535 kernel: , proto=6, found af=2, a0: Jul 2 11:28:19 pyr7535 kernel: 192.168.3.238:55590 Jul 2 11:28:19 pyr7535 kernel: , a1: 216.106.102.33 Jul 2 11:28:19 pyr7535 kernel: :18825, proto=6.