From owner-freebsd-pf@FreeBSD.ORG Tue Jul 5 13:48:05 2011 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84FDC106566C for ; Tue, 5 Jul 2011 13:48:05 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay06.ispgateway.de (smtprelay06.ispgateway.de [80.67.31.95]) by mx1.freebsd.org (Postfix) with ESMTP id F2D028FC12 for ; Tue, 5 Jul 2011 13:48:04 +0000 (UTC) Received: from [87.79.155.68] (helo=fabiankeil.de) by smtprelay06.ispgateway.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1Qe5ys-0007HT-VZ; Tue, 05 Jul 2011 15:48:03 +0200 Date: Tue, 5 Jul 2011 15:47:47 +0200 From: Fabian Keil To: Ermal =?ISO-8859-1?Q?Lu=E7i?= Message-ID: <20110705154747.547ac919@fabiankeil.de> In-Reply-To: References: <201106281157.p5SBvP5g048097@svn.freebsd.org> <20110629192224.2283efc8@fabiankeil.de> <4E0F3A2D.60409@userid.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/G6gQI.DPbwhVgPmFOuCczl3"; protocol="application/pgp-signature" X-Df-Sender: 775067 Cc: freebsd-pf@freebsd.org Subject: Re: svn commit: r223637 - in head: . contrib/pf/authpf contrib/pf/ftp-proxy contrib/pf/man contrib/pf/pfctl contrib/pf/pflogd sbin/pflogd sys/conf sys/contrib/altq/altq sys/contrib/pf/net sys/modules s... X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jul 2011 13:48:05 -0000 --Sig_/G6gQI.DPbwhVgPmFOuCczl3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Ermal Lu=E7i wrote: > On Sat, Jul 2, 2011 at 5:33 PM, Pierre Lamy wrote: > > > > > > On 6/29/2011 1:22 PM, Fabian Keil wrote: > >> > >> "Bjoern A. Zeeb" =A0wrote: > >> > >>> Begin forwarded message: > >>> > >>>> From: "Bjoern A. Zeeb" > >>>> Date: June 28, 2011 11:57:25 AM GMT+00:00 > >>>> To: src-committers@freebsd.org, svn-src-all@freebsd.org, > >>>> svn-src-head@freebsd.org > >>>> Subject: svn commit: r223637 - in head: . contrib/pf/authpf > >>>> contrib/pf/ftp-proxy contrib/pf/man contrib/pf/pfctl contrib/pf/pflo= gd > >>>> sbin/pflogd sys/conf sys/contrib/altq/altq sys/contrib/pf/net sys/mo= dules > >>>> s... > >>>> > >>>> Author: bz > >>>> Date: Tue Jun 28 11:57:25 2011 > >>>> New Revision: 223637 > >>>> URL: http://svn.freebsd.org/changeset/base/223637 > >>>> > >>>> Log: > >>>> =A0Update packet filter (pf) code to OpenBSD 4.5. > >> > >> Thanks! > >> > >>> In short; please test! > >> > >> I didn't experience any real problems yet, but running > >> Privoxy-Regression-Test, I reproducible got this log message > >> for one of the tests: > >> > >> Jun 29 18:26:19 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo1, stored af=3D2, a0: 10.0.0.1:50722, a1: 10.0.0.1:12345, proto= =3D6, found > >> af=3D2, a0: 10.0.0.1:50722, a1: 10.0.0.1:12345, proto=3D6. > >> > >> This didn't happen with the previous pf version. > >> > >> I tracked it down to a test that does a connect() > >> to a local unbound port. > >> > >> It's also reproducible for every address on the system with: > >> > >> ifconfig -a | awk '/inet / {system("telnet "$2" 12345")}' > >> > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo0, stored af=3D2, a0: 192.168.5.49:61512, a1: 192.168.5.49:1234= 5, > >> proto=3D6, found af=3D2, a0: 192.168.5.49:61512, a1: 192.168.5.49:1234= 5, > >> proto=3D6. > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo0, stored af=3D2, a0: 127.0.0.1:44717, a1: 127.0.0.1:12345, pro= to=3D6, > >> found af=3D2, a0: 127.0.0.1:44717, a1: 127.0.0.1:12345, proto=3D6. > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo1, stored af=3D2, a0: 192.168.6.100:31600, a1: 192.168.6.100:12= 345, > >> proto=3D6, found af=3D2, a0: 192.168.6.100:31600, a1: 192.168.6.100:12= 345, > >> proto=3D6. > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo1, stored af=3D2, a0: 10.0.0.1:20126, a1: 10.0.0.1:12345, proto= =3D6, found > >> af=3D2, a0: 10.0.0.1:20126, a1: 10.0.0.1:12345, proto=3D6. > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo1, stored af=3D2, a0: 10.0.0.1:10895, a1: 10.0.0.2:12345, proto= =3D6, found > >> af=3D2, a0: 10.0.0.1:10895, a1: 10.0.0.2:12345, proto=3D6. > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo1, stored af=3D2, a0: 10.0.0.1:25081, a1: 10.0.0.3:12345, proto= =3D6, found > >> af=3D2, a0: 10.0.0.1:25081, a1: 10.0.0.3:12345, proto=3D6. > >> Jun 29 18:30:49 r500 kernel: pf: state key linking mismatch! dir=3DOUT, > >> if=3Dlo0, stored af=3D2, a0: 192.168.0.106:32448, a1: 192.168.0.106:12= 345, > >> proto=3D6, found af=3D2, a0: 192.168.0.106:32448, a1: 192.168.0.106:12= 345, > >> proto=3D6. > >> > >> 12345 can be replaced with any unbound port it seems. > >> > >> I'm additionally occasionally seeing the message for successfully > >> established connections (both internal and outgoing) but don't > >> know how to reproduce it. > >> > >> Fabian > > > > I also get the state key mismatch problem, it seems that pf is leaking > > states (I assume this is the same problem). I also see a strange NAT is= sue, > > internal IPs leak somewhat on the outside int. Eventually the system ru= ns > > out of state entry slots and connectivity is lost. This is on a -current > > kernel from ~Jun 30, after the 4.5 import. > > > > tun0: flags=3D8151 metric 0 m= tu 1492 > > =A0 =A0 =A0 =A0options=3D80000 > > =A0 =A0 =A0 =A0inet6 fe80::290:bff:fe1a:a674%tun0 prefixlen 64 scopeid = 0xf > > =A0 =A0 =A0 =A0inet6 2607:f0b0:0:1:290:bff:fe1a:a674 prefixlen 64 autoc= onf > > =A0 =A0 =A0 =A0inet 216.106.102.33 --> 209.87.255.1 netmask 0xffffffff > > =A0 =A0 =A0 =A0nd6 options=3D23 > > =A0 =A0 =A0 =A0Opened by PID 3446 > > > > em0 is on the 192.168.3/24 network > > > > [/var/preserve/root] # tcpdump -i tun0 net 192.168= .3.0 > > mask 255.255.255.0 > > tcpdump: verbose output suppressed, use -v or -vv for full protocol dec= ode > > listening on tun0, link-type NULL (BSD loopback), capture size 65535 by= tes > > 11:22:37.030244 IP 192.168.3.99 > 190.252.34.186: ICMP pandora.userid.o= rg > > udp port 16881 unreachable, length 134 > > 11:24:03.137016 IP 192.168.3.99 > 190.252.34.186: ICMP pandora.userid.o= rg > > udp port 16881 unreachable, length 98 > > > > Relevant pf.conf lines: > > int_if =3D "em0" > > ext_if =3D "tun0" > > # NAT > > nat on $ext_if from $int_if:network to any -> ($ext_if) > > > > Here is the info about states leaking: > > > > State Table =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Total = =A0 =A0 =A0 =A0 =A0 =A0 Rate > > =A0current entries =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 108488 > > > > [/var/preserve/root] # pfctl -F states > > 1003 states cleared > > [/var/preserve/root] # pfctl -s info > > Status: Enabled for 0 days 02:21:18 =A0 =A0 =A0 =A0 =A0 Debug: Urgent > > > > Interface Stats for tun0 =A0 =A0 =A0 =A0 =A0 =A0 =A0IPv4 =A0 =A0 =A0 = =A0 =A0 =A0 IPv6 > > =A0Bytes In =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A01252327614 =A0 = =A0 =A0 =A0 =A01907903 > > =A0Bytes Out =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0373783492 =A0 = =A0 =A0 =A0 =A01429003 > > =A0Packets In > > =A0 =A0Passed =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 1341017 = =A0 =A0 =A0 =A0 =A0 =A012360 > > =A0 =A0Blocked =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A045437= =A0 =A0 =A0 =A0 =A0 =A0 =A0831 > > =A0Packets Out > > =A0 =A0Passed =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 1186359 = =A0 =A0 =A0 =A0 =A0 =A013441 > > =A0 =A0Blocked =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 1641= =A0 =A0 =A0 =A0 =A0 =A0 3724 > > > > State Table =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Total = =A0 =A0 =A0 =A0 =A0 =A0 Rate > > =A0current entries =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 125127 > > > > States aren't getting cleared properly. Below is a sample of the state = key > > linking mismatch problem: > > > > Jul =A02 11:28:17 pyr7535 kernel: pf: state key linking mismatch! dir= =3DOUT, > > if=3Dem0, stored af=3D2, a0: >=20 > I just committed a fix for the state key linking mismatch issue. > Can you test with the latest HEAD sources? Works for me, at least the error messages are gone. Thanks a lot. I never experienced the "state leak issue" Pierre described above, but it does seem to take a while until cleared states are reported as such: fk@r500 ~ $sudo pfctl -s all | grep current current entries 1556 fk@r500 ~ $sudo pfctl -F states 1556 states cleared fk@r500 ~ $sudo pfctl -s all | grep current current entries 1259 fk@r500 ~ $sudo pfctl -s all | grep current current entries 1133 fk@r500 ~ $sudo pfctl -s all | grep current current entries 1019 fk@r500 ~ $sudo pfctl -F states 0 states cleared fk@r500 ~ $sudo pfctl -s all | grep current current entries 742 fk@r500 ~ $sudo pfctl -s all | grep current current entries 667 fk@r500 ~ $sudo pfctl -s all | grep current current entries 667 fk@r500 ~ $sudo pfctl -F states 0 states cleared fk@r500 ~ $sudo pfctl -s all | grep current current entries 436 fk@r500 ~ $sudo pfctl -s all | grep current current entries 436 fk@r500 ~ $sudo pfctl -s all | grep current current entries 352 fk@r500 ~ $sudo pfctl -F states 0 states cleared fk@r500 ~ $sudo pfctl -s all | grep current current entries 185 I never looked at this before, so it might have always behaved that way. Fabian --Sig_/G6gQI.DPbwhVgPmFOuCczl3 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iEYEARECAAYFAk4TFg8ACgkQBYqIVf93VJ0UAgCgp9AHqmczWUWSemuKC0WIhPvT SmMAni58NxcpnoKMRpzXjjEjbTw8yzgy =btE1 -----END PGP SIGNATURE----- --Sig_/G6gQI.DPbwhVgPmFOuCczl3--