From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 17:28:17 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE94916A419 for ; Tue, 18 Dec 2007 17:28:16 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id 6CADF13C4D3 for ; Tue, 18 Dec 2007 17:28:16 +0000 (UTC) (envelope-from max@love2party.net) Received: from amd64.laiers.local (dslb-088-066-047-246.pools.arcor-ip.net [88.66.47.246]) by mrelayeu.kundenserver.de (node=mrelayeu8) with ESMTP (Nemesis) id 0ML31I-1J4gEo2dmI-0003Lc; Tue, 18 Dec 2007 18:28:15 +0100 From: Max Laier Organization: FreeBSD To: freebsd-pf@freebsd.org Date: Tue, 18 Dec 2007 18:27:57 +0100 User-Agent: KMail/1.9.7 References: <200712180934.58755.silver.salonen@gmail.com> <14397207.post@talk.nabble.com> In-Reply-To: <14397207.post@talk.nabble.com> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1995150.QsF0deyv5T"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200712181828.04998.max@love2party.net> X-Provags-ID: V01U2FsdGVkX19uopLj4yUVWTEUIobSISenSPKuK8ePhW1jsN0 LkxzBOC1UOfbtsmWDalgt8e/7ul8SzmubNpEaLlumxwBEifU9L 1xoV0ZVeQduTec7eTF6eh9neWlm1JlwZiDYANdVQrQ= Cc: Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 17:28:17 -0000 --nextPart1995150.QsF0deyv5T Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Tuesday 18 December 2007, Atrox wrote: > Atrox wrote: > > Hello! > > > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN > > LAN-to-LAN > > and the problem is that a few times per hour connection drops between > > computers from one LAN to another. At first I blamed OpenVPN, then I > > blamed > > bridge, but now I've realized that the problem is in PF. > > So I've tried increasing TCP-timeouts and setting optimization > > to "aggressive", but well, it's still the same. > > > > I monitor connections by sending TCP packets once per second to some > > other host and wait for reply. I use Nagios-plugins' check_tcp for > > that. The script > > looks like: > > =3D=3D=3D=3D=3D > > while [ 1 ]; do > > pfctl -si |grep mismatch > > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > > pfctl -si |grep mismatch > > sleep 1 > > done > > =3D=3D=3D=3D=3D > > > > So if I let this script into action, I see that in 2-3 minutes, > > check_tcp gets "Operation not permitted" error and just in this > > moment packet-mismatch > > counter is increased by one (on machine with lesser traffic, I get > > the timeout > > in a few hours). That's on both 6.3-PRERELEASE as well as on > > 6.2-RELEASE. I've > > tried connections: > > * along WAN to IPFW-enabled machines > > * along WAN to PF-enabled machines > > * along LAN to PF-enabled machines > > * along LAN to Windows machines > > * along VPN to PF-enabled machines > > * along VPN to Windows machines > > > > Sometimes I get just some connection timeout: CRITICAL - Socket > > timeout after > > 2 seconds (I don't know what could cause that). > > > > I can see this behaviour in about every FreeBSD/PF machine I have. > > > > The basic PF-configuration looks like: > > =3D=3D=3D=3D=3D > > set block-policy return > > set loginterface $ext_if > > set timeout tcp.closed 15 > > set optimization aggressive > > scrub in all no-df > > > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > block log all > > pass quick on lo0 all > > pass out all modulate state > > pass out proto tcp all flags S/SA modulate state > > pass on $int_if all modulate state > > pass on $int_if proto tcp all flags S/SA modulate state > > pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services > > flags > > S/SA modulate state > > =3D=3D=3D=3D=3D > > > > Is PF buggy or have I misconfigured smth? > > Today I installed an OpenBSD-4.2 box just to see whether PF does the > same thing there. And yes, it does. > pf.conf: > =3D=3D=3D=3D=3D > ext_if =3D rl0 > set block-policy return > set loginterface $ext_if > scrub in all no-df > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > pass all modulate state > pass quick on lo0 all > =3D=3D=3D=3D=3D > > I check TCP without "sleep 1" now, and I do it to FreeBSD box without > firewall. state-mismatch gets increased by one, and I get either "No > route to host" or "Socket timeout after 2 seconds". > > Am I still misconfiguring the thing? No idea, you don't give nearly enough data for us to figure out what your=20 setup looks like. Regular tcp state mismatch usually hints that pf isn't=20 seeing all packets of the conversation. This can be caused by triangular=20 routing, load balanceing or if_bridge (which is difficult to get right in=20 some scenarios). You should figure out the exact path your tcp packets are taking (back and= =20 forth) and make sure pf sees all of them. Enabling additional pf logging=20 (pfctl -xm) helps to figure out what kind of mismatch is happening. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart1995150.QsF0deyv5T Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHaAMkXyyEoT62BG0RAgT0AJ4qMX4Jxf3ZciJqtuWa0JAl+SyQYgCeINVX ah60YsBRsxQNfPukoMyCiFA= =/03A -----END PGP SIGNATURE----- --nextPart1995150.QsF0deyv5T--