Date: Wed, 19 Dec 2007 09:18:15 +0200 From: Silver Salonen <silver.salonen@gmail.com> To: freebsd-pf@freebsd.org Subject: Re: occasional "Operation not permitted" on state-mismatch Message-ID: <200712190918.16730.silver.salonen@gmail.com> In-Reply-To: <200712181828.04998.max@love2party.net> References: <200712180934.58755.silver.salonen@gmail.com> <14397207.post@talk.nabble.com> <200712181828.04998.max@love2party.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tuesday 18 December 2007 19:27, Max Laier wrote: > On Tuesday 18 December 2007, Atrox wrote: > > Atrox wrote: > > > Hello! > > > > > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > > > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN > > > LAN-to-LAN > > > and the problem is that a few times per hour connection drops between > > > computers from one LAN to another. At first I blamed OpenVPN, then I > > > blamed > > > bridge, but now I've realized that the problem is in PF. > > > So I've tried increasing TCP-timeouts and setting optimization > > > to "aggressive", but well, it's still the same. > > > > > > I monitor connections by sending TCP packets once per second to some > > > other host and wait for reply. I use Nagios-plugins' check_tcp for > > > that. The script > > > looks like: > > > ===== > > > while [ 1 ]; do > > > pfctl -si |grep mismatch > > > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > > > pfctl -si |grep mismatch > > > sleep 1 > > > done > > > ===== > > > > > > So if I let this script into action, I see that in 2-3 minutes, > > > check_tcp gets "Operation not permitted" error and just in this > > > moment packet-mismatch > > > counter is increased by one (on machine with lesser traffic, I get > > > the timeout > > > in a few hours). That's on both 6.3-PRERELEASE as well as on > > > 6.2-RELEASE. I've > > > tried connections: > > > * along WAN to IPFW-enabled machines > > > * along WAN to PF-enabled machines > > > * along LAN to PF-enabled machines > > > * along LAN to Windows machines > > > * along VPN to PF-enabled machines > > > * along VPN to Windows machines > > > > > > Sometimes I get just some connection timeout: CRITICAL - Socket > > > timeout after > > > 2 seconds (I don't know what could cause that). > > > > > > I can see this behaviour in about every FreeBSD/PF machine I have. > > > > > > The basic PF-configuration looks like: > > > ===== > > > set block-policy return > > > set loginterface $ext_if > > > set timeout tcp.closed 15 > > > set optimization aggressive > > > scrub in all no-df > > > > > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > > block log all > > > pass quick on lo0 all > > > pass out all modulate state > > > pass out proto tcp all flags S/SA modulate state > > > pass on $int_if all modulate state > > > pass on $int_if proto tcp all flags S/SA modulate state > > > pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services > > > flags > > > S/SA modulate state > > > ===== > > > > > > Is PF buggy or have I misconfigured smth? > > > > Today I installed an OpenBSD-4.2 box just to see whether PF does the > > same thing there. And yes, it does. > > pf.conf: > > ===== > > ext_if = rl0 > > set block-policy return > > set loginterface $ext_if > > scrub in all no-df > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > pass all modulate state > > pass quick on lo0 all > > ===== > > > > I check TCP without "sleep 1" now, and I do it to FreeBSD box without > > firewall. state-mismatch gets increased by one, and I get either "No > > route to host" or "Socket timeout after 2 seconds". > > > > Am I still misconfiguring the thing? > > No idea, you don't give nearly enough data for us to figure out what your > setup looks like. Well, as for OpenBSD, I installed the default system and changed only the pf.conf, and it sits in LAN (.../24). It's quite plain actually. > Regular tcp state mismatch usually hints that pf isn't > seeing all packets of the conversation. This can be caused by triangular > routing, load balanceing or if_bridge (which is difficult to get right in > some scenarios). Does using if_bridge matter when I test connections on non-bridged interfaces? I see state-mismatch while testing TCP-connection on servers' external interfaces (along WAN) and these are not in bridge. -- Silver
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200712190918.16730.silver.salonen>