From owner-freebsd-pf@FreeBSD.ORG Sun Dec 16 01:53:38 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CCC716A419; Sun, 16 Dec 2007 01:53:38 +0000 (UTC) (envelope-from bms@FreeBSD.org) Received: from out3.smtp.messagingengine.com (out3.smtp.messagingengine.com [66.111.4.27]) by mx1.freebsd.org (Postfix) with ESMTP id 0016E13C455; Sun, 16 Dec 2007 01:53:37 +0000 (UTC) (envelope-from bms@FreeBSD.org) Received: from compute1.internal (compute1.internal [10.202.2.41]) by out1.messagingengine.com (Postfix) with ESMTP id 4F7667A81C; Sat, 15 Dec 2007 20:53:37 -0500 (EST) Received: from heartbeat1.messagingengine.com ([10.202.2.160]) by compute1.internal (MEProxy); Sat, 15 Dec 2007 20:53:37 -0500 X-Sasl-enc: i0i26hJIk+C1cE7IweXItrQkKeDv1sFLAc7IRl1mE99g 1197770017 Received: from empiric.lon.incunabulum.net (82-35-112-254.cable.ubr07.dals.blueyonder.co.uk [82.35.112.254]) by mail.messagingengine.com (Postfix) with ESMTP id 8AEFEC1D6; Sat, 15 Dec 2007 20:53:36 -0500 (EST) Message-ID: <4764851F.1000304@FreeBSD.org> Date: Sun, 16 Dec 2007 01:53:35 +0000 From: "Bruce M. Simpson" User-Agent: Thunderbird 2.0.0.6 (X11/20070928) MIME-Version: 1.0 To: Max Laier References: <47628E11.7030803@tomjudge.com> <4762AC1E.3030101@FreeBSD.org> <200712142030.14728.max@love2party.net> In-Reply-To: <200712142030.14728.max@love2party.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net , freebsd-pf@freebsd.org Subject: Re: Spurious error from i[pf]_carp X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2007 01:53:38 -0000 Max Laier wrote: > Alternatively you could change IPPROTO_CARP in netinet/in.h to another > unused protocol number. This is really the preferred way of dealing with > mixed CARP and VRRP environments as the CARP packets might in turn > irritate the VRRP routers, too. > This sounds like a common use case. Perhaps there is motivation for making the protocol number used by CARP a loader tunable? [I'd really like it if we had a kernel API for adding the virtual MAC addresses to ifnet too, then again I'd like the cheat for infinite chocolate fudge sundaes in life, bed and breakfast at The Savoy with my choice of actress, etc] > /* no comment */ > No disrespect to anyone intended, just that CARP does duplicate the functionality of VRRP. It's worth reiterating that this is what happens when software patents are allowed to creep in to the nuts and bolts of the operational Internet -- and thus, CARP was born, and thus Tom runs into the issue he has seen. later BMS From owner-freebsd-pf@FreeBSD.ORG Sun Dec 16 16:21:07 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF6FB16A46B for ; Sun, 16 Dec 2007 16:21:07 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from smtp805.mail.ird.yahoo.com (smtp805.mail.ird.yahoo.com [217.146.188.65]) by mx1.freebsd.org (Postfix) with SMTP id 0B89913C447 for ; Sun, 16 Dec 2007 16:21:06 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: (qmail 21377 invoked from network); 16 Dec 2007 15:54:24 -0000 Received: from unknown (HELO ?192.168.1.2?) (thomasjudge@btinternet.com@86.139.146.42 with plain) by smtp805.mail.ird.yahoo.com with SMTP; 16 Dec 2007 15:54:24 -0000 X-YMail-OSG: vwv9nTQVM1myiwPXjo0O5OiOoXoeTIE0AwJqPLtNK_l99g53 Message-ID: <47654B58.7070500@tomjudge.com> Date: Sun, 16 Dec 2007 15:59:20 +0000 From: Tom Judge User-Agent: Thunderbird 1.5.0.13 (X11/20070824) MIME-Version: 1.0 To: "Bruce M. Simpson" References: <47628E11.7030803@tomjudge.com> <4762AC1E.3030101@FreeBSD.org> <200712142030.14728.max@love2party.net> <4764851F.1000304@FreeBSD.org> In-Reply-To: <4764851F.1000304@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-pf@freebsd.org, freebsd-net Subject: Re: Spurious error from i[pf]_carp X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2007 16:21:07 -0000 Bruce M. Simpson wrote: > Max Laier wrote: >> Alternatively you could change IPPROTO_CARP in netinet/in.h to another >> unused protocol number. This is really the preferred way of dealing >> with mixed CARP and VRRP environments as the CARP packets might in >> turn irritate the VRRP routers, too. >> This seems to make the most sense to me. At this time it seems (in RELENG_6_2 at least) that because the protocol number is shared with VRRP that tcpdump tries to decode the CARP frames as VRRP frames and although the header/frame is very simple this does not provide a useful decoding of the CARP frame. After the protocol number is changed it would be possible to write a proper carp decoder for tcpdump or at least make any existing decoder be able to tell the difference between VRRP and CARP frames. > This sounds like a common use case. Perhaps there is motivation for > making the protocol number used by CARP a loader tunable? > > [I'd really like it if we had a kernel API for adding the virtual MAC > addresses to ifnet too, then again I'd like the cheat for infinite > chocolate fudge sundaes in life, bed and breakfast at The Savoy with my > choice of actress, etc] >> /* no comment */ >> > No disrespect to anyone intended, just that CARP does duplicate the > functionality of VRRP. > Please correct me if I am wrong, from the limited research I have done, carp was born because Cisco made a patent claim (based on its patents for HSRP) against a VRRP implementation. > It's worth reiterating that this is what happens when software patents > are allowed to creep in to the nuts and bolts of the operational > Internet -- and thus, CARP was born, and thus Tom runs into the issue he > has seen. > > later > BMS > Thoughts? Tom From owner-freebsd-pf@FreeBSD.ORG Mon Dec 17 11:07:02 2007 Return-Path: Delivered-To: freebsd-pf@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFD0C16A4E6 for ; Mon, 17 Dec 2007 11:07:02 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B2BB813C4D3 for ; Mon, 17 Dec 2007 11:07:02 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id lBHB72LX088307 for ; Mon, 17 Dec 2007 11:07:02 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id lBHB71d6088303 for freebsd-pf@FreeBSD.org; Mon, 17 Dec 2007 11:07:01 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Dec 2007 11:07:01 GMT Message-Id: <200712171107.lBHB71d6088303@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-pf@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-pf@FreeBSD.org X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2007 11:07:02 -0000 Current FreeBSD problem reports Critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/111220 pf [pf] repeatable hangs while manipulating pf tables 1 problem total. Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/82271 pf [pf] cbq scheduler cause bad latency o kern/92949 pf [pf] PF + ALTQ problems with latency o kern/110698 pf [pf] nat rule of pf without "on" clause causes invalid o bin/116610 pf [patch] teach tcpdump(1) to cope with the new-style pf o kern/117827 pf [pf] kernel panic with pf and ng 5 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s o kern/93825 pf [pf] pf reply-to doesn't work o kern/106400 pf [pf] fatal trap 12 at restart of PF with ALTQ if ng0 d s conf/110838 pf tagged parameter on nat not working on FreeBSD 5.2 o kern/114095 pf [carp] carp+pf delay with high state limit o kern/114567 pf [pf] LOR pf_ioctl.c + if.c f kern/116645 pf [RFE] pfctl -k does not work in securelevel 3 o kern/118355 pf [pf] [patch] pfctl help message options order false -t 8 problems total. From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 08:03:25 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11DC316A478 for ; Tue, 18 Dec 2007 08:03:25 +0000 (UTC) (envelope-from silver.salonen@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.169]) by mx1.freebsd.org (Postfix) with ESMTP id 9884A13C4E3 for ; Tue, 18 Dec 2007 08:03:24 +0000 (UTC) (envelope-from silver.salonen@gmail.com) Received: by ug-out-1314.google.com with SMTP id y2so75430uge.37 for ; Tue, 18 Dec 2007 00:03:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date:user-agent:mime-version:content-disposition:x-length:x-uid:message-id:content-type:content-transfer-encoding; bh=N4CP9fFj1215kF6VmiJZ1omAo4pyNhpJu7e4lZ10FRk=; b=knwv5xZZ8lj5DsAX+MNmLl8E2fNCr+td913g4w16gvBmNI0hq1cNzIW97tAafvQGXrCgFd4l05knI17C4FEg1YJ/LaDV4Hjqhd20Kp5erzKAWsXzLqBG3ch4vL3WS5NJ5Xh2LGZxqpWvAUHqld8tMhwRqQu/cv7IAgmJBUUevFo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:content-disposition:x-length:x-uid:message-id:content-type:content-transfer-encoding; b=KuGKSaFoOlCyh5f2Ihvp0k7kzWCeiIz4+iT9FR5kpFekBGJ7PxmSLo0G8I2Cw5eDv3TV4vta2lDJG8aWXu+neRaTqWmE9TJHkM99isrVOh8SY5QdY9CMLSoIYx6gR/nluz1jAGrcmjaz4wpuuNrdLjAczOMMAfk28kiJGHxoEzY= Received: by 10.67.29.12 with SMTP id g12mr519459ugj.12.1197963313720; Mon, 17 Dec 2007 23:35:13 -0800 (PST) Received: from ?192.168.8.99? ( [195.50.198.178]) by mx.google.com with ESMTPS id f19sm18816767fka.2007.12.17.23.35.08 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 17 Dec 2007 23:35:08 -0800 (PST) From: Silver Salonen To: freebsd-pf@freebsd.org Date: Tue, 18 Dec 2007 09:34:58 +0200 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Disposition: inline X-Length: 749 X-UID: 49 Message-Id: <200712180934.58755.silver.salonen@gmail.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 08:03:25 -0000 Hello! I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN LAN-to-LAN and the problem is that a few times per hour connection drops between computers from one LAN to another. At first I blamed OpenVPN, then I blamed bridge, but now I've realized that the problem is in PF. So I've tried increasing TCP-timeouts and setting optimization to "aggressive", but well, it's still the same. I monitor connections by sending TCP packets once per second to some other host and wait for reply. I use Nagios-plugins' check_tcp for that. The script looks like: ===== while [ 1 ]; do pfctl -si |grep mismatch /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 pfctl -si |grep mismatch sleep 1 done ===== So if I let this script into action, I see that in 2-3 minutes, check_tcp gets "Operation not permitted" error and just in this moment packet-mismatch counter is increased by one (on machine with lesser traffic, I get the timeout in a few hours). That's on both 6.3-PRERELEASE as well as on 6.2-RELEASE. I've tried connections: * along WAN to IPFW-enabled machines * along WAN to PF-enabled machines * along LAN to PF-enabled machines * along LAN to Windows machines * along VPN to PF-enabled machines * along VPN to Windows machines Sometimes I get just some connection timeout: CRITICAL - Socket timeout after 2 seconds (I don't know what could cause that). I can see this behaviour in about every FreeBSD/PF machine I have. The basic PF-configuration looks like: ===== set block-policy return set loginterface $ext_if set timeout tcp.closed 15 set optimization aggressive scrub in all no-df block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 block log all pass quick on lo0 all pass out all modulate state pass out proto tcp all flags S/SA modulate state pass on $int_if all modulate state pass on $int_if proto tcp all flags S/SA modulate state pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services flags S/SA modulate state ===== Is PF buggy or have I misconfigured smth? -- Silver From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 14:09:07 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D840316A469 for ; Tue, 18 Dec 2007 14:09:07 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id A0A7713C448 for ; Tue, 18 Dec 2007 14:09:07 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1J4cnQ-00045a-1Z for freebsd-pf@freebsd.org; Tue, 18 Dec 2007 05:47:44 -0800 Message-ID: <14397207.post@talk.nabble.com> Date: Tue, 18 Dec 2007 05:47:44 -0800 (PST) From: Atrox To: freebsd-pf@freebsd.org In-Reply-To: <200712180934.58755.silver.salonen@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: silver.salonen@gmail.com References: <200712180934.58755.silver.salonen@gmail.com> Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 14:09:07 -0000 Atrox wrote: > > Hello! > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN > LAN-to-LAN > and the problem is that a few times per hour connection drops between > computers from one LAN to another. At first I blamed OpenVPN, then I > blamed > bridge, but now I've realized that the problem is in PF. > So I've tried increasing TCP-timeouts and setting optimization > to "aggressive", but well, it's still the same. > > I monitor connections by sending TCP packets once per second to some other > host and wait for reply. I use Nagios-plugins' check_tcp for that. The > script > looks like: > ===== > while [ 1 ]; do > pfctl -si |grep mismatch > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > pfctl -si |grep mismatch > sleep 1 > done > ===== > > So if I let this script into action, I see that in 2-3 minutes, check_tcp > gets "Operation not permitted" error and just in this moment > packet-mismatch > counter is increased by one (on machine with lesser traffic, I get the > timeout > in a few hours). That's on both 6.3-PRERELEASE as well as on 6.2-RELEASE. > I've > tried connections: > * along WAN to IPFW-enabled machines > * along WAN to PF-enabled machines > * along LAN to PF-enabled machines > * along LAN to Windows machines > * along VPN to PF-enabled machines > * along VPN to Windows machines > > Sometimes I get just some connection timeout: CRITICAL - Socket timeout > after > 2 seconds (I don't know what could cause that). > > I can see this behaviour in about every FreeBSD/PF machine I have. > > The basic PF-configuration looks like: > ===== > set block-policy return > set loginterface $ext_if > set timeout tcp.closed 15 > set optimization aggressive > scrub in all no-df > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > block log all > pass quick on lo0 all > pass out all modulate state > pass out proto tcp all flags S/SA modulate state > pass on $int_if all modulate state > pass on $int_if proto tcp all flags S/SA modulate state > pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services > flags > S/SA modulate state > ===== > > Is PF buggy or have I misconfigured smth? > Today I installed an OpenBSD-4.2 box just to see whether PF does the same thing there. And yes, it does. pf.conf: ===== ext_if = rl0 set block-policy return set loginterface $ext_if scrub in all no-df block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 pass all modulate state pass quick on lo0 all ===== I check TCP without "sleep 1" now, and I do it to FreeBSD box without firewall. state-mismatch gets increased by one, and I get either "No route to host" or "Socket timeout after 2 seconds". Am I still misconfiguring the thing? -- Silver -- View this message in context: http://www.nabble.com/occasional-%22Operation-not-permitted%22-on-state-mismatch-tp14392406p14397207.html Sent from the freebsd-pf mailing list archive at Nabble.com. From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 17:01:42 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE58716A420 for ; Tue, 18 Dec 2007 17:01:42 +0000 (UTC) (envelope-from jstewart@fusionary.com) Received: from exprod7og103.obsmtp.com (exprod7og103.obsmtp.com [64.18.2.159]) by mx1.freebsd.org (Postfix) with SMTP id 1242F13C4EB for ; Tue, 18 Dec 2007 17:01:41 +0000 (UTC) (envelope-from jstewart@fusionary.com) Received: from source ([72.14.220.157]) by exprod7ob103.postini.com ([64.18.6.12]) with SMTP; Tue, 18 Dec 2007 09:01:40 PST Received: by fg-out-1718.google.com with SMTP id d23so562366fga.31 for ; Tue, 18 Dec 2007 09:01:40 -0800 (PST) Received: by 10.86.79.19 with SMTP id c19mr7815515fgb.67.1197995591160; Tue, 18 Dec 2007 08:33:11 -0800 (PST) Received: by 10.86.33.5 with HTTP; Tue, 18 Dec 2007 08:33:11 -0800 (PST) Message-ID: <69d690bd0712180833l3b67ec76m7f76281cf3f3a07a@mail.gmail.com> Date: Tue, 18 Dec 2007 11:33:11 -0500 From: "Jason Stewart" To: freebsd-pf@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: HFSC and NAT Problems X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 17:01:43 -0000 Hi, I'm having an issue with HFSC where anything coming over my internal interface NATed to the outside is not being placed in the default queue on the external IF. I have a DMZ that's not NATed and packets coming out of the DMZ are being placed in the queue, so I have reason to believe that NAT may be the culprit. Here is my ruleset (IP addresses have been changed to protect the innocent): # pf.conf # 12/14/2007 JMS #### Macros: define common values, so they can be referenced and changed easily. # Interfaces internet = "fxp0" dmz1 = "fxp1" dmz2 = "fxp2" lan = "fxp3" # Networks lan_net = "192.168.0.0/24" outside_net = "x.x.x.x/xx" dmz1_net = "x.x.x.x/xx" dmz2_net = "x.x.x.x/xx" multicast = "224.0.0.251" # Key Machines asterisk = "192.168.0.10" external_ip = "x.x.x.x" dns_servers = "{x.x.x.x x.x.x.x}" rogue = "x.x.x.x" # Macros dmz2_svcs = "{www https 8080 smtp pop3 imap pop3s imaps ssh ftp >1024}" ldap = "{ldap ldaps}" sql = "{1433 3306}" netbios_tcp = "{139}" netbios_udp = "{137 138}" netbios_tcp_udp = "{445 135}" voip = "{5059:5082 8000:20000 4569}" rogue_svcs = "{687 625 311 8079:9000}" #### End Macros # Tables: similar to macros, but more flexible for many addresses. table { x.x.x.x x.x.x.x x.x.x.x x.x.x.x } ### Options set block-policy return set loginterface $internet set skip on lo0 scrub in all #### End Options #### Queueing and traqffic shaping altq on $internet hfsc bandwidth 1.5Mb queue { std, voip, tcpack } queue std bandwidth 33% priority 1 hfsc (default) queue voip bandwidth 34% priority 7 hfsc (realtime 60%) queue tcpack bandwidth 33% priority 6 hfsc (red realtime 20%) #### End Queueing ### NAT/RDR Rules nat on $internet from $lan_net to any -> ($internet) nat-anchor "ftp-proxy/*" rdr-anchor "ftp-proxy/*" rdr pass on $lan proto tcp from any to any port 21 -> 127.0.0.1 port 8021 rdr pass on $internet proto { udp tcp } from ! $lan_net to $external_ip port $voip tag VOIP -> $asterisk #### End NAT rules #### Filtering rules ## Rules for all interfaces nchor "ftp-proxy/*" # POLICY: Block all incoming by default, only filter incoming to make life easier. block in log all antispoof quick for { lo0 $lan } block in log quick on ! $lan proto tcp from any to any port 8021 # Allow mDNS reflector pass in quick on ! $internet proto udp from any to $multicast port 5353 ## ------------------------------------------------------------------------- ## Internet (fxp0) ## ------------------------------------------------------------------------- # Skip Logging, faster response to ident block in quick on $internet proto {tcp udp} from any to $external_ip port 41899 block return-rst in quick on $internet proto tcp from any to any port 113 # Allow traffic from this machine (I thought the pass out catch all rule would be sufficient, # but for reasons that I don't understand it does not work) pass out quick on $internet from $external_ip to any keep state queue(std) # SQL For both DMZs pass in quick on $internet proto tcp from any to {$dmz1_net $dmz2_net} port $sql keep state # Incoming from net into DMZ2 Allowed Services pass in quick on {$internet $dmz2} proto { tcp udp } from any to $dns_servers port domain keep state pass in quick on {$internet $dmz2} proto tcp from any to $dmz2_net port $dmz2_svcs keep state pass in quick on {$internet $dmz2} proto tcp from any to $rogue port $rogue_svcs keep state # Outgoing SQL connections from DMZ2 pass in quick on {$internet $dmz2} proto tcp from $dmz2_net to any port $sql keep state ## Push ident through faster pass out quick on $internet proto tcp from any port 113 to any flags R/RSFUP queue(std) ## Process tagged for VOIP packets and everything else gets pushed into the std queue pass out quick on $internet tagged VOIP keep state queue(voip) ### TESTING - Try to force traffic into the queue pass out quick on $internet proto tcp from any to any queue(std, tcpack) pass out quick on $internet tagged STD keep state queue(std, tcpack) pass out quick on $internet from any to any keep state queue(std tcpack) ## ------------------------------------------------------------------------- ## LAN (fxp3) ## ------------------------------------------------------------------------- ## Tag SIP and IAX2 with VOIP tag for later queueing pass in quick on $lan proto udp from $lan_net to any port $voip tag VOIP keep state # TESTING - Try to tag packets to force into std queue. Probably does not work with NAT pass in quick on $lan from $lan_net to any tag STD keep state ## Trust the lan (for now) pass in quick on $lan from {! $dmz1_net ! $dmz2_net} to any keep state ## ------------------------------------------------------------------------- ## DMZ1 (fxp1) ## ------------------------------------------------------------------------- pass in quick on $dmz1 proto { tcp udp } from any to $dns_servers port domain keep state pass in quick on $dmz1 proto tcp from any to $dmz1_net port $sql keep state ## ------------------------------------------------------------------------- ## DMZ2 (fxp2) ## ------------------------------------------------------------------------- ## Skip logging on these block in quick on $dmz2 proto udp from $dmz2_net to 208.254.161.255 port 137 # Also see macro on incoming for fxp0 pass in quick on $dmz2 proto {tcp udp} from $dns_servers to any port domain keep state pass in quick on $dmz2 proto { tcp udp } from $lan_net to $dmz2_net port 161 keep state pass in quick on $dmz2 proto { tcp udp } from $dmz2_net to $lan_net port {135 137} keep state pass in quick on $dmz2 proto icmp from any to $dmz2_net keep state # Quickly pass DNS pass out quick on $dmz2 proto { tcp udp } from any to any port 53 keep state # Only filter incoming to make things easier pass out keep state Here's some output from pfctl -vvvsq with the T1 line maxed out with clients downloading files from the lan. queue root_fxp0 bandwidth 1.50Mb priority 0 {std, voip, tcpack} [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ] [ qlength: 0/ 50 ] [ measured: 0.0 packets/s, 0 b/s ] queue std bandwidth 495Kb hfsc( default ) [ pkts: 626828 bytes: 102182679 dropped pkts: 0 bytes: 0 ] [ qlength: 0/ 50 ] [ measured: 101.9 packets/s, 66.23Kb/s ] queue voip bandwidth 510Kb priority 7 hfsc( realtime 900Kb ) [ pkts: 1580 bytes: 170382 dropped pkts: 0 bytes: 0 ] [ qlength: 0/ 50 ] [ measured: 0.2 packets/s, 433.60 b/s ] queue tcpack bandwidth 495Kb priority 6 hfsc( red realtime 300Kb ) [ pkts: 66393 bytes: 4435542 dropped pkts: 0 bytes: 0 ] [ qlength: 0/ 50 ] [ measured: 0.1 packets/s, 28.80 b/s ] The trafffic for TCP acks seems to be OK, while the bandwidth for the std queue seems WAY low. From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 17:28:17 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE94916A419 for ; Tue, 18 Dec 2007 17:28:16 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id 6CADF13C4D3 for ; Tue, 18 Dec 2007 17:28:16 +0000 (UTC) (envelope-from max@love2party.net) Received: from amd64.laiers.local (dslb-088-066-047-246.pools.arcor-ip.net [88.66.47.246]) by mrelayeu.kundenserver.de (node=mrelayeu8) with ESMTP (Nemesis) id 0ML31I-1J4gEo2dmI-0003Lc; Tue, 18 Dec 2007 18:28:15 +0100 From: Max Laier Organization: FreeBSD To: freebsd-pf@freebsd.org Date: Tue, 18 Dec 2007 18:27:57 +0100 User-Agent: KMail/1.9.7 References: <200712180934.58755.silver.salonen@gmail.com> <14397207.post@talk.nabble.com> In-Reply-To: <14397207.post@talk.nabble.com> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1995150.QsF0deyv5T"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200712181828.04998.max@love2party.net> X-Provags-ID: V01U2FsdGVkX19uopLj4yUVWTEUIobSISenSPKuK8ePhW1jsN0 LkxzBOC1UOfbtsmWDalgt8e/7ul8SzmubNpEaLlumxwBEifU9L 1xoV0ZVeQduTec7eTF6eh9neWlm1JlwZiDYANdVQrQ= Cc: Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 17:28:17 -0000 --nextPart1995150.QsF0deyv5T Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Tuesday 18 December 2007, Atrox wrote: > Atrox wrote: > > Hello! > > > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN > > LAN-to-LAN > > and the problem is that a few times per hour connection drops between > > computers from one LAN to another. At first I blamed OpenVPN, then I > > blamed > > bridge, but now I've realized that the problem is in PF. > > So I've tried increasing TCP-timeouts and setting optimization > > to "aggressive", but well, it's still the same. > > > > I monitor connections by sending TCP packets once per second to some > > other host and wait for reply. I use Nagios-plugins' check_tcp for > > that. The script > > looks like: > > =3D=3D=3D=3D=3D > > while [ 1 ]; do > > pfctl -si |grep mismatch > > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > > pfctl -si |grep mismatch > > sleep 1 > > done > > =3D=3D=3D=3D=3D > > > > So if I let this script into action, I see that in 2-3 minutes, > > check_tcp gets "Operation not permitted" error and just in this > > moment packet-mismatch > > counter is increased by one (on machine with lesser traffic, I get > > the timeout > > in a few hours). That's on both 6.3-PRERELEASE as well as on > > 6.2-RELEASE. I've > > tried connections: > > * along WAN to IPFW-enabled machines > > * along WAN to PF-enabled machines > > * along LAN to PF-enabled machines > > * along LAN to Windows machines > > * along VPN to PF-enabled machines > > * along VPN to Windows machines > > > > Sometimes I get just some connection timeout: CRITICAL - Socket > > timeout after > > 2 seconds (I don't know what could cause that). > > > > I can see this behaviour in about every FreeBSD/PF machine I have. > > > > The basic PF-configuration looks like: > > =3D=3D=3D=3D=3D > > set block-policy return > > set loginterface $ext_if > > set timeout tcp.closed 15 > > set optimization aggressive > > scrub in all no-df > > > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > block log all > > pass quick on lo0 all > > pass out all modulate state > > pass out proto tcp all flags S/SA modulate state > > pass on $int_if all modulate state > > pass on $int_if proto tcp all flags S/SA modulate state > > pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services > > flags > > S/SA modulate state > > =3D=3D=3D=3D=3D > > > > Is PF buggy or have I misconfigured smth? > > Today I installed an OpenBSD-4.2 box just to see whether PF does the > same thing there. And yes, it does. > pf.conf: > =3D=3D=3D=3D=3D > ext_if =3D rl0 > set block-policy return > set loginterface $ext_if > scrub in all no-df > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > pass all modulate state > pass quick on lo0 all > =3D=3D=3D=3D=3D > > I check TCP without "sleep 1" now, and I do it to FreeBSD box without > firewall. state-mismatch gets increased by one, and I get either "No > route to host" or "Socket timeout after 2 seconds". > > Am I still misconfiguring the thing? No idea, you don't give nearly enough data for us to figure out what your=20 setup looks like. Regular tcp state mismatch usually hints that pf isn't=20 seeing all packets of the conversation. This can be caused by triangular=20 routing, load balanceing or if_bridge (which is difficult to get right in=20 some scenarios). You should figure out the exact path your tcp packets are taking (back and= =20 forth) and make sure pf sees all of them. Enabling additional pf logging=20 (pfctl -xm) helps to figure out what kind of mismatch is happening. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart1995150.QsF0deyv5T Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHaAMkXyyEoT62BG0RAgT0AJ4qMX4Jxf3ZciJqtuWa0JAl+SyQYgCeINVX ah60YsBRsxQNfPukoMyCiFA= =/03A -----END PGP SIGNATURE----- --nextPart1995150.QsF0deyv5T-- From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 17:29:18 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC69C16A506 for ; Tue, 18 Dec 2007 17:29:18 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id 5E69413C4CC for ; Tue, 18 Dec 2007 17:29:18 +0000 (UTC) (envelope-from max@love2party.net) Received: from amd64.laiers.local (dslb-088-066-047-246.pools.arcor-ip.net [88.66.47.246]) by mrelayeu.kundenserver.de (node=mrelayeu4) with ESMTP (Nemesis) id 0ML21M-1J4gFl0v9h-00033g; Tue, 18 Dec 2007 18:29:16 +0100 From: Max Laier Organization: FreeBSD To: freebsd-pf@freebsd.org Date: Tue, 18 Dec 2007 18:29:09 +0100 User-Agent: KMail/1.9.7 References: <20071213171525.GC44706@zeus.kimaker.com> <20071213175328.GE44706@zeus.kimaker.com> In-Reply-To: <20071213175328.GE44706@zeus.kimaker.com> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart4161526.hIHNKZR2gt"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200712181829.11076.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+Fs8bdfGXuo8b7qQFRvob2ECLyMuSXEI+XqaX GPFNgusATwySKmlUuX9bw6NBkPLnYK7kBx74c3lF9ALns1YPAX vQiKVeTOS3QU1rAW0j6/qLqs+cvkAqcMM2p0yqjCcM= Cc: Subject: Re: PF not routing traffic to IPv4 interface when IPv6 is enabled X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 17:29:19 -0000 --nextPart4161526.hIHNKZR2gt Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Thursday 13 December 2007, Ryan Phillips wrote: > Ryan Phillips said: > > Hi all, > > > > I just noticed that on my Freebsd server the PF firewall is not > > filtering packets for ipv4. Recently another admin enabled IPv6 > > within rc.conf, but we never assigned an IP6 IP. The interface has > > one IPv4 IP and many aliases for other IP4 IPs we own. > > Clarification: The IPv6 interface does have a ipv6 link local address. You don't give enough information to debug this problem. At very least=20 your ruleset (-vvgsr) and ifconfig should be supplied. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart4161526.hIHNKZR2gt Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHaANnXyyEoT62BG0RAr5QAJ0eUQ613BNpXVGPrM1xuTADCLnXngCeOATx LuLjCeqF7Xq4rS0y7U6GweQ= =qH6Z -----END PGP SIGNATURE----- --nextPart4161526.hIHNKZR2gt-- From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 20:09:58 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76B2316A41B for ; Tue, 18 Dec 2007 20:09:58 +0000 (UTC) (envelope-from kian.mohageri@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.181]) by mx1.freebsd.org (Postfix) with ESMTP id 213DA13C458 for ; Tue, 18 Dec 2007 20:09:57 +0000 (UTC) (envelope-from kian.mohageri@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so4501963pyb.3 for ; Tue, 18 Dec 2007 12:09:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=9qI8iAoVKWkZiAHDgxGIOI7od+3CgyaBzzyoA+kGMTw=; b=Rr5x6gQ2NaPZHJ7LRB98khZlJr/xgd5IEtmEAIIRPSLnwvuiC90RVmVr1AyOvc7DB0HI4GssFw5hmRSWsRyMSB7Q0vyBW/2m0gU8ByGuQGf+cm+QtT97X782CTE7RqZXyPrMJJ3Pgx+yuJdP7gqwEru8r5x7YYvs/zCb81l/drk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=cLS/QjK9WwhrNCS478SOpaZ4hIso0AFWwHA6Z4T1058Crk7RRzMOqZQZMspCIqM4P8p6wkPQEnrIAFRcl+UqNpcCZOpiC0LnDYIJqzcCrVPojCoaQcARCITHpp+pO3sVAvwNndAX6zWcb61fOQbE9zVnWywqnMx4MMHdd4KcrGs= Received: by 10.65.213.4 with SMTP id p4mr7695587qbq.53.1198007062070; Tue, 18 Dec 2007 11:44:22 -0800 (PST) Received: by 10.65.116.4 with HTTP; Tue, 18 Dec 2007 11:44:21 -0800 (PST) Message-ID: Date: Tue, 18 Dec 2007 11:44:21 -0800 From: "Kian Mohageri" To: "Silver Salonen" In-Reply-To: <200712180934.58755.silver.salonen@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200712180934.58755.silver.salonen@gmail.com> Cc: freebsd-pf@freebsd.org Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 20:09:58 -0000 On Dec 17, 2007 11:34 PM, Silver Salonen wrote: > Hello! > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN LAN-to-LAN > and the problem is that a few times per hour connection drops between > computers from one LAN to another. At first I blamed OpenVPN, then I blamed > bridge, but now I've realized that the problem is in PF. > So I've tried increasing TCP-timeouts and setting optimization > to "aggressive", but well, it's still the same. > > I monitor connections by sending TCP packets once per second to some other > host and wait for reply. I use Nagios-plugins' check_tcp for that. The script > looks like: > ===== > while [ 1 ]; do > pfctl -si |grep mismatch > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > pfctl -si |grep mismatch > sleep 1 > done > ===== > My guess is that you're re-using a source port and are mismatching an existing state on the source or destination host (or something in between) because the state hasn't expired before the new connection attempt takes place. Can't be sure though... -Kian From owner-freebsd-pf@FreeBSD.ORG Tue Dec 18 20:35:33 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4A4216A41A for ; Tue, 18 Dec 2007 20:35:33 +0000 (UTC) (envelope-from thompsa@FreeBSD.org) Received: from heff.fud.org.nz (203-109-251-39.static.bliink.ihug.co.nz [203.109.251.39]) by mx1.freebsd.org (Postfix) with ESMTP id 5C76C13C478 for ; Tue, 18 Dec 2007 20:35:33 +0000 (UTC) (envelope-from thompsa@FreeBSD.org) Received: by heff.fud.org.nz (Postfix, from userid 1001) id 49FA27E89; Wed, 19 Dec 2007 09:21:53 +1300 (NZDT) Date: Wed, 19 Dec 2007 09:21:53 +1300 From: Andrew Thompson To: freebsd-pf@freebsd.org Message-ID: <20071218202153.GF85906@heff.fud.org.nz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.16 (2007-06-09) Subject: pf eratta X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2007 20:35:33 -0000 Hi, I had someone ask if we are affected by eratta 004 here, http://openbsd.org/errata42.html Our pf is at version 4.1 so this may have been introduced afterwards, anyone know for sure? cheers, Andrew From owner-freebsd-pf@FreeBSD.ORG Wed Dec 19 07:11:58 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2007D16A417 for ; Wed, 19 Dec 2007 07:11:58 +0000 (UTC) (envelope-from silver.salonen@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 9960613C455 for ; Wed, 19 Dec 2007 07:11:57 +0000 (UTC) (envelope-from silver.salonen@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so535514fgg.35 for ; Tue, 18 Dec 2007 23:11:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:message-id; bh=KohI1Z+RYkIr2QKL4XbROSBr4ujeYI8QRJxMN14ZrbM=; b=Dirkw1eoa8U3q77SwRDAHNAbBMfoK36Lqk9MNpLD2jUYm4DKq2OU0FAj9AAWf3eYoWuPAzza3f3dM7REB6GD3vXj+pWodjRrqO7P1ObrRERjoXl1hrd3aFpmKoLeyb/wVKkv4Ci6Kug+WdYHIIS1b5FpAdL4J9ZsVjQDfPT766c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:message-id; b=Mx8hYfsD7ll7/oRgqa3NL+PhjBs2VVqx8+Z6+9+t/c45sGvs9ge/OeJf4M1e6a+gfAQp/yZ+1emVpVeH1orvsYUh+wRFZm8USq2fVXEDO6GeZGWG3TQ3SoErDuAGKJp/T9hTLTzU7R5/0au9BH9StQ54xKIz/wimvTAOH498gIg= Received: by 10.86.96.18 with SMTP id t18mr8597144fgb.13.1198048316419; Tue, 18 Dec 2007 23:11:56 -0800 (PST) Received: from ?192.168.8.99? ( [195.50.198.178]) by mx.google.com with ESMTPS id j12sm20568057fkf.2007.12.18.23.11.54 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 18 Dec 2007 23:11:55 -0800 (PST) From: Silver Salonen To: freebsd-pf@freebsd.org Date: Wed, 19 Dec 2007 09:11:45 +0200 User-Agent: KMail/1.9.7 References: <200712180934.58755.silver.salonen@gmail.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200712190911.46211.silver.salonen@gmail.com> Cc: Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2007 07:11:58 -0000 On Tuesday 18 December 2007 21:44, Kian Mohageri wrote: > On Dec 17, 2007 11:34 PM, Silver Salonen wrote: > > Hello! > > > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN LAN-to-LAN > > and the problem is that a few times per hour connection drops between > > computers from one LAN to another. At first I blamed OpenVPN, then I blamed > > bridge, but now I've realized that the problem is in PF. > > So I've tried increasing TCP-timeouts and setting optimization > > to "aggressive", but well, it's still the same. > > > > I monitor connections by sending TCP packets once per second to some other > > host and wait for reply. I use Nagios-plugins' check_tcp for that. The script > > looks like: > > ===== > > while [ 1 ]; do > > pfctl -si |grep mismatch > > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > > pfctl -si |grep mismatch > > sleep 1 > > done > > ===== > > > > My guess is that you're re-using a source port and are mismatching an > existing state on the source or destination host (or something in > between) because the state hasn't expired before the new connection > attempt takes place. > > Can't be sure though... > > -Kian Yup, googling a bit about openbsd, pf and "no route to host" turned up that it's "the port reuse issue". Although FreeBSD is supposed to be protected against it (http://www.freebsd.org/releases/4.11R/relnotes-i386.html#NET-PROTO), it seems not to be. So question is that how can I avoid the issue without removing keeping states from my rules. Actually I did the latter yesterday and connections started dropping as my default rule is to block everything. So now I'm keeping states on only outgoing connections and it's better this way, but not perfect though. -- Silver From owner-freebsd-pf@FreeBSD.ORG Wed Dec 19 07:18:23 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41BED16A418 for ; Wed, 19 Dec 2007 07:18:23 +0000 (UTC) (envelope-from silver.salonen@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id B269E13C45B for ; Wed, 19 Dec 2007 07:18:22 +0000 (UTC) (envelope-from silver.salonen@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so536085fgg.35 for ; Tue, 18 Dec 2007 23:18:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:message-id; bh=xeQSEisIClGw58E6U5zzz2QyTyzFeXHOBRAwDIxFjWc=; b=FX3O08FNO17WdwWM5XMAyXP8DZEYa2T6xp6RNTjYmYnyiR8m4R0dD9e+Bymz3zPumAUD297XvTVXNDlP4v6BBnJ1Zwe59Td9LGRqipVeYeH2WfTXaUYNLc8uVXFG9+/TTkogu56RnJzt5LBY6tol5400ycBxkxcE9NmG42Hyjt4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:message-id; b=i1qCiFswzD5Q597AiGSma/Z5QB/DeziLuWzV/Mp2dKGqFV4Pa3NldqIqx/hMMrtSCViPOsiAg8HFAx8o9phhBcywCdgML2/AFc++ey+lG5clQG4di1MIjAhF3cGU77+NndswY+R6TgY/aX8MA+NWZ5xwfOfmOoaQHt6h4uqSXo8= Received: by 10.86.73.17 with SMTP id v17mr8574198fga.74.1198048701545; Tue, 18 Dec 2007 23:18:21 -0800 (PST) Received: from ?192.168.8.99? ( [195.50.198.178]) by mx.google.com with ESMTPS id e32sm2426605fke.2007.12.18.23.18.19 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 18 Dec 2007 23:18:20 -0800 (PST) From: Silver Salonen To: freebsd-pf@freebsd.org Date: Wed, 19 Dec 2007 09:18:15 +0200 User-Agent: KMail/1.9.7 References: <200712180934.58755.silver.salonen@gmail.com> <14397207.post@talk.nabble.com> <200712181828.04998.max@love2party.net> In-Reply-To: <200712181828.04998.max@love2party.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200712190918.16730.silver.salonen@gmail.com> Cc: Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2007 07:18:23 -0000 On Tuesday 18 December 2007 19:27, Max Laier wrote: > On Tuesday 18 December 2007, Atrox wrote: > > Atrox wrote: > > > Hello! > > > > > > I have some FreeBSD-boxes (2x6.3-PRERELEASE (installed on 08.Dec), > > > 1x6.2-RELEASE) with PF configured. They are connected with OpenVPN > > > LAN-to-LAN > > > and the problem is that a few times per hour connection drops between > > > computers from one LAN to another. At first I blamed OpenVPN, then I > > > blamed > > > bridge, but now I've realized that the problem is in PF. > > > So I've tried increasing TCP-timeouts and setting optimization > > > to "aggressive", but well, it's still the same. > > > > > > I monitor connections by sending TCP packets once per second to some > > > other host and wait for reply. I use Nagios-plugins' check_tcp for > > > that. The script > > > looks like: > > > ===== > > > while [ 1 ]; do > > > pfctl -si |grep mismatch > > > /usr/local/libexec/nagios/check_tcp -H $host -p $port -t 2 > > > pfctl -si |grep mismatch > > > sleep 1 > > > done > > > ===== > > > > > > So if I let this script into action, I see that in 2-3 minutes, > > > check_tcp gets "Operation not permitted" error and just in this > > > moment packet-mismatch > > > counter is increased by one (on machine with lesser traffic, I get > > > the timeout > > > in a few hours). That's on both 6.3-PRERELEASE as well as on > > > 6.2-RELEASE. I've > > > tried connections: > > > * along WAN to IPFW-enabled machines > > > * along WAN to PF-enabled machines > > > * along LAN to PF-enabled machines > > > * along LAN to Windows machines > > > * along VPN to PF-enabled machines > > > * along VPN to Windows machines > > > > > > Sometimes I get just some connection timeout: CRITICAL - Socket > > > timeout after > > > 2 seconds (I don't know what could cause that). > > > > > > I can see this behaviour in about every FreeBSD/PF machine I have. > > > > > > The basic PF-configuration looks like: > > > ===== > > > set block-policy return > > > set loginterface $ext_if > > > set timeout tcp.closed 15 > > > set optimization aggressive > > > scrub in all no-df > > > > > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > > block log all > > > pass quick on lo0 all > > > pass out all modulate state > > > pass out proto tcp all flags S/SA modulate state > > > pass on $int_if all modulate state > > > pass on $int_if proto tcp all flags S/SA modulate state > > > pass in on $ext_if proto tcp from any to ($ext_if) port $tcp_services > > > flags > > > S/SA modulate state > > > ===== > > > > > > Is PF buggy or have I misconfigured smth? > > > > Today I installed an OpenBSD-4.2 box just to see whether PF does the > > same thing there. And yes, it does. > > pf.conf: > > ===== > > ext_if = rl0 > > set block-policy return > > set loginterface $ext_if > > scrub in all no-df > > block drop out quick on $ext_if from ($ext_if) to 0.0.0.0 > > pass all modulate state > > pass quick on lo0 all > > ===== > > > > I check TCP without "sleep 1" now, and I do it to FreeBSD box without > > firewall. state-mismatch gets increased by one, and I get either "No > > route to host" or "Socket timeout after 2 seconds". > > > > Am I still misconfiguring the thing? > > No idea, you don't give nearly enough data for us to figure out what your > setup looks like. Well, as for OpenBSD, I installed the default system and changed only the pf.conf, and it sits in LAN (.../24). It's quite plain actually. > Regular tcp state mismatch usually hints that pf isn't > seeing all packets of the conversation. This can be caused by triangular > routing, load balanceing or if_bridge (which is difficult to get right in > some scenarios). Does using if_bridge matter when I test connections on non-bridged interfaces? I see state-mismatch while testing TCP-connection on servers' external interfaces (along WAN) and these are not in bridge. -- Silver From owner-freebsd-pf@FreeBSD.ORG Wed Dec 19 09:55:47 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46A2D16A420 for ; Wed, 19 Dec 2007 09:55:47 +0000 (UTC) (envelope-from horcicka@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.186]) by mx1.freebsd.org (Postfix) with ESMTP id CC9A713C458 for ; Wed, 19 Dec 2007 09:55:46 +0000 (UTC) (envelope-from horcicka@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so2901271rvb.43 for ; Wed, 19 Dec 2007 01:55:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=SejR5KXIy1b8WBGKqzyZVlPaE7EcmCR1r1tXIgqMzmg=; b=CE5rmAohj/bzBvbL4KQ0lREBAN74ZbhFiXG68Z8seh2CQCUQ4QwQNAiqFX7eFX0i5DrenfvwwBswOpBYHfwk3xpug4xyp4xoS8ZPCKtw3rAWnVv9r1qei3qKCM7qP41uIcbqvZ3jlxnQYnBxab4jKrFpExKdha0xkerQ9gUnNe8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=e0xhc9l6DSoDLWCNX0OV1ltysxePFD2dTFU9IJvXsnwRQhRsBIEM/Qks74iXNkqdtEDKesPChJnMEgkaJHjbyzfjpZuVhNRykkqRVz/z9UmrC2zzQ3aZcRpD/NWeWxrh4ldjc9IBfsFiyIDpzCJqi530PwWKUYnlHd0UBAdH1JI= Received: by 10.141.42.10 with SMTP id u10mr4022841rvj.256.1198056654844; Wed, 19 Dec 2007 01:30:54 -0800 (PST) Received: by 10.141.197.3 with HTTP; Wed, 19 Dec 2007 01:30:54 -0800 (PST) Message-ID: <437bc1590712190130l31bdc573jc95f8c385962bfd2@mail.gmail.com> Date: Wed, 19 Dec 2007 10:30:54 +0100 From: "Martin Horcicka" Sender: horcicka@gmail.com To: "Kian Mohageri" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200712180934.58755.silver.salonen@gmail.com> X-Google-Sender-Auth: 72499b35fd2f0ea7 Cc: freebsd-pf@freebsd.org Subject: Re: occasional "Operation not permitted" on state-mismatch X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2007 09:55:47 -0000 On Dec 18, 2007 8:44 PM, Kian Mohageri wrote: > My guess is that you're re-using a source port and are mismatching an > existing state on the source or destination host (or something in > between) because the state hasn't expired before the new connection > attempt takes place. My guess is the same and this problem can be usually worked around by setting net.inet.ip.portrange.randomized to 0 on the machine where the connection is originated. It does not fix the bug in the FreeBSD's TCP stack but it helps unless there is a very high outgoing connection rate. Martin From owner-freebsd-pf@FreeBSD.ORG Wed Dec 19 15:16:56 2007 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC66016A475 for ; Wed, 19 Dec 2007 15:16:56 +0000 (UTC) (envelope-from peter@bsdly.net) Received: from skapet.datadok.no (cl-426.sto-01.se.sixxs.net [IPv6:2001:16d8:ff00:1a9::2]) by mx1.freebsd.org (Postfix) with ESMTP id A08AE13C4EB for ; Wed, 19 Dec 2007 15:16:56 +0000 (UTC) (envelope-from peter@bsdly.net) Received: from thingy.datadok.no ([194.54.103.97] helo=thingy.datadok.no.bsdly.net ident=peter) by skapet.datadok.no with esmtp (Exim 4.66) (envelope-from ) id 1J50fH-0001fq-5P for freebsd-pf@freebsd.org; Wed, 19 Dec 2007 16:16:55 +0100 To: freebsd-pf@freebsd.org From: peter@bsdly.net (Peter N. M. Hansteen) Date: Wed, 19 Dec 2007 16:16:51 +0100 Message-ID: <877ijasoik.fsf@thingy.datadok.no> User-Agent: Gnus/5.1007 (Gnus v5.10.7) XEmacs/21.4.19 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: "The Book of PF" exists, physical copies documented X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2007 15:16:57 -0000 Dear friends, I have just taken delivery of my box of /The Book of PF/[1] author's copies, and I'm finding I'm a bit at a loss for words when it comes to describing the feeling. The thing exists. And it feels great to finally see the thing for real. (OK, I cheated a bit and had five copies printed locally for OpenCON[2], but these are the real ones, slightly different binding) We didn't manage to release the book the same date as OpenBSD 4.2 (yes, that was the original plan), but the thing has been written, printed and should be on its way to all those who preordred as well as to better bookshops everywhere. I'm not directly involved in distribution and can not make any guarantees about when you'll get yours (a slightly more experienced author has written an explanation of that[3] - only this one was printed in Ann Arbor, Michigan, USA), but even if it's late for the holidays I hope you'll enjoy your copy and find the book useful. Now of course the usual factors interfered even with the author copies. There is a thing about courier companies and 18th century buildings. Even faced with a relatively small one such as the one I live in, DHL managed to make two attempts at delivery at the disused entrance clearly marked 'deliveries at the other door please', so the delivery finally happened at my office - also located in a 18th century building, but at least one where the DHL people have been before. But it finally arrived and life's good :) One interesting factoid (fsvo) is that this happened within hours of the twenty-five thousandth unique visitor (since EuroBSDCon 2006 that is) hitting the book's predecessor, the online PF tutorial[4]. So happy hacking holidays everyone, [1] http://nostarch.com/pf.htm, also see http://www.bsdly.net/~peter/freshbooks.jpg or (slightly faster) http://home.nuug.no/~peter/freshbooks.jpg [2] http://www.undeadly.org/cgi?action=article&sid=20071207191612 - that's me completely obscured at the back to the right in the third picture (ok, I'm in a few others ;)) [3] http://marc.info/?l=openbsd-misc&m=105723966516199&w=2 [4] http://home.nuug.no/~peter/pf/ -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ "Remember to set the evil bit on all malicious network traffic" delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.