From owner-freebsd-pf@FreeBSD.ORG Tue Jul 29 09:25:28 2008 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EFA21065671 for ; Tue, 29 Jul 2008 09:25:28 +0000 (UTC) (envelope-from nejc@skoberne.net) Received: from delusion.skoberne.net (lk.84.20.249.154.dc.cable.static.lj-kabel.net [84.20.249.154]) by mx1.freebsd.org (Postfix) with ESMTP id 373CE8FC1C for ; Tue, 29 Jul 2008 09:25:26 +0000 (UTC) (envelope-from nejc@skoberne.net) Received: from localhost (localhost [127.0.0.1]) by delusion.skoberne.net (Postfix) with ESMTP id BAB97B941 for ; Tue, 29 Jul 2008 11:18:01 +0200 (CEST) Received: from delusion.skoberne.net ([127.0.0.1]) by localhost (delusion.skoberne.net [127.0.0.1]) (amavisd-maia, port 10024) with ESMTP id 65729-07 for ; Tue, 29 Jul 2008 11:17:58 +0200 (CEST) Received: from [192.168.0.7] (pisarna.iskreni.net [213.143.68.31]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: nejkopejko@skoberne.net) by delusion.skoberne.net (Postfix) with ESMTP id CFFA8B940 for ; Tue, 29 Jul 2008 11:17:58 +0200 (CEST) Message-ID: <488EE046.4010602@skoberne.net> Date: Tue, 29 Jul 2008 11:17:58 +0200 From: =?ISO-8859-2?Q?Nejc_=A9koberne?= User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: freebsd-pf@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard Subject: pf randomly blocks specific packets? X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2008 09:25:28 -0000 Hello, I have a FreeBSD 7.0 system with jails (and services in them). In one of the jails there is an Apache server, which also runs on the host system (and forwards traffic using mod_proxy to the jailed Apache). Everything works as expected, I only have problems with pf which seems to block certain packets randomly (not all of them). This is how my rc.conf on host system looks like (relevant parts): --------------------------------------------------------------------------------- defaultrouter="172.20.2.1" ifconfig_em0="inet 172.20.2.2 netmask 255.255.255.0" pf_enable="YES" pf_rules="/etc/pf.conf" pf_flags="" pflog_enable="YES" pflog_logfile="/var/log/pflog" pflog_flags="" apache22_enable="YES" cloned_interfaces="lo1" ifconfig_lo1="192.168.223.1" jail_enable="YES" jail_sysvipc_allow="YES" jail_list="mail" jail_mail_rootdir="/usr/jail/j/mail" jail_mail_hostname="mail" jail_mail_ip="192.168.223.10" jail_mail_interface="lo1 netmask 255.255.255.0" jail_mail_devfs_enable="YES" jail_mail_procfs_enable="YES" jail_mail_devfs_ruleset="devfsrules_jail" --------------------------------------------------------------------------------- This is how my pf.conf looks like: --------------------------------------------------------------------------------- int_Trust = "em0" int_Loop = "lo0" int_Jails = "lo1" int_jail_mail = "{" $int_Trust "}" addr_net_Private = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }" addr_net_Local = "{" $int_Trust:network ", 172.20.2.0/24, 192.168.0.0/16 }" addr_sysSvarun = "a.b.c.d" addr_jail_mail = "192.168.223.10" svc_TCP_HOST_Private = "{ ssh, iwebmin, itelnet }" svc_TCP_HOST_Public = "{ ssh, http, https, domain }" svc_UDP_HOST_Public = "{ domain }" svc_TCP_jail_mail = "{ smtp, smtps, pop3, pop3s, imap, imaps }" ICMPTypes = "echoreq" AllProtocols = "{ tcp, udp, ipv6, icmp, esp, ipencap, gre }" table persist file "/usr/local/etc/trusted.addresses" set loginterface $int_Trust scrub on $int_Trust all no-df random-id reassemble tcp nat on $int_Trust from $int_Jails:network to any -> $int_Trust rdr pass on $int_jail_mail proto tcp from any to $int_jail_mail port \ $svc_TCP_jail_mail -> $addr_jail_mail block log all pass in quick on $int_Trust from $addr_sysSvarun to any keep state pass quick on $int_Loop all pass quick on $int_Jails all pass quick inet proto icmp all icmp-type $ICMPTypes keep state pass in on $int_Trust from any to any keep state pass out on $int_Trust from any to $addr_net_Local keep state pass out on $int_Trust from $int_Trust to any keep state pass out on lo1 from 192.168.223.10 to 192.168.223.10 keep state --------------------------------------------------------------------------------- So as you can see there is a "pass quick on $int_Jails all" line. Which, as far as I understand, should do exactly that. But, when I do "tcpdump -n -r /var/log/pflog", I get these: 10:22:56.353027 IP 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:23:06.744057 IP 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:23:27.330096 IP 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:23:47.918481 IP 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:24:08.508126 IP 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:24:29.096918 IP 192.168.223.10.53777 > 192.168.223.10.80: R 1:1(0) ack 1 win 8960 10:33:12.341285 IP 192.168.223.10.51214 > 192.168.223.10.80: F 1457218003:1457218003(0) ack 1764186631 win 8960 10:33:12.637811 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:13.029827 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:13.609705 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:14.561443 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:16.256344 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:19.073348 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:24.504722 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:35.163039 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:33:56.274140 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:34:38.293842 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:35:20.310801 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:36:02.326561 IP 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 10:36:44.339793 IP 192.168.223.10.51214 > 192.168.223.10.80: R 1:1(0) ack 1 win 8960 or, if I pass "-e -ttt" parameters to tcpdump as well, these: rule 0/0(match): block out on lo1: 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.53777 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.53777 > 192.168.223.10.80: R 1:1(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 1457218003:1457218003(0) ack 1764186631 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: F 0:0(0) ack 1 win 8960 rule 0/0(match): block out on lo1: 192.168.223.10.51214 > 192.168.223.10.80: R 1:1(0) ack 1 win 8960 Which means, pf blocks these packets. Why would this be? The consequence of blocking these packets are, that I get this in the hosts's Apache server: [Fri Jul 25 09:57:10 2008] [error] (1)Operation not permitted: proxy: HTTP: attempt to connect to 192.168.223.10:80 (mail) failed [Fri Jul 25 09:57:10 2008] [error] ap_proxy_connect_backend disabling worker for (mail) [Fri Jul 25 09:57:10 2008] [error] proxy: HTTP: disabled connection for (mail) which disables connections to the jailed Apache for a while. Which is very annoying. However, this (that Apache get's blocked) doesn't happen always, I would say 10-20 times daily. Any ideas? Thanks, Nejc