From owner-freebsd-bugs@FreeBSD.ORG Wed Jun 30 12:20:08 2010 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 580421065674 for ; Wed, 30 Jun 2010 12:20:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1BE7F8FC22 for ; Wed, 30 Jun 2010 12:20:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o5UCK7HS037585 for ; Wed, 30 Jun 2010 12:20:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o5UCK7TO037583; Wed, 30 Jun 2010 12:20:07 GMT (envelope-from gnats) Resent-Date: Wed, 30 Jun 2010 12:20:07 GMT Resent-Message-Id: <201006301220.o5UCK7TO037583@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Alexey Guskov Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21F51106566C for ; Wed, 30 Jun 2010 12:10:14 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id 107F48FC14 for ; Wed, 30 Jun 2010 12:10:14 +0000 (UTC) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o5UCADwR042398 for ; Wed, 30 Jun 2010 12:10:13 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id o5UCADUN042375; Wed, 30 Jun 2010 12:10:13 GMT (envelope-from nobody) Message-Id: <201006301210.o5UCADUN042375@www.freebsd.org> Date: Wed, 30 Jun 2010 12:10:13 GMT From: Alexey Guskov To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/148260: pf rdr incompatible with dummynet X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 12:20:08 -0000 >Number: 148260 >Category: kern >Synopsis: pf rdr incompatible with dummynet >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Jun 30 12:20:07 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Alexey Guskov >Release: 7.2-RELEASE >Organization: Areal Company >Environment: FreeBSD test.local 7.2-RELEASE FreeBSD 7.2-RELEASE #1: Tue Jun 29 09:04:57 UTC 2010 root@tamaez.office1.a-real.ru:/usr/src/sys/i386/compile/GATE i386 >Description: The problem is the same as described here: http://lists.freebsd.org/pipermail/freebsd-net/2006-July/011076.html I'm using two firewalls: ipfw for dummynet traffic shaping and pf for rules, forwards etc. When a dummynet pipe is used and rdr rule exists pf (i.e. for transparent proxy) tcp connections doesn't get established. For example: we have a pipe for traffic going to user, and rdr rule for trasparent proxy. Ipfw processes incoming packets before pf. Here's how traffic goes thru kernel: SYN packet: [network interface] -> ... -> pfil_run_hooks() -> ipfw_check_in() -> pf_check_in() /the packet is being redirected and pf state entry is created/ -> ... -> [squid] The packet is successfully delivered. ACK packet: [squid] -> ... -> ip_output() -> ... -> pfil_run_hooks() -> pf_check_out() /the packet is being unredirected using state table entry/ -> ipfw_check_out() -> ip_dn_io_ptr() NB: ipfw stops processing the packet after it bin injected into dummynet. Heres what happens next: ip_dn_ip_ptr() -> /some pipe vodoo magic/ -> dummynet_send() -> ip_output() TADA! After coming out from pipe the packet is being re-injected into main packet-processing routine! Let's see what's gonna happen: ip_output() -> ... -> pfil_run_hooks() -> pf_check_out() /pf cannot find entry in state table because the packet has already been unredirected and violently kills it/ -> OH SHI-- So, tcp connection never gets established. >How-To-Repeat: pf.conf -- rdr on le1 proto tcp from { 192.168.1.0/24 } to any port { 80 } -> (le1) port 3128 -- ipfw.conf -- ipfw pipe 1 config bw 16KByte/s ipfw 100 add pipe 1 all from any to 192.168.1.101 -- make ipfw process incoming packets before pf --- # ipfw disable firewall # ipfw enable firewall -- >Fix: When ipfw send a packet to dummynet it tags it with PACKET_TAG_DUMMYNET and saves rule number that triggered this action. When the packet comes to ipfw for the second time, ipfw can see that it already has been checked and continues processing from the next rule. The problem is pf doesn't know that the packet already has been processed by him. So we'll teach him: --- sys/contrib/pf/net/pf_ioctl.c 2010-06-14 06:09:06.000000000 +0400 +++ nsys/contrib/pf/net/pf_ioctl.c 2010-06-30 14:20:55.000000000 +0400 @@ -3636,6 +3637,7 @@ */ struct ip *h = NULL; int chk; + struct m_tag *dn_tag; if ((*m)->m_pkthdr.len >= (int)sizeof(struct ip)) { /* if m_pkthdr.len is less than ip header, pf will handle. */ @@ -3643,7 +3645,13 @@ HTONS(h->ip_len); HTONS(h->ip_off); } - chk = pf_test(PF_IN, ifp, m, NULL, inp); + + dn_tag = m_tag_find(*m, PACKET_TAG_DUMMYNET, NULL); + if (dn_tag == NULL) + chk = pf_test(PF_IN, ifp, m, NULL, inp); + else + chk = PF_PASS; + if (chk && *m) { m_freem(*m); *m = NULL; @@ -3671,6 +3679,7 @@ */ struct ip *h = NULL; int chk; + struct m_tag *dn_tag; /* We need a proper CSUM befor we start (s. OpenBSD ip_output) */ if ((*m)->m_pkthdr.csum_flags & CSUM_DELAY_DATA) { @@ -3683,7 +3692,13 @@ HTONS(h->ip_len); HTONS(h->ip_off); } - chk = pf_test(PF_OUT, ifp, m, NULL, inp); + + dn_tag = m_tag_find(*m, PACKET_TAG_DUMMYNET, NULL); + if (dn_tag == NULL) + chk = pf_test(PF_OUT, ifp, m, NULL, inp); + else + chk = PF_PASS; + if (chk && *m) { m_freem(*m); *m = NULL; >Release-Note: >Audit-Trail: >Unformatted: