From owner-freebsd-ipfw@FreeBSD.ORG Mon Apr 9 11:07:14 2012 Return-Path: Delivered-To: freebsd-ipfw@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F8EC10656AA for ; Mon, 9 Apr 2012 11:07:14 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 8881E8FC0C for ; Mon, 9 Apr 2012 11:07:14 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q39B7EYi039636 for ; Mon, 9 Apr 2012 11:07:14 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q39B7DIF039634 for freebsd-ipfw@FreeBSD.org; Mon, 9 Apr 2012 11:07:13 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 9 Apr 2012 11:07:13 GMT Message-Id: <201204091107.q39B7DIF039634@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-ipfw@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-ipfw@FreeBSD.org X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Apr 2012 11:07:14 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/166406 ipfw [ipfw] ipfw does not set ALTQ identifier for ipv6 traf o kern/165190 ipfw [ipfw] [lo] [patch] loopback interface is not marking f kern/163873 ipfw [ipfw] ipfw fwd does not work with 'via interface' in o kern/158066 ipfw [ipfw] ipfw + netgraph + multicast = multicast packets o kern/157796 ipfw [ipfw] IPFW in-kernel NAT nat loopback / Default Route o kern/157689 ipfw [ipfw] ipfw nat config does not accept nonexistent int o kern/156770 ipfw [ipfw] [dummynet] [patch]: performance improvement and f kern/155927 ipfw [ipfw] ipfw stops to check packets for compliance with o bin/153252 ipfw [ipfw][patch] ipfw lockdown system in subsequent call o kern/153161 ipfw [ipfw] does not support specifying rules with ICMP cod o kern/152113 ipfw [ipfw] page fault on 8.1-RELEASE caused by certain amo o kern/148827 ipfw [ipfw] divert broken with in-kernel ipfw o kern/148689 ipfw [ipfw] antispoof wrongly triggers on link local IPv6 a o kern/148430 ipfw [ipfw] IPFW schedule delete broken. o kern/148091 ipfw [ipfw] ipfw ipv6 handling broken. o kern/143973 ipfw [ipfw] [panic] ipfw forward option causes kernel reboo o kern/143621 ipfw [ipfw] [dummynet] [patch] dummynet and vnet use result o kern/137346 ipfw [ipfw] ipfw nat redirect_proto is broken o kern/137232 ipfw [ipfw] parser troubles o kern/135476 ipfw [ipfw] IPFW table breaks after adding a large number o o kern/129036 ipfw [ipfw] 'ipfw fwd' does not change outgoing interface n p kern/128260 ipfw [ipfw] [patch] ipfw_divert damages IPv6 packets o kern/127230 ipfw [ipfw] [patch] Feature request to add UID and/or GID l f kern/122963 ipfw [ipfw] tcpdump does not show packets redirected by 'ip s kern/121807 ipfw [request] TCP and UDP port_table in ipfw o kern/121122 ipfw [ipfw] [patch] add support to ToS IP PRECEDENCE fields o kern/116009 ipfw [ipfw] [patch] Ignore errors when loading ruleset from o bin/104921 ipfw [patch] ipfw(8) sometimes treats ipv6 input as ipv4 (a o kern/104682 ipfw [ipfw] [patch] Some minor language consistency fixes a o kern/103454 ipfw [ipfw] [patch] [request] add a facility to modify DF b o kern/103328 ipfw [ipfw] [request] sugestions about ipfw table o kern/102471 ipfw [ipfw] [patch] add tos and dscp support o kern/97951 ipfw [ipfw] [patch] ipfw does not tie interface details to o kern/95084 ipfw [ipfw] [regression] [patch] IPFW2 ignores "recv/xmit/v o kern/86957 ipfw [ipfw] [patch] ipfw mac logging o bin/83046 ipfw [ipfw] ipfw2 error: "setup" is allowed for icmp, but s o kern/82724 ipfw [ipfw] [patch] [request] Add setnexthop and defaultrou o bin/78785 ipfw [patch] ipfw(8) verbosity locks machine if /etc/rc.fir o kern/60719 ipfw [ipfw] Headerless fragments generate cryptic error mes s kern/55984 ipfw [ipfw] [patch] time based firewalling support for ipfw o kern/48172 ipfw [ipfw] [patch] ipfw does not log size and flags o kern/46159 ipfw [ipfw] [patch] [request] ipfw dynamic rules lifetime f a kern/26534 ipfw [ipfw] Add an option to ipfw to log gid/uid of who cau 43 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Thu Apr 12 12:53:20 2012 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2C0DA106566B for ; Thu, 12 Apr 2012 12:53:20 +0000 (UTC) (envelope-from dmk.sbor@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id DDE6E8FC16 for ; Thu, 12 Apr 2012 12:53:19 +0000 (UTC) Received: by yhgm50 with SMTP id m50so1217067yhg.13 for ; Thu, 12 Apr 2012 05:53:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5nWmTsc7ecIiYzW1PugI/7XVK2MirPPwS7Q5AznUTF8=; b=dxtCFStcWg21uofl4ZqIh7MMB0xXGY2B55PzF2TmX7RtVD/b6N95MUY4By3KTFSnbq hOeyH6V7t4O+S6lzjDQJK05XTY7hpQOtCsq3DZQBaCEkJCzRQ2xVk6fv+4ccJ7Z8Wqus 5HoRpcrrrjzK34yrv7HUvxDCEFhM3kywkxQisaMfqYkbh8uHH/2xAyCgALOt1RNcan6s fTzIQBk7nnd2DiV7L25yyN4t3nGCpE3Lzp/6zQ1Hpgmo9o4Sbwjx3rKn0FuS9boJeo6L NoDTp0TzNfqQJNta8NokaFi+x3DSQvGu1FCX5ULV1YRuGZvmXJvR5ljyaaS35nuTv5Ec nxpw== MIME-Version: 1.0 Received: by 10.236.175.164 with SMTP id z24mr1885469yhl.101.1334235198670; Thu, 12 Apr 2012 05:53:18 -0700 (PDT) Received: by 10.146.168.1 with HTTP; Thu, 12 Apr 2012 05:53:18 -0700 (PDT) Date: Thu, 12 Apr 2012 16:53:18 +0400 Message-ID: From: "Dmitry S. Kasterin" To: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Stateful IPFW - too many connections in FIN_WAIT_2 or LAST_ACK states X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Apr 2012 12:53:20 -0000 Hello! I have rather simple ipfw ruleset like this: 00001 allow all from any to any via lo0 00010 check-state 00101 allow tcp from me to any out setup keep-state 65533 deny log ip from any to any 65534 deny ip6 from any to any Actually, there are a few rules for upd, icmp and so on, but the main idea here is to allow only outgoing (tcp) connections and handle them using dynamic rules. The first thing I have found was enormously high counter value on "deny log ip from any to any" rule. For that moment my workstation was placed in a small private (and "clean") network, so this value was considered suspicious. Later I've discovered that many tcp connections have FIN_WAIT_2 or LAST_ACK state. In order to determine what's going on I've carried out some experiments (shown below). Briefly: from my point of view, ipfw sometimes handles the last phase of a connection improperly. I was unable to reliably reproduce this behaviour - sometimes it happens, but in the most cases not. But when it happens, it leads to "frozen" connections. Of course, this can be just a symptom of software misconfiguration or maybe my mistake. So I need an opinion from people with deep knoweledge of ipfw and network stack. Any information or comments are much appreciated! PS I'm running 9.0-STABLE with custom kernel. I. 1) Flush ipfw, reset counters, load fresh ruleset from file. 2) Run tcpdump on network interface (e.g. re0) and ipfw log interface (ipfw0) # tcpdump -i re0 -p -w # tcpdump -i ipfw0 -p -w 3) Disable proxy and make a query to a webserver, e.g. $ lynx www.freebsd.org 4) Check ipfw counter and connections # ipfw -deS show ; netstat -n -p tcp In the most cases this test gives "normal" result. But under some circumstances the result may be like this (w.x.y.z is an IP address of my workstation): # ipfw -deS show ; netstat -n -p tcp 00001 0 0 set 0 allow ip from any to any via lo0 ... 00010 0 0 set 0 check-state 00101 47 28622 set 0 allow tcp from me to any out setup keep-state ... 65533 6 312 set 0 deny log logamount 16 ip from any to any ## Dynamic rules (1): 00101 13 5620 (0s) STATE tcp w.x.y.z 26051 <-> 69.147.83.34 80 Active Internet connections Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 w.x.y.z.13414 69.147.83.34.80 LAST_ACK So, the page (www.freebsd.org) was loaded and packets were counted. But the dynamic rule entry is invalid - it has port 26051 instead of 13414. The analysis of dump files has shown: a) Dump from re0 has only one TCP stream: w.x.y.z:13414 <-> 69.147.83.34:80 b) Dump from ipfw0: N Time Source Destination Protocol Length Info 1 0.000000 69.147.83.34 w.x.y.z TCP 66 http > 13414 [ACK] Seq=1 Ack=1 Win=8325 Len=0 ... 2 0.557615 w.x.y.z 69.147.83.34 TCP 66 13414 > http [FIN, ACK] Seq=0 Ack=1 Win=1040 Len=0 ... 3 1.947625 w.x.y.z 69.147.83.34 TCP 66 13414 > http [FIN, ACK] Seq=0 Ack=1 Win=1040 Len=0 ... 4 4.527624 w.x.y.z 69.147.83.34 TCP 66 13414 > http [FIN, ACK] Seq=0 Ack=1 Win=1040 Len=0 ... 5 9.487633 w.x.y.z 69.147.83.34 TCP 66 13414 > http [FIN, ACK] Seq=0 Ack=1 Win=1040 Len=0 ... 6 19.207616 w.x.y.z 69.147.83.34 TCP 66 13414 > http [FIN, ACK] Seq=0 Ack=1 Win=1040 Len=0 ... It looks like network stack tries to finalize connection but the corresponding packets are dropped. II. 1) Slightly change the ruleset: 00001 allow ip from any to any via lo0 00010 check-state 00101 allow tcp from me to any out setup keep-state 11001 allow log tcp from any 80 to me in 11002 allow log tcp from me to any dst-port 80 out 65533 deny ip from any to any 65534 deny ip6 from any to any The idea is to see packets which were blocked in the previous test. 2) Flush ipfw, see I. 2) Run tcpdump, see I. 3) $ lynx www.freebsd.org 4) "ipfw -deS show" and "netstat -n -p tcp" # ipfw -deS show 00001 0 0 set 0 allow ip from any to any via lo0 00010 0 0 set 0 check-state 00101 33 22942 set 0 allow tcp from me to any out setup keep-state 11001 2 96 set 1 allow log logamount 16 tcp from any 80 to me in 11002 0 0 set 1 allow log logamount 16 tcp from me to any dst-port 80 out 65533 0 0 set 0 deny ip from any to any An there are no waiting connections. The dump from ipfw0 contains: No. Time Source Destination Protocol Length Info 1 0.000000 69.147.83.34 w.x.y.z TCP 66 http > 29470 [ACK] Seq=1 Ack=1 Win=8325 Len=0 ... III. 1) Change ruleset back to: 00001 allow ip from any to any via lo0 00010 check-state 00101 allow tcp from me to any out setup keep-state 65534 deny ip6 from any to any 65535 deny ip from any to any 2) Browse the Internet. 3) I've visited some pages and now netstat output looks like: # netstat -an -f inet | grep FIN_WAIT tcp4 0 0 w.x.y.z.47536 173.194.32.21.443 FIN_WAIT_2 tcp4 0 0 w.x.y.z.47533 173.194.32.21.443 FIN_WAIT_2 tcp4 0 0 w.x.y.z.47532 173.194.32.21.443 FIN_WAIT_2 tcp4 0 0 w.x.y.z.47531 173.194.32.21.443 FIN_WAIT_2 tcp4 0 0 w.x.y.z.24851 199.7.55.72.80 FIN_WAIT_2 tcp4 0 0 w.x.y.z.24731 74.125.224.79.80 FIN_WAIT_2 tcp4 0 0 w.x.y.z.11578 213.180.204.69.80 FIN_WAIT_2 tcp4 0 0 w.x.y.z.11577 213.180.204.143.80 FIN_WAIT_2