From owner-freebsd-bugs@FreeBSD.ORG Sat Mar 15 19:00:06 2008 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E569C106566C for ; Sat, 15 Mar 2008 19:00:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DC5C98FC13 for ; Sat, 15 Mar 2008 19:00:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m2FJ05Y5062696 for ; Sat, 15 Mar 2008 19:00:05 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m2FJ05p6062695; Sat, 15 Mar 2008 19:00:05 GMT (envelope-from gnats) Resent-Date: Sat, 15 Mar 2008 19:00:05 GMT Resent-Message-Id: <200803151900.m2FJ05p6062695@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Alexander Zagrebin Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD285106566B for ; Sat, 15 Mar 2008 18:50:27 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id C4EB68FC15 for ; Sat, 15 Mar 2008 18:50:27 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m2FIl3Z8022530 for ; Sat, 15 Mar 2008 18:47:03 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.2/8.14.1/Submit) id m2FIl3U7022529; Sat, 15 Mar 2008 18:47:03 GMT (envelope-from nobody) Message-Id: <200803151847.m2FIl3U7022529@www.freebsd.org> Date: Sat, 15 Mar 2008 18:47:03 GMT From: Alexander Zagrebin To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/121743: ipfw in-kernel nat loses fragmented packets X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2008 19:00:06 -0000 >Number: 121743 >Category: kern >Synopsis: ipfw in-kernel nat loses fragmented packets >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Mar 15 19:00:05 UTC 2008 >Closed-Date: >Last-Modified: >Originator: Alexander Zagrebin >Release: 7.0-RELEASE >Organization: - >Environment: FreeBSD 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sat Mar 15 19:18:40 MSK 2008 alex@:/usr/src/sys/i386/compile/KERNEL i386 >Description: When trying to use ipfw in-kernel nat, I observed, that it loses fragmented packets (see section "How to repeat the problem"). The problem is in current ipfw code (sys/netinet/ip_fw2.c). After processing packet with LibAliasIn, ipfw analyses retcode and drops the packet if retcode != PKT_ALIAS_OK. But LibAliasIn, when processing the first fragment of the packet, returns PKT_ALIAS_FOUND_HEADER_FRAGMENT instead. This code is not error, therefore ipfw should consider this fact, when deciding to pass or drop a packet. (See the patch at "Fix to the problem if known") Also, libalias(3) (see section "FRAGMENT HANDLING") suggests the method of fragmented packets processing via LibAliasSaveFragment, LibAliasGetFragment, LibAliasFragmentIn, but neither ipfw nat code, nor user-space natd, doesn't use it now. It can be important, if packet's fragments will reordered during a way. >How-To-Repeat: My internal network is 192.168.1.0/24 External (public) network is 10.0.0.0/8 External interface (xl0) has address 10.255.255.2 Add to kernel config: options IPFIREWALL options IPFIREWALL_NAT options LIBALIAS Add to ipfw config (xl0 is the external interface): .. nat 1 config if xl0 log reset same_ports add 999 count log ip from any to any via xl0 add 1000 nat 1 ip from any to any via xl0 add 1001 count log ip from any to any via xl0 .. To log packets after nat: # sysctl net.inet.ip.fw.one_pass=0 Try to ping the external host (10.0.0.1 in my case) from an internal address, and use packets with a size greater than MTU: # ping -S 192.168.1.1 -s 2000 PING from 192.168.1.1: 2000 data bytes ^C --- 10.0.0.1 ping statistics --- 6 packets transmitted, 0 packets received, 100.0% packet loss So, ping fails. See /var/log/security (my comments are marked with >>>): .. >Fix: --- sys/netinet/ip_fw2.c.orig 2008-02-28 11:28:09.000000000 +0300 +++ sys/netinet/ip_fw2.c 2008-03-15 18:41:52.000000000 +0300 @@ -3568,7 +3568,8 @@ else retval = LibAliasOut(t->lib, c, MCLBYTES); - if (retval != PKT_ALIAS_OK) { + if (retval != PKT_ALIAS_OK && + retval != PKT_ALIAS_FOUND_HEADER_FRAGMENT) { /* XXX - should i add some logging? */ m_free(mcl); badnat: >Release-Note: >Audit-Trail: >Unformatted: >>> Our outbound ICMP echo request before NAT Mar 13 10:39:00 gw kernel: ipfw: 999 Count ICMP:8.0 192.168.1.1 10.0.0.1 out via xl0 >>> Our outbound ICMP echo request after NAT Mar 13 10:39:00 gw kernel: ipfw: 1001 Count ICMP:8.0 10.255.255.2 10.0.0.1 out via xl0 >>> ICMP echo reply (fragment 1) before NAT Mar 13 10:39:00 gw kernel: ipfw: 999 Count ICMP:0.0 10.0.0.1 10.255.255.2 in via xl0 (frag 20433:1480@0+) >>> ICMP echo reply (fragment 2) before NAT Mar 13 10:39:00 gw kernel: ipfw: 999 Count ICMP 10.0.0.1 10.255.255.2 in via xl0 (frag 20433:528@1480) >>> (!) ICMP echo reply (fragment 1) IS LOST! >>> ICMP echo reply (fragment 2) after NAT Mar 13 10:39:00 gw kernel: ipfw: 1001 Count ICMP 10.0.0.1 192.168.1.1 in via xl0 (frag 20433:528@1480) .. Mar 13 10:39:01 gw kernel: ipfw: 999 Count ICMP:8.0 192.168.1.1 10.0.0.1 out via xl0 Mar 13 10:39:01 gw kernel: ipfw: 1001 Count ICMP:8.0 10.255.255.2 10.0.0.1 out via xl0 Mar 13 10:39:01 gw kernel: ipfw: 999 Count ICMP:0.0 10.0.0.1 10.255.255.2 in via xl0 (frag 21967:1480@0+) Mar 13 10:39:01 gw kernel: ipfw: 999 Count ICMP 10.0.0.1 10.255.255.2 in via xl0 (frag 21967:528@1480) Mar 13 10:39:01 gw kernel: ipfw: 1001 Count ICMP 10.0.0.1 192.168.1.1 in via xl0 (frag 21967:528@1480) ..