From owner-freebsd-ipfw@FreeBSD.ORG Wed Mar 23 13:37:49 2011 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF5861065670 for ; Wed, 23 Mar 2011 13:37:49 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 8CA9B8FC08 for ; Wed, 23 Mar 2011 13:37:49 +0000 (UTC) Received: by qyk35 with SMTP id 35so4500300qyk.13 for ; Wed, 23 Mar 2011 06:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=U2iL2P1d1iHwAVk4vDd5DieceX1evW8dUTjtTCArcRE=; b=w2sNy7R1nLjGyZPOOABxs3PP64StQ16vccHTv9EkJkSfUkKq4EIjkZLFB1WJe13tpc AEfjTUVg4UP2Z7sNaRHCuH+MUjrSHQrcF04N6T0m8JhL3si4t2a+ypjUpS+g3ZJxz6RC utMU9DE/jiARxPY9Pf82SXV8cYZg5+o1jIsW0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=QbQ/usQaB+ndlACh0UYA5Yf0IvrOfFjGE3lo6f2l4fB0zZ7TNeA2ZCX1qQ9nvdvm5s apcC2MFWTgMFUD60J9cwVNHXtbq71HqPE2rKq13nHVmfSx5bfPis0wlKfp7Xa6mvKg8d b39bGVfI4PfWs1MGzWgNouN8yA2IkOei9tEDA= MIME-Version: 1.0 Received: by 10.229.66.25 with SMTP id l25mr5700157qci.265.1300885411826; Wed, 23 Mar 2011 06:03:31 -0700 (PDT) Received: by 10.229.226.205 with HTTP; Wed, 23 Mar 2011 06:03:31 -0700 (PDT) Date: Wed, 23 Mar 2011 15:03:31 +0200 Message-ID: From: =?ISO-8859-1?Q?=D6zkan_KIRIK?= To: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: About IPFW in-kernel NAT nat loopback X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Mar 2011 13:37:49 -0000 Hi, I wanna share my experiences about ipfw in-kernel nat problems with you. When a packet injects into ipfw in-kernel nat more then once, operating systems behave abnormally. Let's inspect the ruleset below: ipfw table 12 add 10.0.14.1/32 33 ipfw table 13 add X.Y.128.30/32 33 ipfw nat 33 config redirect_addr 10.0.14.1 X.Y.128.30 ipfw nat 799 config ip 3.3.3.3 reverse 55000 nat tablearg ip from table(12) to not 3.3.3.3 via em3 55000 nat tablearg ip from any to table(13) via em3 55000 nat 799 ip from any to table(13) not via em3 55000 nat tablearg ip from 3.3.3.3 to table(13) 55000 nat tablearg ip from table(12) to 3.3.3.3 55000 nat 799 ip from table(13) to 3.3.3.3 This ruleset, makes static nat. To access from a client ( i.e. 10.0.14.5 ) to X.Y.128.30, i decided to make source address translation to incoming requests. All incoming static nat requests is to be exposed to reverse nat. ( source address translation as 3.3.3.3 ) All of this ruleset works, but under 5-6 mbps static nat traffic load ( and total throughput about 150-200 Mbps ), operating system's default gateway changes unexpectedly. New default gateway shown as the local ip address of static nat ( 10.0.14.1 ). If you apply bandwidthd limiting with dummynet, this problem occurs more frequently. Dummynet catalyze the problem. when i remove the reverse nat rules as : 55000 nat tablearg ip from table(12) to any 55000 nat tablearg ip from any to table(13) everything is fine. ( default gateway doesnt change. ). Altough dummynet is active, problem doesnt seen. I think there is no problem with dummynet. I tried different rulesets for different aims, and then i understood that if you inject packets into in-kernel nat more then once system behaves unexpectedly. Another example attached below: em0: flags=8843 metric 0 mtu 1500 options=219b ether 00:23:8b:89:e4:9c inet6 fe80::223:8bff:fe89:e49c%em0 prefixlen 64 scopeid 0x1 inet X.Y.128.1 netmask 0xffffff00 broadcast X.Y.128.255 inet X.Y.128.4 netmask 0xffffffff broadcast X.Y.128.4 nd6 options=3 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8843 metric 0 mtu 1500 options=219b ether 00:23:8b:89:e4:9d inet6 fe80::223:8bff:fe89:e49d%em1 prefixlen 64 scopeid 0x2 inet 192.168.254.254 netmask 0xffffff00 broadcast 192.168.254.255 nd6 options=3 media: Ethernet autoselect (1000baseT ) status: active em2: flags=8843 metric 0 mtu 1500 options=219b ether 00:23:8b:89:e4:9e inet6 fe80::223:8bff:fe89:e49e%em2 prefixlen 64 scopeid 0x4 inet 10.200.202.254 netmask 0xffffff00 broadcast 10.200.202.255 inet 1.1.184.254 netmask 0xffffff00 broadcast 1.1.184.255 nd6 options=3 media: Ethernet autoselect (1000baseT ) status: active bce0.14: flags=8843 metric 0 mtu 1500 options=103 ether 00:1b:78:38:55:f8 inet6 fe80::223:8bff:fe89:e49c%bce0.14 prefixlen 64 scopeid 0xb inet 10.0.14.254 netmask 0xffffff00 broadcast 10.0.14.255 inet 1.1.3.254 netmask 0xffffff00 broadcast 1.1.3.255 nd6 options=3 media: Ethernet autoselect (1000baseSX ) status: active vlan: 14 parent interface: bce0 ipfw nat 800 config same_ports reset redirect_addr 192.168.254.4 X.Y.128.4 45000 nat 800 ip from 192.168.254.4 to any not via em2 // DMZ: Web_Server 45000 nat 800 ip from any to X.Y.128.4 not via em2 // DMZ: Web_Server With this configuration, when you try to access from a client ( 10.0.14.5 ) to X.Y.128.4 the system hangs immediately. How can we debug this problem ? I think that this problem doesnt belong to misconfiguration. There is something wrong at in-kernel nat when packets reinjected to nat ? I can reproduce this situation. Regards, Thanks for your helps. Ozkan KIRIK