From owner-freebsd-ipfw@FreeBSD.ORG Wed Jul 12 22:48:34 2006 Return-Path: X-Original-To: ipfw@freebsd.org Delivered-To: freebsd-ipfw@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 50C2C16A4DE for ; Wed, 12 Jul 2006 22:48:34 +0000 (UTC) (envelope-from vladone@spaingsm.com) Received: from mail.spaingsm.com (llwb135.servidoresdns.net [217.76.137.82]) by mx1.FreeBSD.org (Postfix) with ESMTP id 51AC743D46 for ; Wed, 12 Jul 2006 22:48:30 +0000 (GMT) (envelope-from vladone@spaingsm.com) Received: from localhost (unknown [88.158.112.6]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.spaingsm.com (Postfix) with ESMTP id C8DC224C6CA for ; Thu, 13 Jul 2006 00:13:05 +0200 (CEST) Date: Thu, 13 Jul 2006 01:48:26 +0300 From: vladone X-Mailer: The Bat! (v3.80.03) Professional X-Priority: 3 (Normal) Message-ID: <19183199.20060713014826@spaingsm.com> To: ipfw@freebsd.org In-Reply-To: <48DC429CB053B64EAD91BDD1DE106A11675DAE@es1.corp.commspeed.net> References: <48DC429CB053B64EAD91BDD1DE106A11675DAE@es1.corp.commspeed.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: Re: IPFW Dummynet Bridge Limiting X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vladone List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jul 2006 22:48:34 -0000 Hello Adam, Wednesday, July 12, 2006, 9:13:11 PM, you wrote: > Hey all, > > I have searched and searched and searched and can't seem to > come up with the answer to this little mystery I have going on here. > Maybe I could get some help from this large group of people who are much > smarter than I am. I have a FreeBSD machine running 6.1-RC that has > three NICs, two of which are acting as a bridge. It's a pretty standard > setup. What I am attempting to accomplish is bandwidth limiting using > dummynet over this bridge. Here's the network layout: > > INTERNET ---- Core Router ---- Bridge (limiter) ---- Border Router ---- > Customer Base > > The reason for the bridge between two routers is because we > also have our server farm between those routers. The customer base > consists of multiple routed networks and they all get public IPs. The > problem I'm having is that the bridge is not limiting any of the > customer IPs. I see packets flowing through the IPFW rules but they're > not being passed to the pipes. I will show the configuration > momentarily. The weird thing is, I am able to unplug the Border Router > from this whole setup and plug a laptop in to the bridge and set it up > so the laptop IP is limited. This setup works fine and I can limit the > laptop the way I expect the rest of the network to be. Here's my > configuration with the Border Router plugged in and the 216.19.50.37 IP > being used in the "Customer Base": > > ---Kernel Config--- > options SMP # Symmetric MultiProcessor > Kernel > options IPFIREWALL # Firewall support > options IPFIREWALL_DEFAULT_TO_ACCEPT > options IPDIVERT > options DUMMYNET # Traffic limiting > options BRIDGE > options HZ=1000 # strongly recommended by > dummynet(4) > device apic # I/O APIC > > ---Sysctl--- > net.inet.ip.fw.enable=1 > net.inet.ip.fw.one_pass=1 > net.link.ether.bridge_cfg=em0,em1 > net.link.ether.bridge.enable=1 > net.link.ether.bridge_ipfw=1 > net.inet.ip.fw.dyn_buckets=256 > net.inet.ip.fw.curr_dyn_buckets=256 > > ---rc.conf--- > defaultrouter="[mydefaultrouter]" > hostname="[myhostname]" > ifconfig_bge0="[mymanagementinterface]" > cloned_interfaces="bridge0" > ifconfig_bridge0="addm em0 addm em1 up" > ifconfig_em0="up" > ifconfig_em1="up" > sshd_enable="YES" > firewall_enable="YES" > firewall_script="/etc/rc.firewall.bwmg" # this just runs ipfw with > the rules supplied in custom_firewall below > firewall_quiet="NO" > firewall_logging="YES" > firewall_flags="" > > ---ifconfig---- > -snip- > em0: flags=8943 mtu 1500 > options=8 > ether 00:04:23:cb:60:aa > media: Ethernet autoselect (100baseTX ) > status: active > em1: flags=8943 mtu 1500 > options=8 > ether 00:04:23:cb:60:ab > media: Ethernet autoselect (100baseTX ) > status: active > lo0: flags=8049 mtu 16384 > inet 127.0.0.1 netmask 0xff000000 > bridge0: flags=8043 mtu 1500 > ether ac:de:48:ce:fe:5c > priority 32768 hellotime 2 fwddelay 15 maxage 20 > member: em1 flags=3 > member: em0 flags=3 > > ---custom_firewall--- > -q flush > -q queue flush > -q pipe flush > add 1 allow all from any to any via lo0 > add 2 deny all from any to 127.0.0.0/8 > add 3 deny all from 127.0.0.0/8 to any > add 4 skipto 65534 all from any to any via bge0 > add 65534 allow all from any to any > add 100 pipe 100 config bw 100Kbit/s > add 10 pipe 100 all from any to 216.19.50.37 recv em0 > > # ipfw show 10 > 00010 11430 925353 pipe 100 all from any to 216.19.50.37 > recv em0 > > # ipfw pipe show 100 > 00100: 100.000 Kbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail > mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 > BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes > Pkt/Byte Drp > 0 icmp 216.109.112.135/0 216.19.50.37/0 11434 925679 0 > 0 0 > > I have tried many different configurations including > changing net.inet.ip.fw.one_pass to 0, changing the ipfw rule to recv > and xmit on BOTH devices of the bridge, changing the ipfw rule from all > to tcp and ip, and changing the rule from "any to 216.19.50.37" to > "216.19.50.37 to any" (recv and xmit on both interfaces). I've also > tried the kernel without IPDIVERT and with if_bridge. As I stated > before, the odd thing is that when I plug directly into it with an IP of > 216.19.0.225 (can't use the other one here) and modify the rules to > reflect the new IP, the limiting works just fine. I have a feeling this > is where the problem is, but I can't quite think of any reason why this > wouldn't work. Previously, I had a Linux machine running TC installed in > place of this machine but I personally prefer FreeBSD and feel ipfw is > easier to configure than tc. The Linux machine worked just fine. > > Could anyone possibly help with this little problem? I'm > stuck. Also, if I forgot to include any information, I apologize. I'm a > bit spacey when I write emails. Just let me know what I missed and I can > explain further. Thanks. > > Adam > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to > "freebsd-ipfw-unsubscribe@freebsd.org" I dont't use it bridge but some thinks that can help u: 1. use corect syctl variables form: net.link.ether.bridge.ipfw instead net.link.ether.bridge_ipfw (probably an wrong typing) 2. read the end from man page about bridge, and net.inet.ip.fw.one_pass variable. "Also remember that bridged packets are accepted after the first pass through the firewall irrespective of the setting of the sysctl variable net.inet.ip.fw.one_pass, and that some ipfw(8) actions such as divert do not apply to bridged packets. It might be useful to have a rule of the form skipto 20000 ip from any to any bridged " 3. Luigi Rizzo say in his documentation: "there is always one pass for bridged packets" -- Best regards, vladone mailto:vladone@spaingsm.com