From owner-freebsd-ipfw@FreeBSD.ORG Wed Jul 12 18:13:44 2006 Return-Path: X-Original-To: freebsd-ipfw@freebsd.org Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3DD316A4E0 for ; Wed, 12 Jul 2006 18:13:44 +0000 (UTC) (envelope-from adamt@commspeed.net) Received: from es1.corp.commspeed.net (es1.corp.commspeed.net [216.19.2.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF11A43D7D for ; Wed, 12 Jul 2006 18:13:16 +0000 (GMT) (envelope-from adamt@commspeed.net) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Wed, 12 Jul 2006 11:13:11 -0700 Message-ID: <48DC429CB053B64EAD91BDD1DE106A11675DAE@es1.corp.commspeed.net> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: IPFW Dummynet Bridge Limiting Thread-Index: Acal3tXug0ySFVeITS279NU1/82DSg== From: "Adam M. Towarnyckyj" To: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: IPFW Dummynet Bridge Limiting X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jul 2006 18:13:44 -0000 Hey all, =20 I have searched and searched and searched and can't seem to come up with the answer to this little mystery I have going on here. Maybe I could get some help from this large group of people who are much smarter than I am. I have a FreeBSD machine running 6.1-RC that has three NICs, two of which are acting as a bridge. It's a pretty standard setup. What I am attempting to accomplish is bandwidth limiting using dummynet over this bridge. Here's the network layout: =20 INTERNET ---- Core Router ---- Bridge (limiter) ---- Border Router ---- Customer Base =20 The reason for the bridge between two routers is because we also have our server farm between those routers. The customer base consists of multiple routed networks and they all get public IPs. The problem I'm having is that the bridge is not limiting any of the customer IPs. I see packets flowing through the IPFW rules but they're not being passed to the pipes. I will show the configuration momentarily. The weird thing is, I am able to unplug the Border Router from this whole setup and plug a laptop in to the bridge and set it up so the laptop IP is limited. This setup works fine and I can limit the laptop the way I expect the rest of the network to be. Here's my configuration with the Border Router plugged in and the 216.19.50.37 IP being used in the "Customer Base": =20 ---Kernel Config--- options SMP # Symmetric MultiProcessor Kernel options IPFIREWALL # Firewall support options IPFIREWALL_DEFAULT_TO_ACCEPT options IPDIVERT options DUMMYNET # Traffic limiting options BRIDGE options HZ=3D1000 # strongly recommended by dummynet(4) device apic # I/O APIC =20 ---Sysctl--- net.inet.ip.fw.enable=3D1 net.inet.ip.fw.one_pass=3D1 net.link.ether.bridge_cfg=3Dem0,em1 net.link.ether.bridge.enable=3D1 net.link.ether.bridge_ipfw=3D1 net.inet.ip.fw.dyn_buckets=3D256 net.inet.ip.fw.curr_dyn_buckets=3D256 =20 ---rc.conf--- defaultrouter=3D"[mydefaultrouter]" hostname=3D"[myhostname]" ifconfig_bge0=3D"[mymanagementinterface]" cloned_interfaces=3D"bridge0" ifconfig_bridge0=3D"addm em0 addm em1 up" ifconfig_em0=3D"up" ifconfig_em1=3D"up" sshd_enable=3D"YES" firewall_enable=3D"YES" firewall_script=3D"/etc/rc.firewall.bwmg" # this just runs ipfw with the rules supplied in custom_firewall below firewall_quiet=3D"NO" firewall_logging=3D"YES" firewall_flags=3D"" =20 ---ifconfig---- -snip- em0: flags=3D8943 mtu = 1500 options=3D8 ether 00:04:23:cb:60:aa media: Ethernet autoselect (100baseTX ) status: active em1: flags=3D8943 mtu = 1500 options=3D8 ether 00:04:23:cb:60:ab media: Ethernet autoselect (100baseTX ) status: active lo0: flags=3D8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000=20 bridge0: flags=3D8043 mtu 1500 ether ac:de:48:ce:fe:5c priority 32768 hellotime 2 fwddelay 15 maxage 20 member: em1 flags=3D3 member: em0 flags=3D3 =20 ---custom_firewall--- -q flush -q queue flush -q pipe flush add 1 allow all from any to any via lo0 add 2 deny all from any to 127.0.0.0/8 add 3 deny all from 127.0.0.0/8 to any add 4 skipto 65534 all from any to any via bge0 add 65534 allow all from any to any add 100 pipe 100 config bw 100Kbit/s add 10 pipe 100 all from any to 216.19.50.37 recv em0 =20 # ipfw show 10 00010 11430 925353 pipe 100 all from any to 216.19.50.37 recv em0 =20 # ipfw pipe show 100 00100: 100.000 Kbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 icmp 216.109.112.135/0 216.19.50.37/0 11434 925679 0 0 0 =20 I have tried many different configurations including changing net.inet.ip.fw.one_pass to 0, changing the ipfw rule to recv and xmit on BOTH devices of the bridge, changing the ipfw rule from all to tcp and ip, and changing the rule from "any to 216.19.50.37" to "216.19.50.37 to any" (recv and xmit on both interfaces). I've also tried the kernel without IPDIVERT and with if_bridge. As I stated before, the odd thing is that when I plug directly into it with an IP of 216.19.0.225 (can't use the other one here) and modify the rules to reflect the new IP, the limiting works just fine. I have a feeling this is where the problem is, but I can't quite think of any reason why this wouldn't work. Previously, I had a Linux machine running TC installed in place of this machine but I personally prefer FreeBSD and feel ipfw is easier to configure than tc. The Linux machine worked just fine. =20 Could anyone possibly help with this little problem? I'm stuck. Also, if I forgot to include any information, I apologize. I'm a bit spacey when I write emails. Just let me know what I missed and I can explain further. Thanks. =20 Adam