From owner-freebsd-net@FreeBSD.ORG Tue May 2 18:06:11 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 70E9916A709 for ; Tue, 2 May 2006 18:06:11 +0000 (UTC) (envelope-from julian@elischer.org) Received: from a50.ironport.com (a50.ironport.com [63.251.108.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 32EDB43D62 for ; Tue, 2 May 2006 18:06:04 +0000 (GMT) (envelope-from julian@elischer.org) Received: from unknown (HELO [10.251.19.131]) ([10.251.19.131]) by a50.ironport.com with ESMTP; 02 May 2006 11:06:02 -0700 Message-ID: <44579F89.6020703@elischer.org> Date: Tue, 02 May 2006 11:06:01 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.13) Gecko/20060414 X-Accept-Language: en-us, en MIME-Version: 1.0 To: tpeixoto@widesoft.com.br References: <49594.200.230.201.250.1146063341.squirrel@www.widemail.com.br> <444F8E89.2050905@wildcard.net.uk> <56286.200.230.201.250.1146067775.squirrel@www.widemail.com.br> <1146073590.1089.80.camel@sky.mediasat.ro> <59615.200.230.201.250.1146083577.squirrel@www.widemail.com.br> <445038CA.2050008@pacific.net.sg> <4456AD8E.2060703@widesoft.com.br> <4456B415.3080901@elischer.org> <4456BF4A.7050107@widesoft.com.br> <4456D19F.7030101@elischer.org> <4456D553.30202@elischer.org> <4456D6A3.8080503@elischer.org> <59701.200.230.201.250.1146589752.squirrel@www.widemail.com.br> In-Reply-To: <59701.200.230.201.250.1146589752.squirrel@www.widemail.com.br> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: Lee Johnston , freebsd-net@freebsd.org, mihai@duras.ro Subject: Re: Packet loss with traffic shaper and routing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2006 18:06:11 -0000 tpeixoto@widesoft.com.br wrote: >Hello. >I think I should give some 'real world' examples. > > >/etc/rc.firewall: > >[Ss][Hh][Aa][Pp][Ee][Rr]) > setup_loopback > > . /etc/rc.shaper > > ${fwcmd} add 65000 pass all from any to any > ;; > > >/etc/rc.shaper: > >${fwcmd} pipe 1 config bw 512Kbit/s >${fwcmd} pipe 2 config bw 512Kbit/s >${fwcmd} add pipe 1 all from any to any MAC any 00:11:22:33:44:55 in >${fwcmd} add pipe 2 all from any to any MAC 00:11:22:33:44:55 any out >${fwcmd} pipe 3 config bw 256Kbit/s >${fwcmd} pipe 4 config bw 256Kbit/s >${fwcmd} add pipe 3 all from any to any MAC any 66:77:88:99:aa:bb in >${fwcmd} add pipe 4 all from any to any MAC 66:77:88:99:aa:bb any out >${fwcmd} pipe 5 config bw 128Kbit/s >${fwcmd} pipe 6 config bw 128Kbit/s >${fwcmd} add pipe 5 all from any to any MAC any 00:01:02:03:04:05 in >${fwcmd} add pipe 6 all from any to any MAC 00:01:02:03:04:05 any out >${fwcmd} pipe 7 config bw 512Kbit/s >${fwcmd} pipe 8 config bw 1024Kbit/s >${fwcmd} add pipe 7 all from any to any MAC any 06:07:08:09:0a:0b in >${fwcmd} add pipe 8 all from any to any MAC 06:07:08:09:0a:0b any out >${fwcmd} pipe 9 config bw 64Kbit/s >${fwcmd} pipe 10 config bw 64Kbit/s >${fwcmd} add pipe 9 all from any to any MAC any ab:cd:ef:00:11:22 in >${fwcmd} add pipe 10 all from any to any MAC ab:cd:ef:00:11:22 any out > > OK, so, put the MACs in numerical order: 00:01:02:03:04:05 00:11:22:33:44:55 06:07:08:09:0a:0b 66:77:88:99:aa:bb ab:cd:ef:00:11:22 work out MASKS that divide them into a binary set. e.g. 1 skipto 10 all from any to not MAC 00:00:00:00:00:00/8 2 skipto 5 all from any to not MAC 00:01:00:00:00:00/16 3 pipe 1 ip from any to any 5 pipe 2 ip from any to any 10 skipto 12 all from any to not MAC 06:00:00:00:00:00/8 11 pipe 3 all from any to any 12 skipto 14 all from any to not MAC 66:00:00:00:00:00/8 13 pipe 4 all from any to any 14 pipe 5 all from any to any now, if you continue this on, you will run 16 rules to divide the 1600 rules up to find the right pipe. > >This example is for 5 clients. We have 1600. >As you can see, there are 2 rules and 2 pipes per host, not 1600. > > >If we try rc.firewall like this... > >setup_loopback >${fwcmd} add 65000 pass all from any to any > >... we are ok. Interrupts are low. > >So, following your line of thought, I tried a simple test... > >setup_loopback >${fwcmd} skipto 65000 ip from any to any MAC any any >. /etc/rc.shaper >${fwcmd} add 65000 pass all from any to any > >This way, the packets will never pass through shaper rules, but interrupts >still get very high. > > I don't see how that proves anything >Basically, we need a solution to shape each MAC address with its specifics >download e upload speeds. >Given the tests, I don't see how skipto can help, but if you believe that >tablearg (which I am not familiar with) might help, we can try it with >7.x. > > Tablearg only works with IP addresses. >Thanks. > > > > >>oops, forgot to fix my cut-n- pastes.. corrected triage below.. >> >> >>Julian Elischer wrote: >> >> >> >>>Julian Elischer wrote: >>> >>> >>> >>>>tpeixoto@widesoft.com.br wrote: >>>> >>>> >>>> >>>>>>That would do it.. >>>>>> >>>>>>In all versions of FreeBSD >>>>>>you can use the skipto rule to make sure that only a few rules are >>>>>>run for any >>>>>>address. Use it to to a binary search for the right pipe.' >>>>>>carefully using 'skipto' and 'table' can make it efficient to do >>>>>>very complex >>>>>>filters like that. >>>>>> >>>>>> >>>>>> >>>>>Sorry, but I didn't realized how to use that as we have to shape >>>>>each user individually, i.e., each MAC address on the LAN has its >>>>>own download and upload speeds. >>>>> >>>>>Could you clarify how to improve the situation with the tools you >>>>>mentioned? >>>>> >>>>> >>>> >>>> >>>> >>>>Assuming you can not use "tablearg" yet (it will make this REALLY EASY) >>>>then if you have 30 IPs you want to shape from 1.1.1.1 to 1.1.1.30 >>>> >>>> >>> >>> >>>then, consider the following example using IP addresses. >>> >>> >>> >>>> >>>> >>>> >>>ipfw add 1000 skipto 1110 ip from any to 1.1.1.16/28 >>>ipfw add 1010 skipto 1032 ip from any to 1.1.1.8/29 >>>ipfw add 1012 skipto 1021 ip from any to 1.1.1.4./30 >>> >>> >>>ipfw add 1013 [anything] ip from any to 1.1.1.0 >>> >>> >>>ipfw add 1014 [anything] ip from any to 1.1.1.1 >>>ipfw add 1015 [anything] ip from any to 1.1.1.2 >>>ipfw add 1016 [anything] ip from any to 1.1.1.3 >>> >>> >>>ipfw add 1021 anything] ip from any to 1.1.1.4 >>>ipfw add 1022 [anything] ip from any to 1.1.1.5 >>>ipfw add 1023 [anything] ip from any to 1.1.1.6 >>>ipfw add 1024 [anything] ip from any to 1.1.1.7 >>> >>> >>>ipfw add 1032 skipto 1051 ip from any to 1.1.1.12./30 >>> >>>ipfw add 1040 [anything] ip from any to 1.1.1.8 >>>ipfw add 1041 [anything] ip from any to 1.1.1.9 >>>ipfw add 1042 [anything] ip from any to 1.1.1.10 >>>ipfw add 1043 [anything] ip from any to 1.1.1.11 >>> >>> >>>ipfw add 1051 [anything] ip from any to 1.1.1.12 >>>ipfw add 1052 [anything] ip from any to 1.1.1.13 >>>ipfw add 1053 [anything] ip from any to 1.1.1.14 >>>ipfw add 1054 [anything] ip from any to 1.1.1.15 >>> >>> >>>ipfw add 1110 skipto 1132 ip from any to 1.1.1.24/29 >>>ipfw add 1112 skipto 1121 ip from any to 1.1.1.20./30 >>>ipfw add 1113 [anything] ip from any to 1.1.1.16 >>>ipfw add 1114 [anything] ip from any to 1.1.1.17 >>>ipfw add 1115 [anything] ip from any to 1.1.1.18 >>> >>> >>>ipfw add 1116 [anything] ip from any to 1.1.1.19 >>> >>> >>>ipfw add 1121 anything] ip from any to 1.1.1.20 >>>ipfw add 1122 [anything] ip from any to 1.1.1.21 >>>ipfw add 1123 [anything] ip from any to 1.1.1.22 >>>ipfw add 1124 [anything] ip from any to 1.1.1.23 >>> >>> >>>ipfw add 1132 skipto 1151 ip from any to 1.1.1.28./30 >>> >>>ipfw add 1140 [anything] ip from any to 1.1.1.24 >>>ipfw add 1141 [anything] ip from any to 1.1.1.25 >>>ipfw add 1142 [anything] ip from any to 1.1.1.26 >>>ipfw add 1143 [anything] ip from any to 1.1.1.27 >>> >>> >>>ipfw add 1151 [anything] ip from any to 1.1.1.28 >>>ipfw add 1152 [anything] ip from any to 1.1.1.29 >>>ipfw add 1153 [anything] ip from any to 1.1.1.30 >>>ipfw add 1154 [anything] ip from any to 1.1.1.31 >>> >>> >>> >>> >>> >>>now this example shows a binary search in IP space, written (including >>>bugs) by hand >>>but if you are willing to write a suitable perl script, you can >>>generate a binary search in MAC address space >>>just as easily. just sort them into order and search.. >>> >>>I'm not going to try it by had, but for 1600 hosts you should only >>>need to go through >>>15 rules per host on average, instead of 1600 rules per host. >>>that should cut down your ipfw cpu usage by 1/100 >>> >>> >>> >>> >>> >>>>freebsd.org" >>>> >>>> > > >