Date: Tue, 26 Oct 2004 15:55:02 -0700 From: Vincent Poy <vincepoy@gmail.com> To: Luigi Rizzo <luigi@freebsd.org>, Andre Oppermann <andre@freebsd.org>, freebsd-current@freebsd.org Subject: Re: Traffic Shaping not working correctly after ipfw coverted to use pfil_hooks API Message-ID: <429af92e041026155572805e89@mail.gmail.com> In-Reply-To: <429af92e041026143322b2d286@mail.gmail.com> References: <429af92e041020205510c66168@mail.gmail.com> <4177B899.5EC32F5F@freebsd.org> <429af92e04102114472add0e51@mail.gmail.com> <417835C7.7060808@freebsd.org> <429af92e04102404115bc7bc80@mail.gmail.com> <20041026133043.A24138@xorpc.icir.org> <429af92e041026143322b2d286@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 26 Oct 2004 13:30:43 -0700, Luigi Rizzo <luigi@freebsd.org> wrote:
> On Sun, Oct 24, 2004 at 04:37:32PM +0200, Andre Oppermann wrote:
> > [bouncing over to Luigi]
> >
> > Luigi, do you have any idea what might be going wrong here?
>
> no, sorry... I have to say the ipfw/natd/dummynet configuration is rather
> convoluted here so it is a bit hard to tell whether the
> problem is in dummynet calls or divert sockets.
For some reason, unless I add the divert lines, then I would have
problems connecting to things on the local LAN. But apparently taking
those lines out work correctly now so the new rules look like this:
root@bigbang [3:23pm][/home/vince] >> ipfw show
00049 25897703 8070600411 skipto 100 ip from 208.201.244.224/29 to any
00050 3036002 974553273 divert 8668 ip from any to any via xl0
00100 307228 50597030 allow ip from any to any via lo0
00200 0 0 deny ip from any to 127.0.0.0/8
00300 0 0 deny ip from 127.0.0.0/8 to any
63000 471 24670 allow ip from any to 10.0.0.0/8 out
63001 50 2607 allow ip from any to 172.16.0.0/12 out
63002 2856 189431 allow ip from any to 192.168.0.0/16 out
63003 126127 13511498 allow ip from any to 208.201.244.224/29 out
63004 9340689 3805111693 queue 1 tcp from any to any tcpflags ack out
63005 236 20124 queue 2 tcp from any to any dst-port 22,23 out
63006 3183061 318083534 queue 2 udp from any to any not dst-port 80,443 out
63007 248561 22392323 queue 3 ip from any to any dst-port 80,443 out
63008 33410 4846163 queue 4 ip from any to any out
65000 15619696 4840095386 allow ip from any to any
65535 1 46 deny ip from any to any
The reason for line 49 is that unless I put that there, then all the
static IP's will
connect to remote sites with the IP of the FreeBSD box instead of the actual
IP itself.
> I am also confused by the numbers in the initial report:
>
> > > > >>Vincent Poy wrote:
> > > > >>
> > > > >>>However, after the latest -CURRENT upgrade, it will do 200KB/sec down
> > > > >>>and 52KB/sec up. If I only download only, then it does show
> > > > >>>650KB/sec. Normally, when I change the bandwidth to a number lower
> > > > >>>than 480Kbps for the pipe, the download speeds would go up when
> > > > >>>downloading. However, I have tried in 10kbps steps down to 350kbps
> > > > >>>but it still did not top 200KB/sec in downloading.
>
> there is a mix of two different notations, Kbps and KB/sec, and
> i cannot make sense of them.
> Finally, I am curious as to why one would mix the upload and download
> traffic, i believe *DSL data rates are independent in the two
> directions unlike analog modems...
Sorry for the confusion. I have a 6016Kilobit/sec down -
608Kilobit/sec up ADSL connection except I have 8 static IP's which is
the reason for the /29. Due to the DSL using ATM, there is a 13% or
so for overhead. So the maximum speeds I get on the line is
650KiloBytes/sec down and 65KiloBytes/sec up but because the pipe
sizes are smaller on the outgoing than the incoming, unless I do
traffic shaping using ipfw2/dummynet then the downloads slow down when
I am uploading as the acknowledgement packets do not have priority
going out.
I am not mixing upload and download traffic as ADSL is Assymmetric
which means that the downloading capacity is bigger than the uploading
capacity.
In the March 6, 2004 -CURRENT and before, ipfw2/dummynet would work
correctly as I only need the traffic shaping for upload and not
download so by setting the upload pipe to 480Kilobits/sec out of
608Kilobits/sec, it would let me download at near 100% of the speeds
when I am uploading. However, with the October 22, 2004 -CURRENT
using the same exact configuration and I did manually do all the same
changes to my customized kernel as the GENERIC kernel, the traffic
shaping is not working correctly as when I upload and don't download,
the speed is correct.
Here is what my speeds with ftp with the upload pipe set to 480Kilobits/sec.
upload only:
10485760 bytes sent in 03:15 (52.30 KB/s)
download only:
10485760 bytes received in 00:16 (614.07 KB/s)
Now, if I did a upload/download at the same time, this is where
ipfw2/dummynet doesn't work correctly:
upload: 10485760 bytes sent in 03:21 (50.76 KB/s)
download: 10485760 bytes received in 00:57 (176.92 KB/s)
So you can see that while the upload is working at the correct speed,
the download is only doing 176.92KB/sec instead of anywhere near
614.07KB/sec which seems like ipfw2/dummynet isn't sending the ack
packets on the upload pipe before the other traffic while the pipe
speed is working.
This is with the above rules and this is what the queues and pipes show:
root@bigbang [9:50pm][/usr/temp/gtalk-0.99.10] >> ipfw pipe show
00001: 480.000 Kbit/s 0 ms 50 sl. 0 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
q00001: weight 100 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 208.201.244.225/3254 64.12.185.119/80 9467851 3826687467 30 19360
q00002: weight 66 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 udp 208.201.244.225/2979 217.12.4.104/53 3189174 318678628 0 0 7
q00003: weight 33 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 208.201.244.225/3254 64.12.185.119/80 248711 22400111 0 0 338
q00004: weight 1 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 208.201.244.226/3746 216.155.193.173/5050 33492 4850675 0 0 0
root@bigbang [3:46pm][/usr/temp/gtalk-0.99.10] >> ipfw queue show
q00001: weight 100 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 208.201.244.225/3254 64.12.185.119/80 9473364 3830013686 7 10504
q00002: weight 66 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 udp 208.201.244.225/2979 217.12.4.104/53 3189317 318690863 2 263 7
q00003: weight 33 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 208.201.244.225/3254 64.12.185.119/80 248715 22400319 0 0 338
q00004: weight 1 pipe 1 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 208.201.244.226/3746 216.155.193.173/5050 33492 4850675 0 0 0
Using these lines in /etc/rc.firewall in addition to the above ipfw
show of the rules:
${fwcmd} enable one_pass
# Define our upload pipe
${fwcmd} pipe 1 config bw 608Kbit/s
# Define a high-priority queue
${fwcmd} queue 1 config pipe 1 weight 100
# Define a medium-high-priority queue
${fwcmd} queue 2 config pipe 1 weight 66
# Define a medium-low-priority queue
${fwcmd} queue 3 config pipe 1 weight 33
# Define a low-priority queue
${fwcmd} queue 4 config pipe 1 weight 1
Hope this explains it better.
Cheers,
Vince
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?429af92e041026155572805e89>