From owner-freebsd-current@FreeBSD.ORG Tue Oct 26 22:55:04 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1106D16A4CF for ; Tue, 26 Oct 2004 22:55:04 +0000 (GMT) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2B75A43D2D for ; Tue, 26 Oct 2004 22:55:03 +0000 (GMT) (envelope-from vincepoy@gmail.com) Received: by wproxy.gmail.com with SMTP id 68so240446wri for ; Tue, 26 Oct 2004 15:55:02 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=n+4xjR96dR9ljw1opQGFa6H9e1dW5O7nuEsKEGYxX8Ze40G8RiVbo3TD7kTFxzMQhRvRNviuygew+1e90QxrugXJKSFEGwIcdjZmUHHtLb1XqNhyGZ1ac8ul0HVLjrzCui1R5CXYo0QIol6aW9gykN3RMA3dN1WsLp7jPC0FS1c= Received: by 10.38.96.46 with SMTP id t46mr790322rnb; Tue, 26 Oct 2004 15:55:02 -0700 (PDT) Received: by 10.38.14.49 with HTTP; Tue, 26 Oct 2004 15:55:02 -0700 (PDT) Message-ID: <429af92e041026155572805e89@mail.gmail.com> Date: Tue, 26 Oct 2004 15:55:02 -0700 From: Vincent Poy To: Luigi Rizzo , Andre Oppermann , freebsd-current@freebsd.org In-Reply-To: <429af92e041026143322b2d286@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <429af92e041020205510c66168@mail.gmail.com> <4177B899.5EC32F5F@freebsd.org> <429af92e04102114472add0e51@mail.gmail.com> <417835C7.7060808@freebsd.org> <429af92e04102404115bc7bc80@mail.gmail.com> <20041026133043.A24138@xorpc.icir.org> <429af92e041026143322b2d286@mail.gmail.com> Subject: Re: Traffic Shaping not working correctly after ipfw coverted to use pfil_hooks API X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Vincent Poy List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Oct 2004 22:55:04 -0000 On Tue, 26 Oct 2004 13:30:43 -0700, Luigi Rizzo wrote: > On Sun, Oct 24, 2004 at 04:37:32PM +0200, Andre Oppermann wrote: > > [bouncing over to Luigi] > > > > Luigi, do you have any idea what might be going wrong here? > > no, sorry... I have to say the ipfw/natd/dummynet configuration is rather > convoluted here so it is a bit hard to tell whether the > problem is in dummynet calls or divert sockets. For some reason, unless I add the divert lines, then I would have problems connecting to things on the local LAN. But apparently taking those lines out work correctly now so the new rules look like this: root@bigbang [3:23pm][/home/vince] >> ipfw show 00049 25897703 8070600411 skipto 100 ip from 208.201.244.224/29 to any 00050 3036002 974553273 divert 8668 ip from any to any via xl0 00100 307228 50597030 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 63000 471 24670 allow ip from any to 10.0.0.0/8 out 63001 50 2607 allow ip from any to 172.16.0.0/12 out 63002 2856 189431 allow ip from any to 192.168.0.0/16 out 63003 126127 13511498 allow ip from any to 208.201.244.224/29 out 63004 9340689 3805111693 queue 1 tcp from any to any tcpflags ack out 63005 236 20124 queue 2 tcp from any to any dst-port 22,23 out 63006 3183061 318083534 queue 2 udp from any to any not dst-port 80,443 out 63007 248561 22392323 queue 3 ip from any to any dst-port 80,443 out 63008 33410 4846163 queue 4 ip from any to any out 65000 15619696 4840095386 allow ip from any to any 65535 1 46 deny ip from any to any The reason for line 49 is that unless I put that there, then all the static IP's will connect to remote sites with the IP of the FreeBSD box instead of the actual IP itself. > I am also confused by the numbers in the initial report: > > > > > >>Vincent Poy wrote: > > > > >> > > > > >>>However, after the latest -CURRENT upgrade, it will do 200KB/sec down > > > > >>>and 52KB/sec up. If I only download only, then it does show > > > > >>>650KB/sec. Normally, when I change the bandwidth to a number lower > > > > >>>than 480Kbps for the pipe, the download speeds would go up when > > > > >>>downloading. However, I have tried in 10kbps steps down to 350kbps > > > > >>>but it still did not top 200KB/sec in downloading. > > there is a mix of two different notations, Kbps and KB/sec, and > i cannot make sense of them. > Finally, I am curious as to why one would mix the upload and download > traffic, i believe *DSL data rates are independent in the two > directions unlike analog modems... Sorry for the confusion. I have a 6016Kilobit/sec down - 608Kilobit/sec up ADSL connection except I have 8 static IP's which is the reason for the /29. Due to the DSL using ATM, there is a 13% or so for overhead. So the maximum speeds I get on the line is 650KiloBytes/sec down and 65KiloBytes/sec up but because the pipe sizes are smaller on the outgoing than the incoming, unless I do traffic shaping using ipfw2/dummynet then the downloads slow down when I am uploading as the acknowledgement packets do not have priority going out. I am not mixing upload and download traffic as ADSL is Assymmetric which means that the downloading capacity is bigger than the uploading capacity. In the March 6, 2004 -CURRENT and before, ipfw2/dummynet would work correctly as I only need the traffic shaping for upload and not download so by setting the upload pipe to 480Kilobits/sec out of 608Kilobits/sec, it would let me download at near 100% of the speeds when I am uploading. However, with the October 22, 2004 -CURRENT using the same exact configuration and I did manually do all the same changes to my customized kernel as the GENERIC kernel, the traffic shaping is not working correctly as when I upload and don't download, the speed is correct. Here is what my speeds with ftp with the upload pipe set to 480Kilobits/sec. upload only: 10485760 bytes sent in 03:15 (52.30 KB/s) download only: 10485760 bytes received in 00:16 (614.07 KB/s) Now, if I did a upload/download at the same time, this is where ipfw2/dummynet doesn't work correctly: upload: 10485760 bytes sent in 03:21 (50.76 KB/s) download: 10485760 bytes received in 00:57 (176.92 KB/s) So you can see that while the upload is working at the correct speed, the download is only doing 176.92KB/sec instead of anywhere near 614.07KB/sec which seems like ipfw2/dummynet isn't sending the ack packets on the upload pipe before the other traffic while the pipe speed is working. This is with the above rules and this is what the queues and pipes show: root@bigbang [9:50pm][/usr/temp/gtalk-0.99.10] >> ipfw pipe show 00001: 480.000 Kbit/s 0 ms 50 sl. 0 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 q00001: weight 100 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 208.201.244.225/3254 64.12.185.119/80 9467851 3826687467 30 19360 q00002: weight 66 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 udp 208.201.244.225/2979 217.12.4.104/53 3189174 318678628 0 0 7 q00003: weight 33 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 208.201.244.225/3254 64.12.185.119/80 248711 22400111 0 0 338 q00004: weight 1 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 208.201.244.226/3746 216.155.193.173/5050 33492 4850675 0 0 0 root@bigbang [3:46pm][/usr/temp/gtalk-0.99.10] >> ipfw queue show q00001: weight 100 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 208.201.244.225/3254 64.12.185.119/80 9473364 3830013686 7 10504 q00002: weight 66 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 udp 208.201.244.225/2979 217.12.4.104/53 3189317 318690863 2 263 7 q00003: weight 33 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 208.201.244.225/3254 64.12.185.119/80 248715 22400319 0 0 338 q00004: weight 1 pipe 1 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 0 tcp 208.201.244.226/3746 216.155.193.173/5050 33492 4850675 0 0 0 Using these lines in /etc/rc.firewall in addition to the above ipfw show of the rules: ${fwcmd} enable one_pass # Define our upload pipe ${fwcmd} pipe 1 config bw 608Kbit/s # Define a high-priority queue ${fwcmd} queue 1 config pipe 1 weight 100 # Define a medium-high-priority queue ${fwcmd} queue 2 config pipe 1 weight 66 # Define a medium-low-priority queue ${fwcmd} queue 3 config pipe 1 weight 33 # Define a low-priority queue ${fwcmd} queue 4 config pipe 1 weight 1 Hope this explains it better. Cheers, Vince