Date: Mon, 11 Oct 2010 17:16:09 +0200 From: Luigi Rizzo <rizzo@iet.unipi.it> To: Dmukha Nikolay <cosmic17@yandex.ru> Cc: freebsd-net@freebsd.org Subject: Re: Strange behavior of packet scheduling in ipfw3 Message-ID: <20101011151609.GB41572@onelab2.iet.unipi.it> In-Reply-To: <138981286808043@web117.yandex.ru> References: <138981286808043@web117.yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
[top post as i am in a rush] first, there is (fixed in HEAD, but not yet in -stable) an issue with io_fast=1 so try to set it to 0. Secondly, because of the way scheduling works, you can't expect packets to bypass the pipe, even if the individual flow is below its share. Even the best scheduling algorithm has a worst case delay which is at least 1 max-sized packet (the constant is actually 3..6 times for QFQ). cheers luigi On Mon, Oct 11, 2010 at 06:40:43PM +0400, Dmukha Nikolay wrote: > Hello! > > The system is: > FreeBSD mysystem 8.0-STABLE-201005 FreeBSD 8.0-STABLE-201005 #0: Wed Jul 28 12:04:29 MSD 2010 root@mysystem:/usr/src/sys/amd64/compile/MYKERNEL amd64 > > There is firewall "ipfw3" from Luigi Rizzo with packet scheduling. > There is part of firewall config (tariff with 1Mbit/s speed, for example), below (the rules for another speeds are the same): > $IPFW pipe 11 config bw 1040Kbit/s mask dst-ip 0xffffffff > $IPFW pipe 12 config bw 1040Kbit/s mask src-ip 0xffffffff > ########pipe 11 > $IPFW sched 11 config type QFQ mask dst-ip 0xffffff00 > $IPFW queue 111 config sched 11 weight 10 > $IPFW queue 112 config sched 11 weight 8 > $IPFW queue 113 config sched 11 weight 4 > $IPFW queue 114 config sched 11 weight 1 > $IPFW add queue 111 ip from any to table\(10\) via igb0 out proto udp src-port 5060 > $IPFW add queue 112 ip from any to table\(10\) via igb0 out proto tcp src-port 80,443,8080 > $IPFW add queue 113 ip from any to table\(10\) via igb0 out proto tcp src-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 > $IPFW add queue 113 ip from any to table\(10\) via igb0 out proto icmp > $IPFW add queue 114 ip from any to table\(10\) via igb0 out > $IPFW add queue 111 ip from any to table\(10\) via igb2 out proto udp src-port 5060 > $IPFW add queue 112 ip from any to table\(10\) via igb2 out proto tcp src-port 80,443,8080 > $IPFW add queue 113 ip from any to table\(10\) via igb2 out proto tcp src-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 > $IPFW add queue 113 ip from any to table\(10\) via igb2 out proto icmp > $IPFW add queue 114 ip from any to table\(10\) via igb2 out > ########pipe 12 > $IPFW sched 12 config type QFQ mask src-ip 0xffffff00 > $IPFW queue 121 config sched 12 weight 10 > $IPFW queue 122 config sched 12 weight 8 > $IPFW queue 123 config sched 12 weight 4 > $IPFW queue 124 config sched 12 weight 1 > $IPFW add queue 1210 ip from table\(11\) to any via igb1 out proto udp dst-port 5060 > $IPFW add queue 122 ip from table\(11\) to any via igb1 out proto tcp dst-port 80,443,8080 > $IPFW add queue 123 ip from table\(11\) to any via igb1 out proto tcp dst-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 > $IPFW add queue 123 ip from table\(11\) to any via igb1 out proto icmp > $IPFW add queue 124 ip from table\(11\) to any via igb1 out > $IPFW add queue 121 ip from table\(11\) to any via igb3 out proto udp dst-port 5060 > $IPFW add queue 122 ip from table\(11\) to any via igb3 out proto tcp dst-port 80,443,8080 > $IPFW add queue 123 ip from table\(11\) to any via igb3 out proto tcp dst-port 5223, 2009, 2106, 3724, 6112, 6881-6999, 7777, 27000-27050, 42292 > $IPFW add queue 123 ip from table\(11\) to any via igb3 out proto icmp > $IPFW add queue 124 ip from table\(11\) to any via igb3 out > > Firstly, we have been tested firewall by ourself. And we had no any bad results or any problems or maybe we have not seen them in our synthetic tests. After that we have started this firewall in production. A few months later we received a message from our subscriber with speed 1Mbit/s. He had a problems with online game (big answer delay from the server). We spent a lot of time to solve this problem. Finaly we solved it. The reason was in packet scheduling: > 1. we`ve tried to give to subscriber another channel (4Mbit/s) with packet scheduling - there are no such problems; > 2. we`ve tried to "turn off" the packet scheduling on 1Mbit channel - there are no such problems. > > The utilization of subscibers channel was always 0.4Mbit/s. But the traffic from this subscriber was go on under the packet scheduling rules. That`s very strange because of: > 1. net.inet.ip.dummynet.io_fast=1; > 2. subscribers channel utilization 0.4Mbit/s. > > As I know with this option, with this firewall config and with this channel utilization (0.4Mbit/s) traffic should bypass the pipe without packet scheduling. > > Why subscribers traffic with all these conditions doesn`t bypass through pipe without any delays? Why his traffic was on packet scheduling rules? > Thanks. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101011151609.GB41572>