Date: Mon, 19 Oct 2009 16:30:05 GMT From: alexus <alexus@alexus.org> To: freebsd-ipfw@FreeBSD.org Subject: Re: kern/139581: [ipfw] "ipfw pipe" not limiting bandwidth Message-ID: <200910191630.n9JGU5cc042779@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/139581; it has been noted by GNATS.
From: alexus <alexus@alexus.org>
To: Ian Smith <smithi@nimnet.asn.au>
Cc: bug-followup@FreeBSD.org,
freebsd@alexus.org
Subject: Re: kern/139581: [ipfw] "ipfw pipe" not limiting bandwidth
Date: Mon, 19 Oct 2009 11:58:41 -0400
On Oct 19, 2009, at 10:24 AM, Ian Smith wrote:
> May be a usage issue; I'll have a go. Partial quoting, out of order.
>
> : I'm trying to limit my apache that runs under daemon to up 2Mbit/s
> : when I do "ipfw pipe show" I don't see anything in my slots other
> then
> : very first entry that never chage, nor does it limits my traffic, as
> : if I look at my MRTG i see way more traffic then 2Mbit/s
>
> Unless you specify masks on your pipes you'll only ever see the first
> connection that used that pipe, that's normal.
ok
new set of rules
su-3.2# cat /etc/ipfw.rules
flush
pipe flush
pipe 1 config bw 1Mbit/s mask src-port www
pipe 2 config bw 1Mbit/s mask src-port www
add 100 allow ip from any to any via lo0
add 200 deny ip from any to 127.0.0.0/8
add 300 deny ip from 127.0.0.0/8 to any
add 8381 pipe 1 tcp from any to any dst-port www uid daemon
add 8382 pipe 2 tcp from any to any src-port www uid daemon
add 65000 pass all from any to any
su-3.2# ipfw show
00100 1476 230632 allow ip from any to any via lo0
00200 0 0 deny ip from any to 127.0.0.0/8
00300 0 0 deny ip from 127.0.0.0/8 to any
08381 482 36368 pipe 1 tcp from any to any dst-port 80 uid daemon
08382 620 743113 pipe 2 tcp from any 80 to any uid daemon
65000 6832 5040856 allow ip from any to any
65535 0 0 deny ip from any to any
su-3.2# ipfw pipe show
00001: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/
Byte Drp
0 tcp 64.237.55.83/49492 66.230.133.69/80 509 38156
0 0 0
00002: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/
Byte Drp
0 tcp 66.230.133.69/80 64.237.55.83/49492 656 785292 1
1500 1
su-3.2# ipfw pipe show
00001: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/
Byte Drp
0 tcp 64.237.55.83/49492 66.230.133.69/80 1247 98023
0 0 0
00002: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/
Byte Drp
0 tcp 66.230.133.69/80 64.237.55.83/49492 1475 1453606
0 0 1
su-3.2#
in this case i did specify mask for pipe, yet when I'm issuing ipfw
pipe show I still don't see anything in terms of slots that being in use
su-3.2# sysctl net.inet.ip.dummynet.pipe_slot_limit
net.inet.ip.dummynet.pipe_slot_limit: 100
su-3.2#
seems like at all time I see only 1 slot being utilized and as I
mention before it never changes.
>
> MRTG sees all traffic on an interface, and your ipfw stats indicate at
> least 25% more traffic than that due to your webserver, so it's not
> clear how you could tell if your pipe was exceeding 2Mbit/s or not?
>
I obviously do have other traffic then www, but majority of it is www.
but I see why you coming with this, so let me just give you an example.
if I at peak time shutdown my apache, my traffic drops dramatically
and by dramatically i mean at least 90% (and in most cases more)
my traffic went to as much as 10mbps with supposedly limited pipe of
2mbps, when I set it to 1mbps it seems to be almost there...
> Also, it's recommended not to run your inbound and outbound traffic
> through the one pipe, unless simulating half-duplex connections; see
> explanation in ipfw(8), EXAMPLES section under TRAFFIC SHAPING.
i thought about that and as you suggested i did separate them into 2
separate pipes (see on top)
>
> : su-3.2# ipfw show
> : 00100 1249368 205115325 allow ip from any to any via lo0
> : 00200 0 0 deny ip from any to 127.0.0.0/8
> : 00300 0 0 deny ip from 127.0.0.0/8 to any
> : 08380 2838075 3586421013 pipe 1 tcp from any 80 to any uid daemon
> : 08380 2097473 136454502 pipe 1 tcp from any to any dst-port 80 uid
> daemon
> : 65000 5740679 4716157064 allow ip from any to any
> : 65535 0 0 deny ip from any to any
>
> 3.586 GiB outbound from the webserver (served data)
> 0.136 GiB inbound to the webserver (requests, acks)
> + ---
> 3.722 GiB through the pipe.
> but
> 4.716 GiB passed from any to any, either way.
>
> So there's about 1 Gig of extra traffic shown here, assuming you have
> net.inet.ip.fw.one_pass=0 and all traffic eventually hits rule 65000
> (and 4.7G extra traffic if net.inet.ip.fw.one_pass=1) but there's not
> enough info to see whether or not it's on the interface MRTG watches?
>
> : su-3.2# ipfw pipe show
> : 00001: 2.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail
> : mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
> : BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes
> Pkt/Byte Drp
> : 0 tcp 64.237.55.83/59388 208.80.152.3/80 4936077 3723134341 0 0
> 30179
>
> Total packets and bytes match the above, indicating that this was done
> just after the ipfw show. 0.6% dropped packets indicates some
> limiting
> happening, but with a shared in/outbound pipe, not in which direction.
>
> If this is still an issue, please:
>
> . be more precise than "way more traffic" if you have more data?
> . say whether the extra ~25% traffic shown is on the same interface
> as the webserver, ie the interface MRTG monitors, or not?
> . the value of sysctl net.inet.ip.fw.one_pass ?
>
> cheers, Ian
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200910191630.n9JGU5cc042779>