From owner-freebsd-ipfw@FreeBSD.ORG Mon Oct 19 16:30:05 2009 Return-Path: Delivered-To: freebsd-ipfw@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9FAA106568F for ; Mon, 19 Oct 2009 16:30:05 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BEC408FC13 for ; Mon, 19 Oct 2009 16:30:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n9JGU5dn042782 for ; Mon, 19 Oct 2009 16:30:05 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n9JGU5cc042779; Mon, 19 Oct 2009 16:30:05 GMT (envelope-from gnats) Date: Mon, 19 Oct 2009 16:30:05 GMT Message-Id: <200910191630.n9JGU5cc042779@freefall.freebsd.org> To: freebsd-ipfw@FreeBSD.org From: alexus Cc: Subject: Re: kern/139581: [ipfw] "ipfw pipe" not limiting bandwidth X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alexus List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2009 16:30:06 -0000 The following reply was made to PR kern/139581; it has been noted by GNATS. From: alexus To: Ian Smith Cc: bug-followup@FreeBSD.org, freebsd@alexus.org Subject: Re: kern/139581: [ipfw] "ipfw pipe" not limiting bandwidth Date: Mon, 19 Oct 2009 11:58:41 -0400 On Oct 19, 2009, at 10:24 AM, Ian Smith wrote: > May be a usage issue; I'll have a go. Partial quoting, out of order. > > : I'm trying to limit my apache that runs under daemon to up 2Mbit/s > : when I do "ipfw pipe show" I don't see anything in my slots other > then > : very first entry that never chage, nor does it limits my traffic, as > : if I look at my MRTG i see way more traffic then 2Mbit/s > > Unless you specify masks on your pipes you'll only ever see the first > connection that used that pipe, that's normal. ok new set of rules su-3.2# cat /etc/ipfw.rules flush pipe flush pipe 1 config bw 1Mbit/s mask src-port www pipe 2 config bw 1Mbit/s mask src-port www add 100 allow ip from any to any via lo0 add 200 deny ip from any to 127.0.0.0/8 add 300 deny ip from 127.0.0.0/8 to any add 8381 pipe 1 tcp from any to any dst-port www uid daemon add 8382 pipe 2 tcp from any to any src-port www uid daemon add 65000 pass all from any to any su-3.2# ipfw show 00100 1476 230632 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 08381 482 36368 pipe 1 tcp from any to any dst-port 80 uid daemon 08382 620 743113 pipe 2 tcp from any 80 to any uid daemon 65000 6832 5040856 allow ip from any to any 65535 0 0 deny ip from any to any su-3.2# ipfw pipe show 00001: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/ Byte Drp 0 tcp 64.237.55.83/49492 66.230.133.69/80 509 38156 0 0 0 00002: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/ Byte Drp 0 tcp 66.230.133.69/80 64.237.55.83/49492 656 785292 1 1500 1 su-3.2# ipfw pipe show 00001: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/ Byte Drp 0 tcp 64.237.55.83/49492 66.230.133.69/80 1247 98023 0 0 0 00002: 1.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/ Byte Drp 0 tcp 66.230.133.69/80 64.237.55.83/49492 1475 1453606 0 0 1 su-3.2# in this case i did specify mask for pipe, yet when I'm issuing ipfw pipe show I still don't see anything in terms of slots that being in use su-3.2# sysctl net.inet.ip.dummynet.pipe_slot_limit net.inet.ip.dummynet.pipe_slot_limit: 100 su-3.2# seems like at all time I see only 1 slot being utilized and as I mention before it never changes. > > MRTG sees all traffic on an interface, and your ipfw stats indicate at > least 25% more traffic than that due to your webserver, so it's not > clear how you could tell if your pipe was exceeding 2Mbit/s or not? > I obviously do have other traffic then www, but majority of it is www. but I see why you coming with this, so let me just give you an example. if I at peak time shutdown my apache, my traffic drops dramatically and by dramatically i mean at least 90% (and in most cases more) my traffic went to as much as 10mbps with supposedly limited pipe of 2mbps, when I set it to 1mbps it seems to be almost there... > Also, it's recommended not to run your inbound and outbound traffic > through the one pipe, unless simulating half-duplex connections; see > explanation in ipfw(8), EXAMPLES section under TRAFFIC SHAPING. i thought about that and as you suggested i did separate them into 2 separate pipes (see on top) > > : su-3.2# ipfw show > : 00100 1249368 205115325 allow ip from any to any via lo0 > : 00200 0 0 deny ip from any to 127.0.0.0/8 > : 00300 0 0 deny ip from 127.0.0.0/8 to any > : 08380 2838075 3586421013 pipe 1 tcp from any 80 to any uid daemon > : 08380 2097473 136454502 pipe 1 tcp from any to any dst-port 80 uid > daemon > : 65000 5740679 4716157064 allow ip from any to any > : 65535 0 0 deny ip from any to any > > 3.586 GiB outbound from the webserver (served data) > 0.136 GiB inbound to the webserver (requests, acks) > + --- > 3.722 GiB through the pipe. > but > 4.716 GiB passed from any to any, either way. > > So there's about 1 Gig of extra traffic shown here, assuming you have > net.inet.ip.fw.one_pass=0 and all traffic eventually hits rule 65000 > (and 4.7G extra traffic if net.inet.ip.fw.one_pass=1) but there's not > enough info to see whether or not it's on the interface MRTG watches? > > : su-3.2# ipfw pipe show > : 00001: 2.000 Mbit/s 0 ms 50 sl. 1 queues (1 buckets) droptail > : mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 > : BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes > Pkt/Byte Drp > : 0 tcp 64.237.55.83/59388 208.80.152.3/80 4936077 3723134341 0 0 > 30179 > > Total packets and bytes match the above, indicating that this was done > just after the ipfw show. 0.6% dropped packets indicates some > limiting > happening, but with a shared in/outbound pipe, not in which direction. > > If this is still an issue, please: > > . be more precise than "way more traffic" if you have more data? > . say whether the extra ~25% traffic shown is on the same interface > as the webserver, ie the interface MRTG monitors, or not? > . the value of sysctl net.inet.ip.fw.one_pass ? > > cheers, Ian >