From owner-freebsd-ipfw@FreeBSD.ORG Sat Jun 9 20:07:10 2012 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C6461106566C for ; Sat, 9 Jun 2012 20:07:10 +0000 (UTC) (envelope-from mike@magicislandtechnologies.com) Received: from mail.magicislandtechnologies.com (mail.magicislandtechnologies.com [74.208.96.3]) by mx1.freebsd.org (Postfix) with ESMTP id 7F0728FC17 for ; Sat, 9 Jun 2012 20:07:10 +0000 (UTC) Received: (qmail 10442 invoked from network); 10 Jun 2012 04:11:12 +0400 Received: from adsl-99-121-29-49.dsl.sfldmi.sbcglobal.net (HELO ?99.121.29.49?) (99.121.29.49) by mail.magicislandtechnologies.com with (DHE-RSA-AES256-SHA encrypted) SMTP; 10 Jun 2012 04:11:12 +0400 Message-ID: <4FD3B05E.3050006@magicislandtechnologies.com> Date: Sat, 09 Jun 2012 16:21:50 -0400 From: Michael Spratt User-Agent: Thunderbird 2.0.0.22 (X11/20090605) MIME-Version: 1.0 To: "Alexander V. Chernikov" References: <4FD3224A.3080700@FreeBSD.org> In-Reply-To: <4FD3224A.3080700@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Sami Halabi , freebsd-ipfw@freebsd.org Subject: Re: ipfw rules consuming CPU X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jun 2012 20:07:10 -0000 I have Linux & FreeBSD systems running ipfw with 80 rules with 70Mb/s symmetric, passing traffic for about 1000-1200 hosts. Alexander V. Chernikov wrote: > On 09.06.2012 01:56, Sami Halabi wrote: >> Hi, >> >> I Manage a FreeBSD server as an edge router& firewall. >> the setup has 10G interfaces (ixgbe-82599EB) and 1G >> interfaces(em-82571EB& >> bce-BCM5709) connected to 10G/1G switches. >> >> With the following setup i get higher cpu usage: >> bce1-upstream provider with little bandwidth, so i use pipes to limit >> users, and subnets >> ix0 - Internet Exchange >> >> some rules. >> . >> . >> .from 4000 starts pipes for specefic ips bandwidth allocations >> 04000 6210053001 5845967300616 pipe 1003 ip from 182.46.92.13 >> to any >> out xmit bce1 >> 04100 41289897537 3064110648124 pipe 1004 ip from any to >> 182.46.92.13 >> in recv bce1 > You should use pipe tablearg for that. Traversing 4k rules effectively > kills all performance. > >> . >> . >> . >> .7000 is the wider pipeline for the whole block >> 07000 9127154724 4651308720315 pipe 1000 ip from >> 182.46.92.0/24 to >> any out xmit bce1 >> 07100 4837016828 458027989917 pipe 1002 ip from any to >> 182.46.92.0/24 in recv bce1 >> last rule default to accept... >> >> specefic pipes (1003-...) have limits say between 1-10Mbps, and the >> wider >> pipe (1000 and 1002) has a global limit of 40MBps that should be >> reached by >> all other non-specefic ips, config like this: >> #Wide >> ipfw pipe 1000 config bw 40Mbit/s queue 200Kbytes >> ipfw pipe 1002 config bw 40Mbit/s queue 200Kbytes >> #specefic >> ipfw pipe 1003 config bw 9Mbit/s queue 200Kbytes >> ipfw pipe 1004 config bw 9Mbit/s queue 200Kbytes >> ipfw pipe 1005 config bw 3Mbit/s queue 200Kbytes >> ipfw pipe 1006 config bw 3Mbit/s queue 200Kbytes >> ipfw pipe 1007 config bw 5Mbit/s queue 200Kbytes >> ipfw pipe 1008 config bw 5Mbit/s queue 200Kbytes >> ipfw pipe 1009 config bw 10Mbit/s queue 200Kbytes >> ipfw pipe 1010 config bw 10Mbit/s queue 200Kbytes >> >> >> with this configuration when i have lots of traffic (3-6GB) going via >> ix0 >> (not necessarly the ips described above, lets say to a server in my >> net ip >> 1832.46.93.4 and users behind the Internet Exchange) i see high cpu >> usage >> (70-90%). >> >> my first test was to: ipfw add 1 allow all from any to any, and cpu >> usage >> drops immediatly to 10-15%. >> but that not why i want (i wantto keep thelimits) so I add rule right >> before 4000 and the cpu usage drops down to 10-20%: >> 03020 1669463072808 1493341413029803 allow ip from any to any via ix0 >> >> >> Any advice why this happens? or should it be there in the first place? >> I use FreeBSD 8.1-R-p10-amd64. >> >> Thanks in advance, >> > >