From owner-freebsd-net@FreeBSD.ORG Sat Jun 9 20:19:11 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C6DA5106564A; Sat, 9 Jun 2012 20:19:11 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6E0A38FC1E; Sat, 9 Jun 2012 20:19:11 +0000 (UTC) Received: by obcni5 with SMTP id ni5so5571284obc.13 for ; Sat, 09 Jun 2012 13:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FP48LHnFG2rGApaMwpUftineE5NSzHabfW2hXfQDx2E=; b=dtGjDqsL7YQAmBV0Tyl/0QcmJUxokHecL1CIuHZQYf9cmo0Uc0F8zhOb/tfSzCM21A SA8C7U/xnTY3Id8/cmeWnp9hZ28SpQlpX/Y9fY8w55+vgXMQseHLKrmjEZOjOdywz3W0 FM7TgHrOHTgPd0Rx0iRang6D69wYExlh3SBsjzCtYzlZPS+8tqowNuxO8+X+NnYBlxYP oQXP/Em7PQMv7qrbpKIRnEMBQzsobu+0q+dWaMmMWhFRIJNsvGlGidt3GCdUhb+5Te4b Z4onDd8pAIbZa9SoVGLDNsgfeyD3lKANA43qcfLs5c444VDmwnRx6gIgGw+h6BmDO8YX C6Fg== MIME-Version: 1.0 Received: by 10.182.36.102 with SMTP id p6mr11332923obj.77.1339273150978; Sat, 09 Jun 2012 13:19:10 -0700 (PDT) Received: by 10.182.44.101 with HTTP; Sat, 9 Jun 2012 13:19:10 -0700 (PDT) In-Reply-To: <4FD3B05E.3050006@magicislandtechnologies.com> References: <4FD3224A.3080700@FreeBSD.org> <4FD3B05E.3050006@magicislandtechnologies.com> Date: Sat, 9 Jun 2012 23:19:10 +0300 Message-ID: From: Sami Halabi To: Michael Spratt Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, "Alexander V. Chernikov" , freebsd-ipfw@freebsd.org Subject: Re: ipfw rules consuming CPU X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jun 2012 20:19:11 -0000 on my box with 130 rules 100Mbit the cpu don't go above 5%. I daily manage 1.5-6GB. Thanks in advance, Sami On Sat, Jun 9, 2012 at 11:21 PM, Michael Spratt < mike@magicislandtechnologies.com> wrote: > I have Linux & FreeBSD systems running ipfw with 80 rules with 70Mb/s > symmetric, passing traffic for about 1000-1200 hosts. > > > Alexander V. Chernikov wrote: > >> On 09.06.2012 01:56, Sami Halabi wrote: >> >>> Hi, >>> >>> I Manage a FreeBSD server as an edge router& firewall. >>> the setup has 10G interfaces (ixgbe-82599EB) and 1G >>> interfaces(em-82571EB& >>> bce-BCM5709) connected to 10G/1G switches. >>> >>> With the following setup i get higher cpu usage: >>> bce1-upstream provider with little bandwidth, so i use pipes to limit >>> users, and subnets >>> ix0 - Internet Exchange >>> >>> some rules. >>> . >>> . >>> .from 4000 starts pipes for specefic ips bandwidth allocations >>> 04000 6210053001 5845967300616 pipe 1003 ip from 182.46.92.13 to >>> any >>> out xmit bce1 >>> 04100 41289897537 3064110648124 pipe 1004 ip from any to >>> 182.46.92.13 >>> in recv bce1 >>> >> You should use pipe tablearg for that. Traversing 4k rules effectively >> kills all performance. >> >> . >>> . >>> . >>> .7000 is the wider pipeline for the whole block >>> 07000 9127154724 4651308720315 pipe 1000 ip from 182.46.92.0/24to >>> any out xmit bce1 >>> 07100 4837016828 458027989917 pipe 1002 ip from any to >>> 182.46.92.0/24 in recv bce1 >>> last rule default to accept... >>> >>> specefic pipes (1003-...) have limits say between 1-10Mbps, and the wider >>> pipe (1000 and 1002) has a global limit of 40MBps that should be reached >>> by >>> all other non-specefic ips, config like this: >>> #Wide >>> ipfw pipe 1000 config bw 40Mbit/s queue 200Kbytes >>> ipfw pipe 1002 config bw 40Mbit/s queue 200Kbytes >>> #specefic >>> ipfw pipe 1003 config bw 9Mbit/s queue 200Kbytes >>> ipfw pipe 1004 config bw 9Mbit/s queue 200Kbytes >>> ipfw pipe 1005 config bw 3Mbit/s queue 200Kbytes >>> ipfw pipe 1006 config bw 3Mbit/s queue 200Kbytes >>> ipfw pipe 1007 config bw 5Mbit/s queue 200Kbytes >>> ipfw pipe 1008 config bw 5Mbit/s queue 200Kbytes >>> ipfw pipe 1009 config bw 10Mbit/s queue 200Kbytes >>> ipfw pipe 1010 config bw 10Mbit/s queue 200Kbytes >>> >>> >>> with this configuration when i have lots of traffic (3-6GB) going via ix0 >>> (not necessarly the ips described above, lets say to a server in my net >>> ip >>> 1832.46.93.4 and users behind the Internet Exchange) i see high cpu usage >>> (70-90%). >>> >>> my first test was to: ipfw add 1 allow all from any to any, and cpu usage >>> drops immediatly to 10-15%. >>> but that not why i want (i wantto keep thelimits) so I add rule right >>> before 4000 and the cpu usage drops down to 10-20%: >>> 03020 1669463072808 1493341413029803 allow ip from any to any via ix0 >>> >>> >>> Any advice why this happens? or should it be there in the first place? >>> I use FreeBSD 8.1-R-p10-amd64. >>> >>> Thanks in advance, >>> >>> >> >> > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert