From owner-freebsd-net@FreeBSD.ORG Fri Jun 8 21:56:55 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 588B5106564A; Fri, 8 Jun 2012 21:56:55 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 131098FC16; Fri, 8 Jun 2012 21:56:54 +0000 (UTC) Received: by obcni5 with SMTP id ni5so3822447obc.13 for ; Fri, 08 Jun 2012 14:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/GYoG3cBJzsXQGHWBVOeOqriBwfWsh4PpHLkrEx74+8=; b=Mk1LXAPmJzVq+MJ8HWFSffYG2FWPQbCL2Eft9LRaFDSmKdqlx/oBZhRgvmwNuLyfLj V3q3FpeZWUJ6R54yT0bZgLgfo6X10waLhXAfVZnu4veHLB0L50xeUC6rAShY+TheWsAS vFvcWSfhLpMhmjAAKVp6pdf2lBNeksEmpnuAH4efSs9ekA3AaQ4sy5Wiedq3HE3Hgc6A iCbiQqd8Tg/+zIhmdYZV4YRJijGVCgfVQpo2G7PVsfFWA6cXx0WuJPRg9Ha/1vX7CGrv gYjrKdx6lijwMVVOue8FeA0918wGR23kt+0sjuq0ar198QF4GsI1tJjcv9Kda01A/wFy YsbQ== MIME-Version: 1.0 Received: by 10.60.9.134 with SMTP id z6mr8672465oea.46.1339192614408; Fri, 08 Jun 2012 14:56:54 -0700 (PDT) Received: by 10.182.44.101 with HTTP; Fri, 8 Jun 2012 14:56:54 -0700 (PDT) Date: Sat, 9 Jun 2012 00:56:54 +0300 Message-ID: From: Sami Halabi To: freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: ipfw rules consuming CPU X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2012 21:56:55 -0000 Hi, I Manage a FreeBSD server as an edge router & firewall. the setup has 10G interfaces (ixgbe-82599EB) and 1G interfaces(em-82571EB & bce-BCM5709) connected to 10G/1G switches. With the following setup i get higher cpu usage: bce1-upstream provider with little bandwidth, so i use pipes to limit users, and subnets ix0 - Internet Exchange some rules. . . .from 4000 starts pipes for specefic ips bandwidth allocations 04000 6210053001 5845967300616 pipe 1003 ip from 182.46.92.13 to any out xmit bce1 04100 41289897537 3064110648124 pipe 1004 ip from any to 182.46.92.13 in recv bce1 . . . .7000 is the wider pipeline for the whole block 07000 9127154724 4651308720315 pipe 1000 ip from 182.46.92.0/24 to any out xmit bce1 07100 4837016828 458027989917 pipe 1002 ip from any to 182.46.92.0/24 in recv bce1 last rule default to accept... specefic pipes (1003-...) have limits say between 1-10Mbps, and the wider pipe (1000 and 1002) has a global limit of 40MBps that should be reached by all other non-specefic ips, config like this: #Wide ipfw pipe 1000 config bw 40Mbit/s queue 200Kbytes ipfw pipe 1002 config bw 40Mbit/s queue 200Kbytes #specefic ipfw pipe 1003 config bw 9Mbit/s queue 200Kbytes ipfw pipe 1004 config bw 9Mbit/s queue 200Kbytes ipfw pipe 1005 config bw 3Mbit/s queue 200Kbytes ipfw pipe 1006 config bw 3Mbit/s queue 200Kbytes ipfw pipe 1007 config bw 5Mbit/s queue 200Kbytes ipfw pipe 1008 config bw 5Mbit/s queue 200Kbytes ipfw pipe 1009 config bw 10Mbit/s queue 200Kbytes ipfw pipe 1010 config bw 10Mbit/s queue 200Kbytes with this configuration when i have lots of traffic (3-6GB) going via ix0 (not necessarly the ips described above, lets say to a server in my net ip 1832.46.93.4 and users behind the Internet Exchange) i see high cpu usage (70-90%). my first test was to: ipfw add 1 allow all from any to any, and cpu usage drops immediatly to 10-15%. but that not why i want (i wantto keep thelimits) so I add rule right before 4000 and the cpu usage drops down to 10-20%: 03020 1669463072808 1493341413029803 allow ip from any to any via ix0 Any advice why this happens? or should it be there in the first place? I use FreeBSD 8.1-R-p10-amd64. Thanks in advance, -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert