From owner-freebsd-net@FreeBSD.ORG Wed Feb 15 20:19:40 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E6ECD16A423 for ; Wed, 15 Feb 2006 20:19:40 +0000 (GMT) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF2C043D45 for ; Wed, 15 Feb 2006 20:19:39 +0000 (GMT) (envelope-from andre@freebsd.org) Received: (qmail 81394 invoked from network); 15 Feb 2006 20:15:52 -0000 Received: from c00l3r.networx.ch (HELO freebsd.org) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 15 Feb 2006 20:15:52 -0000 Message-ID: <43F38CF5.71C326C1@freebsd.org> Date: Wed, 15 Feb 2006 21:20:05 +0100 From: Andre Oppermann X-Mailer: Mozilla 4.8 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Max Laier References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <711b7ec873f31bc5be50ce477313fac3@madhaus.cns.utoronto.ca> <43ECEF7C.2090101@elischer.org> <200602110002.21275.max@love2party.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Marcos Bedinelli , freebsd-net@freebsd.org, Julian Elischer Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Feb 2006 20:19:41 -0000 Max Laier wrote: > > On Friday 10 February 2006 20:54, Julian Elischer wrote: > > Marcos Bedinelli wrote: > > > Hello all, > > > > > > thanks for the replies. Most of you have suggested that I turn on > > > polling and give it a try. The machine is in production, hence I need > > > to schedule downtime for that. > > > > > > The system is mainly being used as a dedicated router. It runs OSPF, > > > BGP and IPFW (around 150 rules). OSPF and BGP are managed by Quagga. > > > The box has 2 gigabit interfaces that handle on average 200Mbp/s - 50K > > > packets/s (inbound and outbound combined), each one of them. > > > > I have found that most people can optimise there ipfw rulests considerably. > > > > for example: a first rule of: > > 1 allow ip from any to any in recv {inside interfacfe} > > 2 allow ip from any to any out xmit {inside interface} > > will cut your ipfw load by 50% immediatly. > > (you should only be filterring on one interface usually) > > > > use 'skipto' rules to immediatly send incoming and outgoing data to > > different rules sets. > > FWIW, pf does some of those optimizations automatically called "skip steps" > and "pfctl -o" restructures the ruleset so that often matching rules are > moved to the top. I know that this does not map directly to IPFW, but it > might still be interesting to have a look at it. >From my profiling with the Agilent tester there seem to be two areas where the packet filters (ipfw in my test case) burn a lot of CPU per packet. That is a) setup of lots of packet variables unconditionally at the entry of ip_fw_chk() no matter whether they get looked at later or not, and b) the switch() going through all the packet inspection options is for some reason not optimized by the compiler and burns even more CPU. Some sort of JIT (as in the new bpf code) which replaces the case testing and jumps directly to the proper place in the switch statement would go a long way of making it way more performant. -- Andre