From owner-freebsd-net@FreeBSD.ORG Fri Jan 22 22:52:16 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86629106566B for ; Fri, 22 Jan 2010 22:52:16 +0000 (UTC) (envelope-from julian@elischer.org) Received: from utility-0.aerioconnect.net (utility-0.aerioconnect.net [216.240.32.11]) by mx1.freebsd.org (Postfix) with ESMTP id 6B1858FC16 for ; Fri, 22 Jan 2010 22:52:15 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by utility-0.aerioconnect.net (8.13.1/8.13.1) with ESMTP id o0MMqDeM030811; Fri, 22 Jan 2010 14:52:13 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 4EEB12D6015; Fri, 22 Jan 2010 14:52:13 -0800 (PST) Message-ID: <4B5A2C32.7050802@elischer.org> Date: Fri, 22 Jan 2010 14:52:34 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Luigi Rizzo References: <20100122224156.GB27338@onelab2.iet.unipi.it> In-Reply-To: <20100122224156.GB27338@onelab2.iet.unipi.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, ??????? Subject: Re: Netgraph performance with ng_ipfw X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 22:52:16 -0000 Luigi Rizzo wrote: > On Sat, Jan 23, 2010 at 03:32:11AM +0600, ??????? wrote: >> Hi, >> I have several routers under heavy load, running FreeBSD 7.2 >> These routers use Netgraph to impelement traffic shaping and accouting >> (using ng_car and ng_netflow nodes). >> Packets are passed from firewall to netgraph using the following rules >> accounting: >> netgraph 100 ip from any to any in >> shaping: >> netgraph tablearg ip from any to table(118) out >> netgraph tablearg ip from table(118) to any in >> >> Table 118 contains users' ip addresses with tablearg referencing configured individual ng_car node. >> At peak, there are 1500-2000 entries in table and configured nodes. >> The problem is that at peak load the router loses packets. After studying the sources & doing some debugging, >> it became clear that packets are being droped at netgraph queue, at ng_alloc_item function: > ... >> When the large number of hooks is present, as in the configuration given in the beginning of this message, >> this would cause an obvious decrease in performance - for each packet passed from ipfw to netgraph, >> 1 to 1500-2000 iterations are needed to find matching hook. And again, it seem to be a trivial task to rewrite >> this code to find hook by hash or even by array. Each node type has the ability to supply a "hook lookup method" if (node->nd_type->findhook != NULL) return (*node->nd_type->findhook)(node, name); I added this so that if a hook had thousands of hooks it could supply a smart method to look them up in a way that is relevant to that node type. I have never used the netgraph ipfw node so I don't know how it works or if this is at all relevant. > > i solved exactly this problem in my recent ipfw rewrite (in HEAD), but it > is not terribly trivial, though, because you should consider memory > costs (the cookie space is large) and the need to handle reconfigurations > without blocking the forwarding for a long time. > > You can use the same techniques (and possibly code) i used in ipfw, > but i suspect it will take a good 1-2 weeks of work to have a production > quality thing (I am not familiar with netgraph code and configuration > tools so i may underestimate the work). > > cheers > luigi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"