From owner-freebsd-net@FreeBSD.ORG Thu Mar 4 18:12:15 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 506FD106564A for ; Thu, 4 Mar 2010 18:12:15 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outv.internet-mail-service.net [216.240.47.245]) by mx1.freebsd.org (Postfix) with ESMTP id 340318FC0C for ; Thu, 4 Mar 2010 18:12:14 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o24ICD46010759 for ; Thu, 4 Mar 2010 10:12:13 -0800 X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id CE31B2D601D for ; Thu, 4 Mar 2010 10:12:12 -0800 (PST) Message-ID: <4B8FF7FB.20208@elischer.org> Date: Thu, 04 Mar 2010 10:12:11 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: freebsd-net@freebsd.org References: <20100304124901.GA98085@traktor.dnepro.net> In-Reply-To: <20100304124901.GA98085@traktor.dnepro.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Subject: Re: Netgraph performance with ng_ipfw X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Mar 2010 18:12:15 -0000 Eugene Perevyazko wrote: > On Sat, Jan 23, 2010 at 03:32:11AM +0600, Евгений wrote: >> Hi, >> I have several routers under heavy load, running FreeBSD 7.2 >> These routers use Netgraph to impelement traffic shaping and accouting >> (using ng_car and ng_netflow nodes). > [skipped] >> The second question is about the effectiveness of Netgraph queueing and ng_ipfw node with SMP kernel... > [skipped] >> one of the ng_queue* processes will take 100% time of one CPU core, when the others will not >> process anything. >> I have seen such behavior on my routers - at peak load, one of ng_queue* processes takes 100% of one core, >> and the other processes are seen in top taking 0% of CPU. >> My question is, can it somehow be fixed? > > I see the same behaviour with ng_nat and ng_ipfw. > 7-S as of Nov 2009 on Xeon E5410 (4 cores). > 16 ng_nat nodes connected to ng_ipfw. > Sometimes one ng_queue eats 100% of one core while other 3 ng_queues sit idle. > I'm going to set queueing on all ng_nat's hooks to see if it will help. an interesting idea. I'm not familiar with the ng_queue & ng_ipfw nodes but scheduling work between threads in netgraph has always been a tricky area and is probably in need to work at the moment.