From owner-freebsd-net@FreeBSD.ORG Mon Apr 29 06:40:59 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A646E9E6 for ; Mon, 29 Apr 2013 06:40:59 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 35248193A for ; Mon, 29 Apr 2013 06:40:58 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3T6euRI006224; Mon, 29 Apr 2013 10:40:56 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3T6etTg006223; Mon, 29 Apr 2013 10:40:55 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 29 Apr 2013 10:40:55 +0400 From: Gleb Smirnoff To: Kajetan Staszkiewicz Subject: Re: pf performance? Message-ID: <20130429064055.GH76816@FreeBSD.org> References: <5176E5C1.9090601@soe.ucsc.edu> <517974DA.5090809@soe.ucsc.edu> <201304260021.11209.vegeta@tuxpowered.net> <201304262150.17215.vegeta@tuxpowered.net> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <201304262150.17215.vegeta@tuxpowered.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org, Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 06:40:59 -0000 Kajetan, On Fri, Apr 26, 2013 at 09:50:17PM +0200, Kajetan Staszkiewicz wrote: K> > > > How do you count the 140kpps value? One interface, both, in, out? I'd K> > > > like to relate this somehow to my values. K> > > K> > > Well, generally we see 80kpps rx and 40kpps tx. But I have seen the rx K> > > spike to 150kpps occasionally. K> > K> > Unfortunately at this moment I have no single machine with such traffic, K> > although maybe I can aggregate some traffic later and check the cpu usage K> > then. K> K> OK, got my CPU usage for 30/40kpps, 55/340Mbit/s (public side, in/out). K> K> CPU is 4-core E5540 @ 2.53GHz with HT disabled. K> K> Cpu usage looks more or less like this: K> CPU 0: 0.0% user, 0.0% nice, 0.4% system, 3.9% interrupt, 95.7% idle K> CPU 1: 0.0% user, 0.0% nice, 1.2% system, 18.1% interrupt, 80.7% idle K> CPU 2: 0.0% user, 0.0% nice, 1.6% system, 5.5% interrupt, 92.9% idle K> CPU 3: 0.0% user, 0.0% nice, 0.8% system, 26.8% interrupt, 72.4% idle K> K> Public network card is pinned to cpu 2, internal to cpu 3, each card has only a K> single irq. Netisr threads are limited to cpu 0 and 1, I use deferred netisr. K> K> So yes, I have 2x less pps than you, but also I have quite a slower cpu and K> there still seems to be much cpu power left. When it comes to contention, the dependence of CPU utilization from amount of data processed isn't linear. While you have low enough pps, a time spent by a CPU in pf is accounted as time of that CPU, and that's all. But if pps are higher, then while that CPU was working in pf, couple more CPUs were spinning on pf lock, and now their CPU time is also utilized and accounted (although they did nothing). -- Totus tuus, Glebius.