From owner-freebsd-net@FreeBSD.ORG Sat Apr 27 05:53:52 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E9AA4820 for ; Sat, 27 Apr 2013 05:53:52 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 5E71E1F43 for ; Sat, 27 Apr 2013 05:53:51 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r3R5rneJ080911; Sat, 27 Apr 2013 09:53:49 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r3R5rn2q080910; Sat, 27 Apr 2013 09:53:49 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Sat, 27 Apr 2013 09:53:49 +0400 From: Gleb Smirnoff To: Olivier Cochard-Labb? Subject: Re: pf performance? Message-ID: <20130427055349.GW76816@glebius.int.ru> References: <5176E5C1.9090601@soe.ucsc.edu> <20130426134224.GV76816@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-net@freebsd.org" , Erich Weiler X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 05:53:53 -0000 On Fri, Apr 26, 2013 at 06:22:18PM +0200, Olivier Cochard-Labb? wrote: O> > In FreeBSD 10 pf is no longer under single lock. On your hardware, O> > I'd expect a measurable performance gain if you migrate to 10. O> O> Compairing 9.1 and current (249908) on my new test-server (HP ProLiant O> DL320 G5, dual-core Xeon 3050, dual Intel NIC). O> Like usual: one unidirectional flow of small packets, values in O> packet-per-seconds: O> O> x 9.1 O> + current O> N Min Max Median Avg Stddev O> x 5 379991 381508 381229 380892.6 667.69926 O> + 5 332833 335502 334726 334223.2 1142.8266 O> Difference at 95.0% confidence O> -46669.4 +/- 1364.98 O> -12.2526% +/- 0.358363% O> (Student's t, pooled s = 935.915) As I already mentioned this is expected and okay result. With an empty state table you've got a fast pf processing, threads do not spend a lot of time in pf, so probability of contention is low, even in case of single lock. Not speaking that you got only 2 cores. In the new pf in 10 we need to do two atomic operations per packet: read-lock the global pf rwlock, then acquire hash slot mutex. While in old pf we only acquired the single pf mutex. So in case when state table is 1 state it is expected that old pf can outperform new one, due to cheaper locking. Not speaking that probability of outperforming is the more the less cores you got. You got only 2. This is not the case the new pf was coded for. But the setup the Erich is running is the case. We probably can get more performance out of new pf simply converting the rwlock to rmlock, may be we will get these 12% in vacuous test back. But I'd like someone with decent hardware and traffic to test that first. I don't want to do this convertsion blindly w/o benchmark and stability test. Unfortunately, as you see, most people avoid running head, waiting at least for 10.0-RELEASE, or even for pfSense catching up on FreeBSD 10. So probably this change won't be tested soon, and thus won't happen soon, -- Totus tuus, Glebius.