From owner-freebsd-net@freebsd.org Tue Jan 3 17:46:37 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 289EFC9D98F for ; Tue, 3 Jan 2017 17:46:37 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFF2A1B11 for ; Tue, 3 Jan 2017 17:46:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cOTAJ-000B86-Sg; Tue, 03 Jan 2017 20:46:27 +0300 Date: Tue, 3 Jan 2017 20:46:27 +0300 From: Slawa Olhovchenkov To: Jordan Caraballo Cc: freebsd-net@freebsd.org Subject: Re: Disappointing packets-per-second performance results on a Dell,PE R530 Message-ID: <20170103174627.GW37118@zxy.spb.ru> References: <8f637e2e-cd59-dc65-8476-30989bea516b@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8f637e2e-cd59-dc65-8476-30989bea516b@gmail.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jan 2017 17:46:37 -0000 On Tue, Jan 03, 2017 at 12:35:42PM -0400, Jordan Caraballo wrote: > We recently tested a Dell R530 with a Chelsio T580 card, under FreeBSD 10.3, 11.0, -STABLE and -CURRENT, and Centos 7. > > Based on our research, including netmap-fwd and with the routing improvements project (https://wiki.freebsd.org/ProjectsRoutingProposal), > we hoped for packets-per-second (pps) in the 5+ million range, or even higher. > > Based on prior testing (http://marc.info/?t=140604252400002&r=1&w=2), we expected 3-4 Million to be easily obtainable. > > Unfortunately, our current results top out at no more than 1.5 M (64 bytes length packets) with FreeBSD, and > surprisingly around 3.2 M (128 bytes length packets) with Centos 7, and we are at a loss as to why. > > Server Description: > Dell PowerEdge R530 with 2 Intel(R) Xeon(R) E5­2695 CPU's, 18 cores per > cpu. Equipped with a Chelsio T-580-CR dual port in an 8x slot. > > ** Can this be a lack in support issue related to the R530's hardware? ** > > Any help appreciated! What hardware configuration? What BIOS setting? What loader.conf/sysctl.conf setting? What `vmstat -i`? What `top -PHS`? what ==== pmcstat -S CPU_CLK_UNHALTED_CORE -l 10 -O sample.out pmcstat -R sample.out -G out.txt pmcstat -c 0 -S CPU_CLK_UNHALTED_CORE -l 10 -O sample0.out pmcstat -R sample0.out -G out0.txt ====