From owner-freebsd-net@freebsd.org Wed Mar 1 14:44:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64B4BCF0572 for ; Wed, 1 Mar 2017 14:44:28 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from smtp.smtpout.orange.fr (smtp05.smtpout.orange.fr [80.12.242.127]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client CN "Bizanga Labs SMTP Client Certificate", Issuer "Bizanga Labs CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 93E4D403 for ; Wed, 1 Mar 2017 14:44:26 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from mail-yw0-f171.google.com ([209.85.161.171]) by mwinf5d62 with ME id qekJ1u0053iAumh03ekJJU; Wed, 01 Mar 2017 15:44:19 +0100 X-ME-Helo: mail-yw0-f171.google.com X-ME-Auth: Y29jaGFyZC1sYWJiZS5vbGl2aWVyQG9yYW5nZS5mcg== X-ME-Date: Wed, 01 Mar 2017 15:44:19 +0100 X-ME-IP: 209.85.161.171 Received: by mail-yw0-f171.google.com with SMTP id p77so32847541ywg.1 for ; Wed, 01 Mar 2017 06:44:18 -0800 (PST) X-Gm-Message-State: AMke39lie3XwLbRMU4bU9fZHWbFNJxxOcDaAwFGavHaJgv3NpPicuT0aExsUU/+yK3qeXH4WTEpQZ9EdxMY/IQ== X-Received: by 10.129.168.193 with SMTP id f184mr2907855ywh.278.1488379457606; Wed, 01 Mar 2017 06:44:17 -0800 (PST) MIME-Version: 1.0 Received: by 10.37.198.205 with HTTP; Wed, 1 Mar 2017 06:43:57 -0800 (PST) In-Reply-To: <6ad029e0-86c6-af3d-8fc3-694d4bcdc683@gmail.com> References: <40a413f3-2c44-ee9d-9961-67114d8dffca@gmail.com> <20170205175531.GA20287@dwarf> <7d349edd-0c81-2e3f-d3b9-27af232de76d@gmail.com> <20170209153409.GG41673@dwarf> <6ad029e0-86c6-af3d-8fc3-694d4bcdc683@gmail.com> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Date: Wed, 1 Mar 2017 15:43:57 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Fwd: Re: Disappointing packets-per-second performance results on a Dell,PE R530 To: "Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP]" Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Mar 2017 14:44:28 -0000 On Tue, Feb 28, 2017 at 3:13 AM, Caraballo-vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP] wrote: > As a summarywe have a Dell R530 with a Chelsio T580 cardwith -CURRENT. > > In an attempt to reduce the time the system was taking to look for the > cpus; we changed the BIOS setting to let the system have 8 visible cores > and tested cxl* and vcxl* chelsio interfaces. Scores are still way lower > than what we expected: > > > Based on previous scores with Centos 7 (over 3M pps), we can assume that > it is not the hardware. However, we are still looking for a reason of > why are we getting these scores. > > Any feedback or suggestion would be highly appreciated. > > =E2=80=8BDuring your bench load, can you display=E2=80=8B =E2=80=8Beach queue usage of your cxl interface and post the result here ? You can use this python script (from Alexander Chernikov): https://github.com/ocochard/BSDRP/blob/master/BSDRP/Files/usr/local/bin/nic= -queue-usage Here is an example showing traffic shared between all 8 queues: [root@hp]~# nic-queue-usage cxl0 [Q0 346K/s] [Q1 343K/s] [Q2 339K/s] [Q3 338K/s] [Q4 338K/s] [Q5 338K/s] [Q6 343K/s] [Q7 346K/s] [QT 2734K/s 3269K/s -> 0K/s] [Q0 347K/s] [Q1 344K/s] [Q2 339K/s] [Q3 339K/s] [Q4 338K/s] [Q5 338K/s] [Q6 343K/s] [Q7 346K/s] [QT 2735K/s 3277K/s -> 0K/s] [Q0 344K/s] [Q1 341K/s] [Q2 338K/s] [Q3 338K/s] [Q4 337K/s] [Q5 337K/s] [Q6 342K/s] [Q7 345K/s] [QT 2727K/s 3262K/s -> 0K/s] [Q0 355K/s] [Q1 352K/s] [Q2 348K/s] [Q3 349K/s] [Q4 348K/s] [Q5 347K/s] [Q6 352K/s] [Q7 355K/s] [QT 2809K/s 3381K/s -> 0K/s] [Q0 351K/s] [Q1 348K/s] [Q2 344K/s] [Q3 343K/s] [Q4 342K/s] [Q5 344K/s] [Q6 349K/s] [Q7 352K/s] [QT 2776K/s 3288K/s -> 0K/s] [Q0 344K/s] [Q1 341K/s] [Q2 338K/s] [Q3 339K/s] [Q4 338K/s] [Q5 338K/s] [Q6 343K/s] [Q7 346K/s] [QT 2731K/s 3261K/s -> 0K/s] Thanks