From owner-freebsd-net@freebsd.org Tue Jan 24 15:28:31 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7585CBFCF2 for ; Tue, 24 Jan 2017 15:28:31 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from smtp.smtpout.orange.fr (smtp01.smtpout.orange.fr [80.12.242.123]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client CN "Bizanga Labs SMTP Client Certificate", Issuer "Bizanga Labs CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 07712C7B for ; Tue, 24 Jan 2017 15:28:30 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from mail-yw0-f179.google.com ([209.85.161.179]) by mwinf5d02 with ME id cFUJ1u00Y3sYHUp03FUKdP; Tue, 24 Jan 2017 16:28:22 +0100 X-ME-Helo: mail-yw0-f179.google.com X-ME-Auth: Y29jaGFyZC1sYWJiZS5vbGl2aWVyQG9yYW5nZS5mcg== X-ME-Date: Tue, 24 Jan 2017 16:28:22 +0100 X-ME-IP: 209.85.161.179 Received: by mail-yw0-f179.google.com with SMTP id l19so163690682ywc.2; Tue, 24 Jan 2017 07:28:19 -0800 (PST) X-Gm-Message-State: AIkVDXJR9cdwykkWvz1NDgR6aQWjYTiV1swNTSmqJ141qCYh0CBTY1l/uGqMlrbTtdPY/MylC6eCzzXAdgR+gg== X-Received: by 10.55.151.199 with SMTP id z190mr31704288qkd.166.1485271697930; Tue, 24 Jan 2017 07:28:17 -0800 (PST) MIME-Version: 1.0 Received: by 10.200.49.99 with HTTP; Tue, 24 Jan 2017 07:27:57 -0800 (PST) In-Reply-To: References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org> <1598d97bf2a.c6bcb76838987.6501340920645175463@nextbsd.org> <574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org> <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> <1598f29d379.ea6360351471.8752933472741761813@nextbsd.org> <1598f3f8588.d20017893749.339651164872952258@nextbsd.org> <1598f42ad77.eeec05be4113.9201780237587761460@nextbsd.org> <159902b73ed.10775291e21533.7488368455500235608@nextbsd.org> <18abdd64-08a6-50ca-fb6b-9c01a3d7b60c@freebsd.org> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Date: Tue, 24 Jan 2017 16:27:57 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending To: Sean Bruno Cc: Matthew Macy , "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2017 15:28:31 -0000 On Tue, Jan 24, 2017 at 3:17 PM, Sean Bruno wrote: > > > Did you increase the number of rx/tx rings to 8 and the number of > descriptors to 4k in your tests or just the defaults? > Tuning are same as described in my previous email (rxd|txd=2048, rx|tx process_limit=-1, max_interrupt_rate=16000). [root@apu2]~# sysctl hw.igb. hw.igb.tx_process_limit: -1 hw.igb.rx_process_limit: -1 hw.igb.num_queues: 0 hw.igb.header_split: 0 hw.igb.max_interrupt_rate: 16000 hw.igb.enable_msix: 1 hw.igb.enable_aim: 1 hw.igb.txd: 2048 hw.igb.rxd: 2048 But I've did a new benchs with default setting, and the performance drop is now about -25% : x head r311848 packets-per-second (default settings) + head r311849 packets-per-second (default settings) +--------------------------------------------------------------------------+ |+ | |+ x | |+ xx| |++ xx| | A|| |A| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 618711 621135 619930.5 619840.8 951.83787 + 5 467389 468740 467778 467864.8 550.40322 Difference at 95.0% confidence -151976 +/- 1133.9 -24.5186% +/- 0.150581% (Student's t, pooled s = 777.476)