From owner-freebsd-net@freebsd.org Fri Jan 27 17:46:16 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 914CECBF6F4 for ; Fri, 27 Jan 2017 17:46:16 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 631276E4; Fri, 27 Jan 2017 17:46:15 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from [192.168.0.6] (67-0-248-244.albq.qwest.net [67.0.248.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id C4B5A1928BA; Fri, 27 Jan 2017 17:46:08 +0000 (UTC) Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending To: =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org> <1598d97bf2a.c6bcb76838987.6501340920645175463@nextbsd.org> <574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org> <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> <1598f29d379.ea6360351471.8752933472741761813@nextbsd.org> Cc: "freebsd-net@freebsd.org" From: Sean Bruno Message-ID: Date: Fri, 27 Jan 2017 10:46:04 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="JBPSXTMuAN4S0GGIIEmRSb9jLKW4DWmwc" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jan 2017 17:46:16 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --JBPSXTMuAN4S0GGIIEmRSb9jLKW4DWmwc Content-Type: multipart/mixed; boundary="6vxjMBqwSAcAE2JVGJLvr6BfJpsGX205F"; protected-headers="v1" From: Sean Bruno To: =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= Cc: "freebsd-net@freebsd.org" Message-ID: Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org> <1598d97bf2a.c6bcb76838987.6501340920645175463@nextbsd.org> <574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org> <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> <1598f29d379.ea6360351471.8752933472741761813@nextbsd.org> In-Reply-To: --6vxjMBqwSAcAE2JVGJLvr6BfJpsGX205F Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 01/11/17 13:34, Olivier Cochard-Labb=C3=A9 wrote: >=20 > On Wed, Jan 11, 2017 at 9:13 PM, Matthew Macy > wrote: >=20 >=20 > > Hmmm ... did your old tests do 4 or 8 queues on this hardware? > > > > Did the old tests run 1024 tx/rx slots or the max 4096? >=20 > That's a great point, only having one thread per core could easily > account for this. I'm hoping Sean can make txq !=3D rxq work so tha= t > you can have 8txqs and 4 rxqs. >=20 >=20 > =E2=80=8BThe netgate RCC-VE 4860 is a 4 cores atom C2558E, and I'm usin= g 2 of > the 4 Gigabit Intel i350 ports. > Lab detail: > https://bsdrp.net/documentation/examples/forwarding_performance_lab_of_= a_netgate_rcc-ve_4860 >=20 > My tunning are (same for both test): > hw.igb.rxd=3D"2048" (it should be useless now) > hw.igb.txd=3D"2048" (it should be useless now) > hw.em.rxd=3D"2048" > hw.em.txd=3D"2048" > hw.igb.rx_process_limit=3D"-1" (It should be useless now too) > hw.em.rx_process_limit=3D"-1" >=20 > dev.igb.2.fc=3D0 > dev.igb.3.fc=3D0 >=20 > I can generate profiling data for you: what kind of data do you want ? Olivier: Matt has come up with some changes that seem to resolve a number of degenerate cases that I saw in my testing after dissecting your test case. In your spare time, can you try applying and testing the linked patch here to see if your results match my results? https://people.freebsd.org/~sbruno/iflib/iflib_update.diff sean --6vxjMBqwSAcAE2JVGJLvr6BfJpsGX205F-- --JBPSXTMuAN4S0GGIIEmRSb9jLKW4DWmwc Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEEuq1GMucSHejSCZfdEgHvyh5yfmQFAliLh11fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEJB QUQ0NjMyRTcxMjFERThEMjA5OTdERDEyMDFFRkNBMUU3MjdFNjQACgkQEgHvyh5y fmQHeggAtJMUiL65vzSVJiVqrQwkYOOeiaMTte+B27cvpRDnL7B8SPHqFguPgDSF 3OswtaGztpYRjSQ7UbmPUPRIju5ASEl/otz4vkTJYgYlU4JJFxvuAjc8+UNjB43J Kn0t9lK3S9k+uNpgdDZsMi1lTsfEfXqrYi+UsS6KWgxHIKT/mAEz4e3zsR8bYYS8 y6v9MHyekWqM8FuVoqGxqvUkm4uFCA5L1eQKgWdFN/xlysDBiECTMtQXRhuLz2A5 R0j9QhYpqoY5QnpaDFEiMWOX3N7Bp7pDXaWbISPtCGtFkh//VGXQ9rMj+Y9byRmH //qYGTQKa9bSB7oYwW/h2ebqvSyHAA== =4NHv -----END PGP SIGNATURE----- --JBPSXTMuAN4S0GGIIEmRSb9jLKW4DWmwc--