Date: Tue, 24 Jan 2017 08:31:13 -0700 From: Sean Bruno <sbruno@freebsd.org> To: =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= <olivier@freebsd.org> Cc: Matthew Macy <mmacy@nextbsd.org>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, "freebsd-current@freebsd.org" <freebsd-current@freebsd.org> Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending Message-ID: <b2714d3a-75f7-4959-e390-8ade11d77962@freebsd.org> In-Reply-To: <CA%2Bq%2BTcpcZUJTmycPBF9kS-x4qcqN7V3LdHW=BQ3ttzXLyU3FWw@mail.gmail.com> References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org> <1598d97bf2a.c6bcb76838987.6501340920645175463@nextbsd.org> <574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org> <CA%2Bq%2BTco-dcoU8EZnDEzgoK-v2Q2=U5GF6ASMSj0kwzd_wB5xig@mail.gmail.com> <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> <1598f29d379.ea6360351471.8752933472741761813@nextbsd.org> <CA%2Bq%2BTcpUXXPEQtdMFup6EZzyCKs9Ep%2BnS5SB%2Bfm6bSJSDs34_w@mail.gmail.com> <1598f3f8588.d20017893749.339651164872952258@nextbsd.org> <1598f42ad77.eeec05be4113.9201780237587761460@nextbsd.org> <CA%2Bq%2BTcp5LwrnXt75tNpYpAr1KWx9YpLx5kMHhPR%2BYgAs__n1eA@mail.gmail.com> <159902b73ed.10775291e21533.7488368455500235608@nextbsd.org> <CA%2Bq%2BTcpHmuOGyp5A290WmUvGTnOSse7v8gj4=R8kZ=m51-_s4A@mail.gmail.com> <18abdd64-08a6-50ca-fb6b-9c01a3d7b60c@freebsd.org> <CA%2Bq%2BTcptEN5pcScYo4j3O8OuJHEacZu9ugOz_6b2iFb-CzBFXA@mail.gmail.com> <ad7fdc31-b0dd-2105-1610-cf0f3de42245@freebsd.org> <CA%2Bq%2BTcpcZUJTmycPBF9kS-x4qcqN7V3LdHW=BQ3ttzXLyU3FWw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--Gjupok5Xwh92XDQejKlekdOSOrwjfXdic
Content-Type: multipart/mixed; boundary="xDI9f4ipfWJNETqafQe9wabhuoRSaXFIG";
protected-headers="v1"
From: Sean Bruno <sbruno@freebsd.org>
To: =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= <olivier@freebsd.org>
Cc: Matthew Macy <mmacy@nextbsd.org>,
"freebsd-net@freebsd.org" <freebsd-net@freebsd.org>,
"freebsd-current@freebsd.org" <freebsd-current@freebsd.org>
Message-ID: <b2714d3a-75f7-4959-e390-8ade11d77962@freebsd.org>
Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb
pending
References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org>
<b000a957-8d17-a04d-6275-0d3920aa8a17@freebsd.org>
<CA+q+TcramTrYgYT-s+=aBZzRJV8FmKQqGt=1twPhLBR7AoXkcQ@mail.gmail.com>
<1598d97bf2a.c6bcb76838987.6501340920645175463@nextbsd.org>
<574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org>
<CA+q+Tco-dcoU8EZnDEzgoK-v2Q2=U5GF6ASMSj0kwzd_wB5xig@mail.gmail.com>
<6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org>
<1598f29d379.ea6360351471.8752933472741761813@nextbsd.org>
<CA+q+TcpUXXPEQtdMFup6EZzyCKs9Ep+nS5SB+fm6bSJSDs34_w@mail.gmail.com>
<1598f3f8588.d20017893749.339651164872952258@nextbsd.org>
<1598f42ad77.eeec05be4113.9201780237587761460@nextbsd.org>
<CA+q+Tcp5LwrnXt75tNpYpAr1KWx9YpLx5kMHhPR+YgAs__n1eA@mail.gmail.com>
<159902b73ed.10775291e21533.7488368455500235608@nextbsd.org>
<CA+q+TcpHmuOGyp5A290WmUvGTnOSse7v8gj4=R8kZ=m51-_s4A@mail.gmail.com>
<18abdd64-08a6-50ca-fb6b-9c01a3d7b60c@freebsd.org>
<CA+q+TcptEN5pcScYo4j3O8OuJHEacZu9ugOz_6b2iFb-CzBFXA@mail.gmail.com>
<ad7fdc31-b0dd-2105-1610-cf0f3de42245@freebsd.org>
<CA+q+TcpcZUJTmycPBF9kS-x4qcqN7V3LdHW=BQ3ttzXLyU3FWw@mail.gmail.com>
In-Reply-To: <CA+q+TcpcZUJTmycPBF9kS-x4qcqN7V3LdHW=BQ3ttzXLyU3FWw@mail.gmail.com>
--xDI9f4ipfWJNETqafQe9wabhuoRSaXFIG
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
On 01/24/17 08:27, Olivier Cochard-Labb=C3=A9 wrote:
> On Tue, Jan 24, 2017 at 3:17 PM, Sean Bruno <sbruno@freebsd.org
> <mailto:sbruno@freebsd.org>> wrote:
>=20
>=20
>=20
> Did you increase the number of rx/tx rings to 8 and the number of
> descriptors to 4k in your tests or just the defaults?
>=20
>=20
> Tuning are same as described in my previous email (rxd|txd=3D2048, rx|t=
x
> process_limit=3D-1, max_interrupt_rate=3D16000).
> [root@apu2]~# sysctl hw.igb.
> hw.igb.tx_process_limit: -1
> hw.igb.rx_process_limit: -1
> hw.igb.num_queues: 0
> hw.igb.header_split: 0
> hw.igb.max_interrupt_rate: 16000
> hw.igb.enable_msix: 1
> hw.igb.enable_aim: 1
> hw.igb.txd: 2048
> hw.igb.rxd: 2048
>=20
>=20
Oh, I think you missed my note on these. In order to adjust txd/rxd you
need to tweak the iflib version of these numbers. nrxds/ntxds should be
adjust upwards to your value of 2048. nrxqs/ntxqs should be adjust
upwards to 8, I think, so you can test equivalent settings to the legacy
driver.
Specifically, you may want to adjust these:
dev.em.0.iflib.override_nrxds: 0
dev.em.0.iflib.override_ntxds: 0
dev.em.0.iflib.override_nrxqs: 0
dev.em.0.iflib.override_ntxqs: 0
sean
> But I've did a new benchs with default setting, and the performance dro=
p
> is now about -25% :
>=20
> x head r311848 packets-per-second (default settings)
> + head r311849 packets-per-second (default settings)
> +----------------------------------------------------------------------=
----+
> |+ =
|
> |+ =
x |
> |+ =
xx|
> |++ =
xx|
> | =
A||
> |A| =
|
> +----------------------------------------------------------------------=
----+
> N Min Max Median Avg St=
ddev
> x 5 618711 621135 619930.5 619840.8 951.8=
3787
> + 5 467389 468740 467778 467864.8 550.4=
0322
> Difference at 95.0% confidence
> -151976 +/- 1133.9
> -24.5186% +/- 0.150581%
> (Student's t, pooled s =3D 777.476)
>=20
--xDI9f4ipfWJNETqafQe9wabhuoRSaXFIG--
--Gjupok5Xwh92XDQejKlekdOSOrwjfXdic
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQGTBAEBCgB9FiEEuq1GMucSHejSCZfdEgHvyh5yfmQFAliHc0FfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEJB
QUQ0NjMyRTcxMjFERThEMjA5OTdERDEyMDFFRkNBMUU3MjdFNjQACgkQEgHvyh5y
fmRKDAgAlleMvp13X8F/56RMAHj+zK2gnL3R1BRAzjeG3fGgPTiz/SnqaR9ngadl
d3kYkAxMUyFz5zSmWW3d9BB49No+syppWdBIyH1Pv4wqNsSQOUP5pXl1ot143H84
Y9jEDgN4/QluuQXUaj1YvG7s6n1Bw8wE7SSCcjqvDg0O1L5xTOSQF+atwDd3+o6r
zUOzhdYwKuc0f98vsGTspCu5JQ/FfS/p9Y5F3xvTo9Gj6HtbbBXHuJ3k5eBeHJ0f
aKUhBSpxVrDBLUBpaj7ClO0/166PnBBUU1bRaty5iDJbMmYgnWDtQtvaoCOS2/U7
XM9vSI6D6kuZAY6k7MidzZjFCLP3sQ==
=GH+B
-----END PGP SIGNATURE-----
--Gjupok5Xwh92XDQejKlekdOSOrwjfXdic--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b2714d3a-75f7-4959-e390-8ade11d77962>