Date: Mon, 19 Oct 2015 15:34:47 +0000 From: "Eggert, Lars" <lars@netapp.com> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: "jfv@FreeBSD.org" <jfv@freebsd.org>, "ricera10@gmail.com" <ricera10@gmail.com>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, "net@freebsd.org" <net@freebsd.org> Subject: Re: ixl 40G bad performance? Message-ID: <D845A1C3-77AE-401F-BDD7-AEC10EE6F0F3@netapp.com> In-Reply-To: <CA%2BhQ2%2BiGk5Bx5mZ8v7K9cPdiNcJgBz99kmP8=twFnETw-bi2vw@mail.gmail.com> References: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> <CA%2BhQ2%2Bj5PkmRWe-ydahExfvOj%2BzestFEn%2B5DZ9_acG2Z_v=8_w@mail.gmail.com> <C81B629B-67D5-4E5B-A215-452E837BE673@netapp.com> <CA%2BhQ2%2BiGk5Bx5mZ8v7K9cPdiNcJgBz99kmP8=twFnETw-bi2vw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hi,
in order to eliminate network or hardware weirdness, I've rerun the test with Linux 4.3rc6, where I get 13.1 Gbits/sec throughput and 52 usec flood ping latency. Not great either, but in line with earlier experiments with Mellanox NICs and an untuned Linux system.
On 2015-10-19, at 17:11, Luigi Rizzo <rizzo@iet.unipi.it> wrote:
> I suspect it might not touch the c states, but better check. The safest is
> disable them in the bios.
I'll try that.
>> hw.ixl.dynamic_tx_itr: 0
>> hw.ixl.dynamic_rx_itr: 0
>>
>>
> There must be some other control for the actual (fixed, not dynamic)
> moderation.
The only other sysctls in ixl(4) that look relevant are:
hw.ixl.rx_itr
The RX interrupt rate value, set to 8K by default.
hw.ixl.tx_itr
The TX interrupt rate value, set to 4K by default.
I'll play with those.
>> Also no change with "hw.ixl.ringsz=256" in loader.conf.
>
> Any better success with 2048 slots?
> 3.5 gbit is what I used to see on the ixgbe with tso disabled, probably
> hitting a CPU bound.
Will try.
Thanks!
Lars
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
iQCVAwUBViUNm9ZcnpRveo1xAQihLQP/TL9IlhVgdX5D4soii576IC/ZSqq4xYw3
h2iT2QjQBNks6US3p2pv8ZT+5SkmtbN0c8GPFd3mmJ/PM2aov/JSb2wp+Xmz+zTD
MOqjhGXKs84g5dc1Cb83dlkm2EmrrGxotpIvYp+ioui13BNqxDWFYVvIXyPN/wQP
Dx+sdHklhHo=
=74hg
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D845A1C3-77AE-401F-BDD7-AEC10EE6F0F3>