Date: Tue, 2 Aug 2016 22:11:53 +0200 From: Ben RUBSON <ben.rubson@gmail.com> To: freebsd-net@freebsd.org Subject: Re: Unstable local network throughput Message-ID: <3B164B7B-CBFB-4518-B57D-A96EABB71647@gmail.com> In-Reply-To: <bed13ae3-0b8f-b1af-7418-7bf1b9fc74bc@selasky.org> References: <3C0D892F-2BE8-4650-B9FC-93C8EE0443E1@gmail.com> <bed13ae3-0b8f-b1af-7418-7bf1b9fc74bc@selasky.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 02 Aug 2016, at 21:35, Hans Petter Selasky <hps@selasky.org> wrote: >=20 > Hi, Thank you for your answer Hans Petter ! > The CX-3 driver doesn't bind the worker threads to specific CPU cores = by default, so if your CPU has more than one so-called numa, you'll end = up that the bottle-neck is the high-speed link between the CPU cores and = not the card. A quick and dirty workaround is to "cpuset" iperf and the = interrupt and taskqueue threads to specific CPU cores. My CPUs : 2x E5-2620v3 with DDR4@1866. What is strange is that even without using the card (iPerf on = localhost), as my results show, I have very low and unstable random = throughput (compared to Linux on the same host). > Are you using "options RSS" and "options PCBGROUP" in your kernel = config? I only installed FreeBSD 10.3 and updated it, so I use the GENERIC = kernel. RSS and PCBGROUP are not defined in /usr/src/sys/amd64/conf/GENERIC, so = I think I do not use them. > Are you also testing CX-4 cards from Mellanox? No, I only have CX-3 at my disposal :) Ben PS : in my previous mail I sometimes used GB/s, of course you must read = Gb/s everywhere.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B164B7B-CBFB-4518-B57D-A96EABB71647>