Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 26 Feb 2024 00:06:23 +0100
From:      Hannes Hauswedell <h2+lists2024@fsfe.org>
To:        freebsd-net@freebsd.org
Subject:   Re: NFS performance with 10GBase-T
Message-ID:  <714c3e34-575f-41ab-8149-6e3019f91bf1@fsfe.org>
In-Reply-To: <9F370420-52DC-4895-B2B3-5254B6E470D0@longcount.org>
References:  <034cc6ea-26d8-4520-879a-672459832407@fsfe.org> <9F370420-52DC-4895-B2B3-5254B6E470D0@longcount.org>

next in thread | previous in thread | raw e-mail | index | archive | help
>> Hi everyone,
>>
>> I am coming here from
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=2771971160
>>
>> TL;DR:
>>
>> * I have a FreeBSD14 Server and Client with an Intel X540 (ix) adaptor 
>> each.
>> * I am trying to improve the NFS throughput.
>> * I get 1160 MiB/s via nc, but only ~200 MiB/s via NFS.
>> * Increasing rsize and wsize to 1 MiB increases throughput to 395 MiB/s
>> * But a Linux client achieves 560-600 MiB/s with any rsize.
>> * The mtu is set to 9000 but this doesn't make a difference for the 
>> FreeBSD client (it does make a difference for Linux).
>>
>> I assume < 400 MiB/s is not the expected performance? Do you have any 
>> advice on debugging this?
>>
>> Thank you for your help,
>> Hannes
>>
> 
> Hi Hannes,

thank you for the reply!

>    The link didn’t work .

Sorry, no clue how that copy-pasta happened. The link Michael posted is 
correct:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277197


> Can you tell us more about the file system , 
> what type of disks .

4x4TiB NVME (Lexar NM790) raidz1

The server can maintain speeds over 900 MiB/s copying hundreds of 
Gigabytes from one dataset to another including compression and 
encryption on both ends, so I don't think the server setup is a problem.
nc manages to send data at 1100 MiB/s from the server to the client.

> Details about the server , hardware details.  If 
> you want post a copy of your dmesg to 
> https://dmesgd.nycbug.org/index.cgi 
> <https://dmesgdnycbug.org/index.cgi> that can help too .

https://dmesgd.nycbug.org/index.cgi?do=view&id=7527

If there is anything else that would help, please let me know!

> Also what kinds of clients are you using ?

The FreeBSD14 client is an amd64 machine with the same network adaptor 
as the server (Intel X540 T1).
I have set up a dual-boot with Devuan GNU/Linux. As reported above, 
Linux achieves 600 MiB/s out of the box.

> Do you have any relevant network stats or a topology/ diagram of the 
> shape of things ?

The client and server are connected via CAT7 cables and a NETGEAR 
switch. I could look up the model later, but it can probably be ruled 
out if other tools/OS achieve higher throughput?

Best regards,
Hannes
-- 
Dr. Hannes Hauswedell
C++ Software Engineer
https://hannes.hauswedell.net



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?714c3e34-575f-41ab-8149-6e3019f91bf1>