Date: Mon, 04 Jan 2016 00:34:51 -0500 From: "Mikhail T." <mi+thun@aldan.algebra.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: =?UTF-8?Q?Karli_Sj=c3=b6berg?= <karli.sjoberg@slu.se>, freebsd-fs@FreeBSD.org Subject: Re: NFS reads vs. writes Message-ID: <568A047B.1010000@aldan.algebra.com> In-Reply-To: <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca> References: <8291bb85-bd01-4c8c-80f7-2adcf9947366@email.android.com> <5688D3C1.90301@aldan.algebra.com> <495055121.147587416.1451871433217.JavaMail.zimbra@uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On 03.01.2016 20:37, Rick Macklem wrote:
> This issue isn't new. It showed up when Sun introduced NFS in 1985.
> NFSv3 did change things a little, by allowing UNSTABLE writes.
Thank you very much, Rick, for the detailed explanation.
> If you use "sync=disabled"
> (I'm not a ZFS guy, but I think that is what the ZFS option looks likes) you
> *break* the NFS protocol (ie. violate the RFC) and put your data at some risk,
> but you will typically get better (often much better) write performance.
Yes, indeed. Disabling sync got the writing throughput all the way up to
about 86Mb/s... I still don't fully understand, why local writes are
able to achieve this speed without async and without being considered
dangerous.
> Also, the NFS server was recently tweaked so that it could handle 128K rsize/wsize,
> but the FreeBSD client is limited to MAXBSIZE and this has not been increased
> beyond 64K.
I just tried lowering ZFS' recordsize to 64k to match MAXBSIZE, but that
didn't help NFS-writing (unless sync is disabled, that is).
> If this SSD is dedicated to the ZIL and is one known to have good write performance,
> it should help, but in your case the SSD seems to be the bottleneck.
It is a chunk of an older SSD, that also houses the OS. But it is
usually idle, because executables and libraries are cached in the
abundant RAM. I've seen it do 90+Mb/s (sequential)...
I just tried removing ZIL from the receiving pool -- to force direct
writes -- but it didn't help the case, where the writes go over NFS.
However, the local writes -- with reads from NFS -- went from the 56Mb/s
I was seeing earlier to 90Mb/s!..
There is got to be a better way to do this -- preferably, some
self-tuning smarts... Thanks again. Yours,
-mi
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?568A047B.1010000>