Date: Mon, 21 Oct 2024 09:17:05 -0700 From: Rick Macklem <rick.macklem@gmail.com> To: freebsd-fs@freebsd.org Subject: Re: improving nfs client & server performance Message-ID: <CAM5tNy4scNutJXdOL=UmK_NhObcfbwpnUpL1dFqe3JVeJVWvcQ@mail.gmail.com> In-Reply-To: <ZxZbPmv_11tS5pxZ@int21h> References: <ZxZbPmv_11tS5pxZ@int21h>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Oct 21, 2024 at 6:46=E2=80=AFAM void <void@f-m.fm> wrote:
>
> I'm looking to try to improve nfs client & server performance.
> The problem I'm seeing is that when clients access the server,
> if more than one does it at once, the access speed is very
> 'spiky' and in /var/log/messages on the client, it has things like
>
> Oct 20 07:21:55 kernel: nfs server 192.168.1.10:/zroot/share: not
> responding
> Oct 20 07:21:55 kernel: nfs server 192.168.1.10:/zroot/share: is alive
> again
> Oct 20 07:42:05 kernel: nfs server 192.168.1.10:/zroot/share: not
> responding
> Oct 20 07:42:05 kernel: nfs server 192.168.1.10:/zroot/share: is alive
> again
> Oct 20 08:29:54 kernel: nfs server 192.168.1.10:/zroot/share: not
> responding
>
> If one client is accessing one file at once, the transfer is very fast.
> But syncing like rsync or webdav is very problematic and takes much longe=
r than
> it should.
>
> The server is recent 14-stable and exports nfs via the zfs sharenfs prope=
rty.
> The clients are a mix of freebsd and linux (debian)
>
> I note on the server there's lots of vfs.nfsd sysctl tunables but I'm not=
sure
> if thay are relevant in a zfs sharenfs context. There's even more vfs.zfs=
but
> nothing pertaining directly to nfs.
>
> On a freebsd client, it has these in rc.conf
>
> # nfs client stuff
> nfs_client_enable=3D"YES"
>
> Maybe it needs local locks (-L) unsure how to pass flags to the client st=
arted
> in this way. How would I know if local locks were needed?
>
> I note in defaults/rc.conf there is
> nfs_bufpackets=3D"" # bufspace (in packets) for client
>
> but I'm not sure what the effects would be.
>
> I've so far set 'zfs set sync=3Ddisabled' for the particular vdev and
> 'sysctl vfs.nfsd.maxthreads=3D256' on the server, about to test this.
There are lots of possibilities, but here are a couple to try...
vfs.zfs.dmu_offset_next_sync=3D0 - this makes SEEK_HOLE/SEEK_DATA
much faster,
but less reliable (as in it
might miss
finding a hole)
vfs.nfsd.cachetcp=3D0 - this disables the DRC cache =
for TCP
connections
(if this helps, there are
settings to
try to tune he DRC).
Disabling
the DRC for TCP means that
there is a
slight chance of corruption, due
to duplicate
non-idempotent RPCs being
done after a
TCP reconnect.
Has no
effect on NFSv4.1/4.2 mounts.
rick
> --
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy4scNutJXdOL=UmK_NhObcfbwpnUpL1dFqe3JVeJVWvcQ>