Date: Mon, 12 Oct 2015 19:57:56 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Peter Hodur <petehodur@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: FBSD 10.1 with NFSv4 / high nfsd CPU load (> 300%) Message-ID: <1527559687.33304726.1444694276368.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <CAPv5i6Zdj_1proPmqjBMggtz=wv6zdmGaa%2BLUHcrfD7Lf=Oi0g@mail.gmail.com> References: <CAPv5i6Zdj_1proPmqjBMggtz=wv6zdmGaa%2BLUHcrfD7Lf=Oi0g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Peter Hodur wrote:
> Hello,
>
> I have an issue with new NFS daemon. I have switch to FreeBSD 10.1 RELENG
> and tried also move from NFSv3 to NFSv4. With NFSv3 everything looks ok.
> But when I try mount same FS from client with "-o nfs4", I can see VERY
> high load on server - more than 300% within couple of seconds. And exports
> does not works - everything is too slow. NFSD state in "top" is rpcsvc.
>
> I have tried it with disabled DRC but with same result.
>
So, are the mounts working but just slow or are they hung trying to mount?
In general, NFSv4 is a very different protocol than NFSv3 and not really a
replacement for NFSv3 imho. It does offer features that some need, like better
byte range locking, ACLs, user names instead of uids...
However, if you don't need these features, there really isn't a good reason to
use it, again imho.
If you want to try and isolate what is going on:
- I'd start with "nfsstat -e -s" on the server:
--> Look for which RPCs are being done a lot and how many Opens/ClientIDs
allocated on the server.
For example, if the server is seeing a large number of Open/Close RPCs,
that could just be unavoidable overhead. (There is something called delegations,
which can be used to avoid repeated Opens of the same files, but if your
clients are opening large #s of different files, delegations won't help.)
If there are a large # of Opens allocated (near the bottom of the output,
after the RPC/Op counts), then some of the overhead can be avoided by making
the hash tables much larger. (There are tunables for these, but I can't
remember if they are in 10.1 or only 10.2?)
- If the "nfsstat -e -s" on the server doesn't seem to give you a hint w.r.t. what
is causing the overhead, then I'd use tcpdump to capture packets for a short
period of time and then look at them in wireshark. (Unlike tcpdump, wireshark
knows NFSv4.) Hopefully you'll see a large number of the same RPCs, possibly
failing, which will give you some insight.
- I'd also do a "ps axHl" on the server and see if any of the other daemons,
like nfsuserd seem to be busy. (A busy nfsused would indicate a high miss
rate on the cache of uid<->username mappings in the kernel.)
If you post your "nfsstat -e -s", then I might be able to spot something that
won't be obvious to you.
On the other hand, if your mounts aren't succeeding (or you gets lots of
files owned by "nobody" when you "ls -l" the mount point), then something
is busted in your setup. You should check /var/log/messages to see if there
are problems with your /etc/exports. You can also post your /etc/exports.
(For example, it should look just the same as what you use for NFSv3, but
with the addition of a "V4: ..." line. The "V4: ..." line does not replace
any of the other lines, which are still required for NFSv4.)
Don't know if any of this will help, rick
> On client is FBSD 9.1.
>
>
> Have someone same issue?
>
>
> Thanks
>
> Peter
> _______________________________________________
> freebsd-fs@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1527559687.33304726.1444694276368.JavaMail.zimbra>