Date: Fri, 25 Nov 2016 14:42:53 -0700 From: Alan Somers <asomers@freebsd.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: Rick Macklem <rmacklem@uoguelph.ca>, FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: NFSv4 performance degradation with 12.0-CURRENT client Message-ID: <CAOtMX2ikX7a==pKNdLnzihM0Hw2Ok=Rw2PWx8ioW3TVtzAvDzg@mail.gmail.com> In-Reply-To: <20161125135725.GD54029@kib.kiev.ua> References: <CAOtMX2jJ2XoQyVG1c04QL7NTJn1pg38s=XEgecE38ea0QoFAOw@mail.gmail.com> <20161124090811.GO54029@kib.kiev.ua> <YTXPR01MB0189E0B1DB5B16EE6B388B7DDDB60@YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM> <CAOtMX2hBXAJN_udED-u5%2B6UznR2%2BW88xgb=RqKSZL65Z3%2BcKOw@mail.gmail.com> <YTXPR01MB0189C3E11821E4F7B7DF1814DDB60@YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM> <20161125084106.GX54029@kib.kiev.ua> <YTXPR01MB018969FB21212700C4043AEADD890@YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM> <20161125135725.GD54029@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Nov 25, 2016 at 6:57 AM, Konstantin Belousov
<kostikbel@gmail.com> wrote:
> On Fri, Nov 25, 2016 at 12:54:07PM +0000, Rick Macklem wrote:
>> Well, ideally theer would be a VOP_MMAPDONE() or something like that, which
>> would tell the NFSv4 client that I/O is done on the vnode so it can close it.
>> If there was some way for the NFSv4 VOP_CLOSE() to be able to tell if the file
>> has been mmap'd, that would help since it could close the ones that are not
>> mmap'd on the last descriptor close.
>> (A counter wouldn't be as useful, since NFSv4 would have to keep checking it to
>> see if it can do the close yet, but it might still be doable.)
>
> I thought that the issue was in tracking any opens and mmaps, but from this
> reply it is not that clear. Do you need callback when all opens and mmaps
> have ended, or only opens and mmaps for write ? If later, we already have
> a suitable mechanism VOP_ADD_WRITECOUNT().
Mounting nullfs with the nocache option, ad kib suggested, fixed the
problem. Also, applying kib's patch and then mounting nullfs with
default options also fixed the problem. Here is the nfsstat output
for "ls -al" when using kib's patch. Notice the client has far fewer
opens:
nfsstat -s -e -z
Server Info:
Getattr Setattr Lookup Readlink Read Write Create Remove
494 0 0 0 0 1 0 0
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
0 0 0 0 0 0 0 0
Mknod Fsstat Fsinfo PathConf Commit LookupP SetClId SetClIdCf
0 0 0 0 0 0 0 0
Open OpenAttr OpenDwnGr OpenCfrm DelePurge DeleRet GetFH Lock
0 0 0 0 0 0 0 0
LockT LockU Close Verify NVerify PutFH PutPubFH PutRootFH
0 0 0 0 0 494 0 0
Renew RestoreFH SaveFH Secinfo RelLckOwn V4Create
0 0 0 0 0 0
Server:
Retfailed Faults Clients
0 0 0
OpenOwner Opens LockOwner Locks Delegs
0 0 0 0 0
Server Cache Stats:
Inprog Idem Non-idem Misses CacheSize TCPPeak
0 0 0 495 17280 17280
nfsstat -c -e -z
Client Info:
Rpc Counts:
Getattr Setattr Lookup Readlink Read Write Create Remove
14 0 0 0 0 0 0 0
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
0 0 0 0 0 0 0 0
Mknod Fsstat Fsinfo PathConf Commit SetClId SetClIdCf Lock
0 0 0 0 0 0 0 0
LockT LockU Open OpenCfr
0 0 0 0
OpenOwner Opens LockOwner Locks Delegs LocalOwn LocalOpen LocalLOwn
592 588 0 0 0 0 0 0
LocalLock
0
Rpc Info:
TimedOut Invalid X Replies Retries Requests
0 0 0 0 494
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses
1439 12 960 0 0 0 0 0
BioRLHits Misses BioD Hits Misses DirE Hits Misses
1 0 4 0 1 0
Thanks for the help, guys.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2ikX7a==pKNdLnzihM0Hw2Ok=Rw2PWx8ioW3TVtzAvDzg>