Date: Wed, 17 Apr 2013 21:36:52 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Paul van der Zwan <paulz@vanderzwan.org> Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9.1 NFSv4 client attribute cache not caching ? Message-ID: <986577218.940691.1366249012504.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <495AEA10-9B8F-4A03-B706-79BF43539482@vanderzwan.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Paul van der Zwan wrote:
> On 12 Apr 2013, at 16:28 , Paul van der Zwan <paulz@vanderzwan.org>
> wrote:
>
> >
> > I am running a few VirtualBox VMs with 9.1 on my OpenIndiana server
> > and I noticed that make buildworld seem to take much longer
> > when the clients mount /usr/src and /usr/obj over NFS V4 than when
> > they use V3.
> > Unfortunately I have to use V4 as a buildworld on V3 hangs the
> > server completely...
> > I noticed the number of PUTFH/GETATTR/GETFH calls in in the order of
> > a few thousand per second
> > and if I snoop the traffic I see the same filenames appear over and
> > over again.
> > It looks like the client is not caching anything at all and using a
> > server request everytime.
> > I use the default mount options:
> > 192.168.178.24:/data/ports on /usr/ports (nfs, nfsv4acls)
> > 192.168.178.24:/data/src on /usr/src (nfs, nfsv4acls)
> > 192.168.178.24:/data/obj on /usr/obj (nfs, nfsv4acls)
> >
> >
>
just fyi, on a kernel build test I just did, I am seeing a
much larger number of Lookups for NFSv4 vs NFSv3.
I'll post again if/when I come up with a fix, rick
> I had a look with dtrace
> $ sudo dtrace -n '::getattr:start { @[stack()]=count();}'
> and it seems the vast majority of the calls to getattr are from open()
> and close() system calls.:
> kernel`newnfs_request+0x631
> kernel`nfscl_request+0x75
> kernel`nfsrpc_getattr+0xbe
> kernel`nfs_getattr+0x280
> kernel`VOP_GETATTR_APV+0x74
> kernel`nfs_lookup+0x3cc
> kernel`VOP_LOOKUP_APV+0x74
> kernel`lookup+0x69e
> kernel`namei+0x6df
> kernel`kern_execve+0x47a
> kernel`sys_execve+0x43
> kernel`amd64_syscall+0x3bf
> kernel`0xffffffff80784947
> 26
>
> kernel`newnfs_request+0x631
> kernel`nfscl_request+0x75
> kernel`nfsrpc_getattr+0xbe
> kernel`nfs_close+0x3e9
> kernel`VOP_CLOSE_APV+0x74
> kernel`kern_execve+0x15c5
> kernel`sys_execve+0x43
> kernel`amd64_syscall+0x3bf
> kernel`0xffffffff80784947
> 26
>
> kernel`newnfs_request+0x631
> kernel`nfscl_request+0x75
> kernel`nfsrpc_getattr+0xbe
> kernel`nfs_getattr+0x280
> kernel`VOP_GETATTR_APV+0x74
> kernel`nfs_lookup+0x3cc
> kernel`VOP_LOOKUP_APV+0x74
> kernel`lookup+0x69e
> kernel`namei+0x6df
> kernel`vn_open_cred+0x330
> kernel`vn_open+0x1c
> kernel`kern_openat+0x207
> kernel`kern_open+0x19
> kernel`sys_open+0x18
> kernel`amd64_syscall+0x3bf
> kernel`0xffffffff80784947
> 2512
>
> kernel`newnfs_request+0x631
> kernel`nfscl_request+0x75
> kernel`nfsrpc_getattr+0xbe
> kernel`nfs_close+0x3e9
> kernel`VOP_CLOSE_APV+0x74
> kernel`vn_close+0xee
> kernel`vn_closefile+0xff
> kernel`_fdrop+0x3a
> kernel`closef+0x332
> kernel`kern_close+0x183
> kernel`sys_close+0xb
> kernel`amd64_syscall+0x3bf
> kernel`0xffffffff80784947
> 2530
>
> I had a look at the source of nfs_close and could not find a call to
> nfsrpc_getattr, and I am wondering why close would be calling getattr
> anyway.
> If the file is closed what do we care about it's attributes....
>
>
> Paul
>
> _______________________________________________
> freebsd-fs@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?986577218.940691.1366249012504.JavaMail.root>