From owner-freebsd-fs@FreeBSD.ORG Thu Apr 18 01:37:00 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 518D896B for ; Thu, 18 Apr 2013 01:37:00 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1BA641C2 for ; Thu, 18 Apr 2013 01:36:59 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAG9Nb1GDaFvO/2dsb2JhbABQgzyDMb0mgRZ0gh8BAQEDAQEBASArIAsFFhgCAg0ZAikBCSYGCAcEARwEh20GDKpyklmBI4xFfjQHgjOBEwOTOYEMgkGBI49xgycgMoEFNQ X-IronPort-AV: E=Sophos;i="4.87,496,1363147200"; d="scan'208";a="24406644" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 17 Apr 2013 21:36:52 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 80C41B403E; Wed, 17 Apr 2013 21:36:52 -0400 (EDT) Date: Wed, 17 Apr 2013 21:36:52 -0400 (EDT) From: Rick Macklem To: Paul van der Zwan Message-ID: <986577218.940691.1366249012504.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <495AEA10-9B8F-4A03-B706-79BF43539482@vanderzwan.org> Subject: Re: FreeBSD 9.1 NFSv4 client attribute cache not caching ? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Linux)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 01:37:00 -0000 Paul van der Zwan wrote: > On 12 Apr 2013, at 16:28 , Paul van der Zwan > wrote: > > > > > I am running a few VirtualBox VMs with 9.1 on my OpenIndiana server > > and I noticed that make buildworld seem to take much longer > > when the clients mount /usr/src and /usr/obj over NFS V4 than when > > they use V3. > > Unfortunately I have to use V4 as a buildworld on V3 hangs the > > server completely... > > I noticed the number of PUTFH/GETATTR/GETFH calls in in the order of > > a few thousand per second > > and if I snoop the traffic I see the same filenames appear over and > > over again. > > It looks like the client is not caching anything at all and using a > > server request everytime. > > I use the default mount options: > > 192.168.178.24:/data/ports on /usr/ports (nfs, nfsv4acls) > > 192.168.178.24:/data/src on /usr/src (nfs, nfsv4acls) > > 192.168.178.24:/data/obj on /usr/obj (nfs, nfsv4acls) > > > > > just fyi, on a kernel build test I just did, I am seeing a much larger number of Lookups for NFSv4 vs NFSv3. I'll post again if/when I come up with a fix, rick > I had a look with dtrace > $ sudo dtrace -n '::getattr:start { @[stack()]=count();}' > and it seems the vast majority of the calls to getattr are from open() > and close() system calls.: > kernel`newnfs_request+0x631 > kernel`nfscl_request+0x75 > kernel`nfsrpc_getattr+0xbe > kernel`nfs_getattr+0x280 > kernel`VOP_GETATTR_APV+0x74 > kernel`nfs_lookup+0x3cc > kernel`VOP_LOOKUP_APV+0x74 > kernel`lookup+0x69e > kernel`namei+0x6df > kernel`kern_execve+0x47a > kernel`sys_execve+0x43 > kernel`amd64_syscall+0x3bf > kernel`0xffffffff80784947 > 26 > > kernel`newnfs_request+0x631 > kernel`nfscl_request+0x75 > kernel`nfsrpc_getattr+0xbe > kernel`nfs_close+0x3e9 > kernel`VOP_CLOSE_APV+0x74 > kernel`kern_execve+0x15c5 > kernel`sys_execve+0x43 > kernel`amd64_syscall+0x3bf > kernel`0xffffffff80784947 > 26 > > kernel`newnfs_request+0x631 > kernel`nfscl_request+0x75 > kernel`nfsrpc_getattr+0xbe > kernel`nfs_getattr+0x280 > kernel`VOP_GETATTR_APV+0x74 > kernel`nfs_lookup+0x3cc > kernel`VOP_LOOKUP_APV+0x74 > kernel`lookup+0x69e > kernel`namei+0x6df > kernel`vn_open_cred+0x330 > kernel`vn_open+0x1c > kernel`kern_openat+0x207 > kernel`kern_open+0x19 > kernel`sys_open+0x18 > kernel`amd64_syscall+0x3bf > kernel`0xffffffff80784947 > 2512 > > kernel`newnfs_request+0x631 > kernel`nfscl_request+0x75 > kernel`nfsrpc_getattr+0xbe > kernel`nfs_close+0x3e9 > kernel`VOP_CLOSE_APV+0x74 > kernel`vn_close+0xee > kernel`vn_closefile+0xff > kernel`_fdrop+0x3a > kernel`closef+0x332 > kernel`kern_close+0x183 > kernel`sys_close+0xb > kernel`amd64_syscall+0x3bf > kernel`0xffffffff80784947 > 2530 > > I had a look at the source of nfs_close and could not find a call to > nfsrpc_getattr, and I am wondering why close would be calling getattr > anyway. > If the file is closed what do we care about it's attributes.... > > > Paul > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"