From owner-freebsd-fs@FreeBSD.ORG Mon Oct 16 06:30:02 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48A0116A47C; Mon, 16 Oct 2006 06:30:02 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BAE743D5E; Mon, 16 Oct 2006 06:30:01 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id D3A99328DA5; Mon, 16 Oct 2006 16:29:59 +1000 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 90A642741E; Mon, 16 Oct 2006 16:29:58 +1000 (EST) Date: Mon, 16 Oct 2006 16:29:57 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Mohan Srinivasan In-Reply-To: <20061015205923.49804.qmail@web30813.mail.mud.yahoo.com> Message-ID: <20061016130540.C63585@delplex.bde.org> References: <20061015205923.49804.qmail@web30813.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: fs@freebsd.org, mohans@freebsd.org Subject: Re: lost dotdot caching pessimizes nfs especially X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Oct 2006 06:30:02 -0000 On Sun, 15 Oct 2006, Mohan Srinivasan wrote: > The clearing of the attrcache on nfs_open() is a requirement for close-to-open > consistency, and this change fixed bugs that we saw internally relating to > close-to-open consistency. > >> and associated changes give silly behaviour that almost doubles the >> number of Access RPCs. One of the associated changes clears n_attrstamp >> on close(). Then on open(), since lookup() is called before the above >> is reached, nfs_access_otw() has always just been called, and the above > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> forces another call. > > That is not true with NFSv2 which doesn't have an access call, in which case > nfsspec_access() calls VOP_GETATTR, which may or may not go over the wire. > > Also, what would happen with NFSv3 if we get an access cache hit ? I didn't think about NFSv2 or check the details for NFSv3 until now. It is nfs_lookup() that always calls VOP_GETATTR(), and VOP_GETATTR() must go on the wire in the case being described (lookup after close) since we flushed the attribute cache entry in nfs_close(). The difference for v2 is that nfs_getattr() normally uses a Getattr request for v2 and an Access request for v3. For NFSv3, nfs_lookup()'s behaviour is correct for the attribute cache is not as good as it could easily be for the attribute cache. In nfs_lookup() after a recent close(), in the usual cases all caches are hit except we just cleared the attribute cache, so nfs_lookup() does the following: VOP_ACCESS(); # Cache hit. Access granted. cache_lookup(); # Positive cache hit. VOP_GETATTR(); # Cache miss. Succeeds. # Now we have fresh attributes in the v3 case, but we granted access # based on the old attributes, so we unnecessarily lost full # open/close consistency. In unusual cases, there is an acccess cache miss. Then for v3, VOP_ACCESS() refreshes the attribute cache too, VOP_GETTATR() is a cache hit, and there is full open/close consistency. > If lookup() can be made to pass a flag into nfs_open() that an otw getattr was > done, then we can eliminate the clearing of the attrcache in nfs_open(). But > absent that flag, I don't see how you can eliminate the fetch of fresh attributes > in nfs_open(). Of course something like such a flag is needed. See my previous mail for more details (there should be another flag for nfs_lookup() so that the entire open() is consistent). For nfs_open(), I was thinking more of a generation count. Now I wonder about exclusive locking and blockages. VOP_OPEN() is now exclusively locked, but I don't now if the same lock covers the lookup. With exclusive locking, not even a flag is needed. Without exclusive locking, blocking might be a problem. Bruce