From owner-freebsd-arch@FreeBSD.ORG Sun Apr 13 02:19:33 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 232231065672 for ; Sun, 13 Apr 2008 02:19:33 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.231]) by mx1.freebsd.org (Postfix) with ESMTP id E5C188FC1F for ; Sun, 13 Apr 2008 02:19:32 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by rv-out-0506.google.com with SMTP id b25so362377rvf.43 for ; Sat, 12 Apr 2008 19:19:32 -0700 (PDT) Received: by 10.141.37.8 with SMTP id p8mr2525376rvj.53.1208053172655; Sat, 12 Apr 2008 19:19:32 -0700 (PDT) Received: from ?10.0.1.199? ( [24.94.72.120]) by mx.google.com with ESMTPS id g22sm7193414rvb.5.2008.04.12.19.19.30 (version=SSLv3 cipher=OTHER); Sat, 12 Apr 2008 19:19:32 -0700 (PDT) Date: Sat, 12 Apr 2008 16:20:50 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Alfred Perlstein In-Reply-To: <20080413020855.GA95731@elvis.mu.org> Message-ID: <20080412161417.Q43186@desktop> References: <200804121703.m3CH3StJ081660@chez.mckusick.com> <41ED3941-E5E6-45F0-B880-C1B2861FDE32@rabson.org> <20080412131017.S43186@desktop> <20080412234547.GZ95731@elvis.mu.org> <20080412135135.V43186@desktop> <20080413020855.GA95731@elvis.mu.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kirk McKusick , arch@freebsd.org Subject: Re: VOP_LEASE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Apr 2008 02:19:33 -0000 On Sat, 12 Apr 2008, Alfred Perlstein wrote: > * Jeff Roberson [080412 16:51] wrote: >> >> On Sat, 12 Apr 2008, Alfred Perlstein wrote: >> >>> * Jeff Roberson [080412 16:13] wrote: >>>> On Sat, 12 Apr 2008, Doug Rabson wrote: >>>> >>>>> >>>>> On 12 Apr 2008, at 18:03, Kirk McKusick wrote: >>>>> >>>>>>> Date: Sat, 12 Apr 2008 02:13:15 -1000 (HST) >>>>>>> From: Jeff Roberson >>>>>>> To: arch@freebsd.org >>>>>>> Subject: VOP_LEASE >>>>>>> >>>>>>> As far as I can tell this has never been used. Unless someone can show >>>>>>> me >>>>>>> otherwise I'm going to go ahead and remove it. >>>>>>> >>>>>>> Thanks, >>>>>>> Jeff >>>>>> >>>>>> VOP_LEASE is used by NQNFS and NFSv4. It notifies them when a file >>>>>> is modified locally so that they know to update any outstanding >>>>>> leases (e.g., evict any write lease for the file and do callbacks >>>>>> for any read leases for the file). Deleting VOP_LEASE would break >>>>>> NFS big time. >>>>> >>>>> I think our NQNFS support might have been removed some time ago - I can't >>>>> see any calls to VOP_LEASE in the code right now. Something like >>>>> VOP_LEASE >>>>> would certainly be useful for a hypothetical future NFSv4 server. I >>>>> believe that samba could use it too for its oplocks feature which appears >>>>> to be similar to NQNFS's leases and NFSv4's delegations. >>>> >>>> So the idea with delegations is that close() doesn't actually release the >>>> file entirely to make future access cheaper? >>>> >>>> My issue with VOP_LEASE is only that there are no in kernel >>>> implementations of the VOP. I doubt it is applied regularly in syscalls. >>>> It also seems odd that it is called without a lock. >>>> >>>> Is the intent that the server will trap all accesses to a local vnode in >>>> order to invalidate the client leases? >>> >>> VOP_LEASE is supposed to implemented by a filesystem client. >>> >>> For insance, NFS client with NQNFS would implement the VOP_LEASE >>> and trap those accesses to manage the lease with the remote server. >>> >>> The remote server would get "lease RPCs" from the client and manage >>> the cache appropriately. So just to be clear, this is required for nfsv4 client but not presently used by nfsv4 client? The vnodes we're calling VOP_LEASE on are actually remote files? >> >> So why isn't this done within the actual VOP? If the lease expires >> between calling VOP_LEASE and vn_lock(), VOP_READ() you have to do that >> work all over again anyway. >> >> I don't yet see why this is in filesystem independent code. I'm not >> asserting that it doesn't need to be. I'd just like to understand it >> better. > > The reason to have it is to reduce code duplication and not to be > holding the vnode locks while doing the callbacks into the server > code. > > Let me explain, the reason is 2-fold, one for reducing code duplication > and the other for avoiding holding locks for extended periods. > > Consider a local client contending against a remote client for a > filesystem that supported leases. > > Basically, each and every filesytem would have to explicitly do a > VOP_LEASE at the start of every routine that required notifying the > server making use of the underlying filesystem. So this is for the _server_ side and not the client side. That's what I originally asked. So you want to notify the nfsv4 server code that has mounted a local filesystem that you're going to modify or read a file locally so it can invalidate the client cache. Correct? > > What you really wind up doing is having a vop_stdlocallease that > calls into a generic lease manager that does callbacks into any > server exporting that file. > > So, if you move the lease call INTO the VOP_READ/READDIR/WRITE/etc > you wind up holding vnode locks while doing client communication > when contending with remote servers. Ok but doesn't this open a race? What about: VOP_LEASE() -> invalidate current remote leases <- new lease established vn_lock() VOP_WRITE() vn_unlock() Jeff > > -- > - Alfred Perlstein >