From owner-freebsd-current@FreeBSD.ORG Wed Mar 5 09:04:22 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B093F106566C for ; Wed, 5 Mar 2008 09:04:22 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from anchor-post-35.mail.demon.net (anchor-post-35.mail.demon.net [194.217.242.85]) by mx1.freebsd.org (Postfix) with ESMTP id 3AAB38FC13 for ; Wed, 5 Mar 2008 09:04:22 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from router.rabson.org ([80.177.232.241] helo=itchy.rabson.org) by anchor-post-35.mail.demon.net with esmtp (Exim 4.67) id 1JWpXw-000J9n-I3 for current@freebsd.org; Wed, 05 Mar 2008 09:04:20 +0000 Received: from macbook.rabson.org (macbook.rabson.org [IPv6:2002:50b1:e8f2:1:21e:52ff:fe73:8011]) by itchy.rabson.org (Postfix) with ESMTP id DFD783FB0 for ; Wed, 5 Mar 2008 09:04:19 +0000 (GMT) Message-Id: <34616125-76BE-45C8-BAF6-F3722B30818E@rabson.org> From: Doug Rabson To: current@freebsd.org Content-Type: multipart/signed; boundary=Apple-Mail-60-914876664; micalg=sha1; protocol="application/pkcs7-signature" Mime-Version: 1.0 (Apple Message framework v919.2) Date: Wed, 5 Mar 2008 09:04:19 +0000 X-Mailer: Apple Mail (2.919.2) X-Virus-Scanned: ClamAV 0.92/6135/Wed Mar 5 06:24:05 2008 on itchy.rabson.org X-Virus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: NFS locking revisited X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2008 09:04:22 -0000 --Apple-Mail-60-914876664 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Over the last couple of months, I have been working on a complete re- implementation of the NFS Lock Manager. The new NLM is designed to run in the kernel environment and uses the kernel's fcntl lock infrastructure to store its state. As part of this work, I have augmented the fcntl flock structure to include an indication of which remote system owns the lock and I have added some infrastructure to support asynchronous locking (not currently exposed to userland but required for the NLM). I have also ported the much of the userland sunrpc code to run in the kernel environment to make life easier (in my opinion, this is how all our NFS code should have been done from the start). Anyone interested in this code can find a snapshot patch at http://people.freebsd.org/~dfr/src-lockd-M5-04032008.diff , relative to an approximately two month old snapshot of -current. The current plan is to start committing this work to -current in two or three weeks time, depending on feedback. Highlights include: * Thread-safe kernel RPC client - many threads can use the same RPC client handle safely with replies being de-multiplexed at the socket upcall (typically driven directly by the NIC interrupt) and handed off to whichever thread matches the reply. For UDP sockets, many RPC clients can share the same socket. This allows the use of a single privileged UDP port number to talk to an arbitrary number of remote hosts. * Single-threaded kernel RPC server. Adding support for multi-threaded server would be relatively straightforward and would follow approximately the Solaris KPI. A single thread should be sufficient for the NLM since it should rarely block in normal operation. * Kernel mode NLM server supporting cancel requests and granted callbacks. I've tested the NLM server reasonably extensively - it passes both my own tests and the NFS Connectathon locking tests running on Solaris, Mac OS X and Ubuntu Linux. The only current limitation compared to the userland NLM server is that it doesn't currently support the command-line arguments that specify what addresses and port numbers to listen to. This can and will be fixed soon. * Userland NLM client supported. While the NLM server doesn't have support for the local NFS client's locking needs, it does have to field async replies and granted callbacks from remote NLMs that the local client has contacted. We relay these replies to the userland rpc.lockd over a local domain RPC socket. * IPv6 should be supported but has not been tested since I've been unable to get IPv6 to work properly with the Parallels virtual machines that I've been using for development. * Since both local and remote locks are managed by the same kernel locking code, local and remote processes can safely use file locks for mutual exclusion. Local processes have a slight fairness advantage compared to remote processes when contending to lock a region that has just been unlocked. This could be avoided by enabling the code currently hidden behind '#ifdef ADVLOCKASYNC_TESTING' in kern_descrip.c since that would enforce strict first-come first-served semantics for both local and remote lockers. --Apple-Mail-60-914876664--