From owner-freebsd-fs@FreeBSD.ORG Fri Feb 18 01:28:42 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89E8D106566B for ; Fri, 18 Feb 2011 01:28:42 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 4D3038FC17 for ; Fri, 18 Feb 2011 01:28:41 +0000 (UTC) Received: from lawrence1.loshell.room52.net (ppp59-167-184-191.static.internode.on.net [59.167.184.191]) by lauren.room52.net (Postfix) with ESMTPSA id 7CFE17E84A; Fri, 18 Feb 2011 12:12:16 +1100 (EST) Message-ID: <4D5DC770.2090308@freebsd.org> Date: Fri, 18 Feb 2011 12:12:16 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-AU; rv:1.9.2.13) Gecko/20101214 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: freebsd-fs@freebsd.org Subject: Mounting NFSv4 as root fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Feb 2011 01:28:42 -0000 Hi Rick, I've set up a NFS server to pxeboot a set of testbed clients from. The server filesystem tree the client needs to use as its root has nullfs mounted directories in it. Therefore, NFSv4 is the only useful way to mount it on the client because of the cross mount point traversing capabilities built into v4. I've verified that I can "mount_nfs -o nfsv4 ..." on the command line and see all the files in the tree so I have things working fine on the server side. I was aware our pxeboot only supports NFSv3, but hoped that by specifying "newfs" and "nfsv4" in the fstype and options fields respectively in fstab that things might just work when the mount root step after the kernel boot happens. It doesn't as I found out, because of two issues: 1. I believe there is a bug in the newnfs code. nfs_diskless.c wasn't copied from the old nfsclient and suitably modified for use with newnfs. As a result during boot, the ncl_mountroot() function in nfs_clvfsops.c calls nfs_setup_diskless() which calls into the old nfs code and badness happens from there on in. I have a patch which fixes this issue, though it may be completely the wrong way to do things as I'm very new (as in 24 hours new) to the NFS code. 2. pxeboot stores the filehandle and filehandle length it used to grab the kernel via NFS in the kernel's env and after the kernel has booted, it looks for these variables and reuses them i.e. at no point in the process does the code attempt to upgrade to NFSv4 if the bootstrap uses NFSv3 to grab the kernel. For my particular use case, I'm quite happy for the kernel to be pulled via NFSv3, but can't boot the client without somehow getting the client to switch to NFSv4 at the point where it mount's root after the kernel has finished booting. I tried a very hacky test in mountnfs() in nfs_clvfsops.c to see if I could set the NFSV4 flag, unset the V3 flag and tell the code to forget about the cached file handle set by the loader just to see if the code would try to renegotiate using v4... it crashed and burned. So, before I spend any more time on this, I hope to get your (or anyone else reading for that matter) thoughts on how best to proceed. Some questions: - Could you guesstimate how much work is involved to get v4 support into libstand so that pxeboot can talk v4 natively? I spent quite some time poking at libstand's code last night but don't understand the NFSv4 RPC mechanism enough to attempt writing the basic code to do it yet. The RFC explains the ordering of OPs needed quit well but I don't quite grok how the data structures for interpreting responses work. - Can you think of a hacky simple way to force my client to renegotiate the mount as v4 at the time mount root happens? Cheers, Lawrence