From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 2 16:22:41 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0D5B16A418 for ; Wed, 2 Jan 2008 16:22:41 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id AD11713C468 for ; Wed, 2 Jan 2008 16:22:41 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.storspeed.com (209-163-168-124.static.tenantsolutions.com [209.163.168.124] (may be forged)) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id m02GMSJh056792; Wed, 2 Jan 2008 10:22:29 -0600 (CST) (envelope-from anderson@freebsd.org) Message-ID: <477BBA44.7030803@freebsd.org> Date: Wed, 02 Jan 2008 10:22:28 -0600 From: Eric Anderson User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Danny Braniss References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: freebsd-hackers@freebsd.org Subject: Re: nfs v2/v3 and diskless boot problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jan 2008 16:22:42 -0000 Danny Braniss wrote: >> Danny Braniss wrote: >>> there is an undocumented option: >>> boot-nfsroot-options >>> that the diskeless boot can use. I tried >>> boot-nfsroot-options = "nfsv3" >>> since the pxeboot does the initial mount via nfsv2, and this has at least >>> one problem: removing a file from the readonly / will hang the system. >>> >>> so, the remount to v3 works in the case that the root is served by a Freebsd >>> nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle >>> is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). >>> This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE >>> :-) >>> >>> While I'm trying to come up with a solution, I am wondering if someone >>> can shed some light: >>> - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to >>> 32 bytes will start WW3? >> >> NFSv3 file handles (by spec) can be up to 64bytes. > > true, but in freebsd, look at sys/nfs/nfsproto.h > #define NFSX_V2FH 32 > #define NFSX_V3FH (sizeof (fhandle_t)) > #define NFSX_V4FH 128 > > so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) > > >> I'm not 100% sure what is happening, but it sounds like the file handle >> for the mount point or maybe one of the directories is not getting reset >> on remount. >> >> When do you get the BADHANDLE error? Can you capture a >> tshark/wireshark/tcpdump of the remount and error? > > I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible > for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and > it panics the kernel :-( > > danny oh - looks like this says it all: http://fxr.googlebit.com/source/sys/nfsclient/nfsdiskless.h?v=8-CURRENT#L51 Snippet: 51 /* 52 * I have defined a new structure that can handle an NFS Version 3 file handle 53 * but the kernel still expects the old Version 2 one to be provided. The 54 * changes required in nfs_vfsops.c for using the new are documented there in 55 * comments. (I felt that breaking network booting code by changing this 56 * structure would not be prudent at this time, since almost all servers are 57 * still Version 2 anyhow.) 58 */ Eric