From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 2 15:15:08 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7937C16A418 for ; Wed, 2 Jan 2008 15:15:08 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.freebsd.org (Postfix) with ESMTP id 31AB713C43E for ; Wed, 2 Jan 2008 15:15:08 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1JA5JD-0007S6-6F; Wed, 02 Jan 2008 17:15:07 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Eric Anderson In-reply-to: Your message of Wed, 02 Jan 2008 06:38:41 -0600 . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 02 Jan 2008 17:15:07 +0200 From: Danny Braniss Message-ID: Cc: freebsd-hackers@freebsd.org Subject: Re: nfs v2/v3 and diskless boot problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jan 2008 15:15:08 -0000 > Danny Braniss wrote: > > there is an undocumented option: > > boot-nfsroot-options > > that the diskeless boot can use. I tried > > boot-nfsroot-options = "nfsv3" > > since the pxeboot does the initial mount via nfsv2, and this has at least > > one problem: removing a file from the readonly / will hang the system. > > > > so, the remount to v3 works in the case that the root is served by a Freebsd > > nfs server, but fails if it's NetAPP. The reason is that the v2 filehandle > > is 32 bytes, and when switching to V3 it becomes 28bytes - sizeof(fhandle_t). > > This is not liked by the NetApp, which correctly gives error 1001: BADHANDLE > > :-) > > > > While I'm trying to come up with a solution, I am wondering if someone > > can shed some light: > > - is sizeof(fhandle_t) == 28 bytes is mystical, or changing it to > > 32 bytes will start WW3? > > > NFSv3 file handles (by spec) can be up to 64bytes. true, but in freebsd, look at sys/nfs/nfsproto.h #define NFSX_V2FH 32 #define NFSX_V3FH (sizeof (fhandle_t)) #define NFSX_V4FH 128 so for v3 it's 28 bytes. (fhandle_t is defined in sys/mount.h) > > I'm not 100% sure what is happening, but it sounds like the file handle > for the mount point or maybe one of the directories is not getting reset > on remount. > > When do you get the BADHANDLE error? Can you capture a > tshark/wireshark/tcpdump of the remount and error? I did, and if you look in sys/nfsclient/nfs_vfsops.c, nfs_convert_diskless is responsible for chopping off the 4 extra bytes. BTW, I tried to change the bcopy count to NFSX_V2FH/32, and it panics the kernel :-( danny