Date: Fri, 19 Aug 2011 10:29:17 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Hiroki Sato <hrs@FreeBSD.org> Cc: pjd@FreeBSD.org, current@FreeBSD.org Subject: Re: fsid change of ZFS? Message-ID: <1565511281.69213.1313764157732.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20110819.224310.740411147168584392.hrs@allbsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hiroki Sato wrote: > Hiroki Sato <hrs@freebsd.org> wrote > in <20110819.002046.908756241495481148.hrs@allbsd.org>: > > hr> Hi, > hr> > hr> I have experienced "Stale NFS file handle" issue when switching > hr> between oldnfs and newnfs on a CURRENT box (NFS server exporting > ZFS > hr> mountpoints). The cause was that fsid was changed in the following > hr> conditions and not in the NFS subsystem itself, but I am wondering > if > hr> these are expected behavior... > hr> > hr> First, I tried the following configurations of NFS and ZFS, and > saw > hr> if fsid of the same mountpoint (a mounted ZFS dataset) changed or > hr> not by using statfs(2): > hr> > hr> compile opts kld module fsid[0:1] kld loaded by > hr> > ---------------------------------------------------------------------------- > hr> NFSSERVER+NFSCLIENT zfs 865798fa:8346ef02 loader > hr> > hr> NFSSERVER+NFSCLIENT zfs 865798fa:8346ef07 kldload(8) > hr> > hr> NFSSERVER+NFSCLIENT+ > hr> NFSD+NFSCL zfs 865798fa:8346ef03 loader > hr> > hr> NFSSERVER+NFSCLIENT+ > hr> NFSD+NFSCL zfs 865798fa:8346ef08 kldload(8) > hr> > hr> NFSSERVER+NFSCLIENT nfsd+nfscl+zfs 865798fa:8346ef08 loader > hr> > ---------------------------------------------------------------------------- > > Ah, I found why this happened: > > /* > * The fsid is 64 bits, composed of an 8-bit fs type, which > * separates our fsid from any other filesystem types, and a > * 56-bit objset unique ID. The objset unique ID is unique to > * all objsets open on this system, provided by unique_create(). > * The 8-bit fs type must be put in the low bits of fsid[1] > * because that's where other Solaris filesystems put it. > */ > fsid_guid = dmu_objset_fsid_guid(zfsvfs->z_os); > ASSERT((fsid_guid & ~((1ULL<<56)-1)) == 0); > vfsp->vfs_fsid.val[0] = fsid_guid; > vfsp->vfs_fsid.val[1] = ((fsid_guid>>32) << 8) | > vfsp->mnt_vfc->vfc_typenum & 0xFF; > > Since the vfc_typenum variable is incremented every time a new vfs is > installed, loading order of modules that call vfs_register() affects > ZFS's fsid. > > Anyway, possibility of fsid change is troublesome especially for an > NFS server with a lot of clients running. Can zeroing or setting a > fixed value to the lowest 8-bit of vfs_fsid.val[1] be harmful? > > -- Hiroki Well, the problem is that the fsid needs to be unique among all mounts. The vfs_typenum field is used to try and ensure that it does not end up the same value as a non-ZFS file system. (A) I think making that field a fixed constant should be ok, if the function checks for a conflict by calling vfs_getvfs() to check for one. See vfs_getnewfsid() for how this is done. (There is a mutex lock that needs to be held while doing it.) Alternately, if ZFS can call vfs_getnewfsid() instead of doing its own, that might be nicer? (B) Another way to fix this would be to modify vfs_register() to look up file systems in a table (by vfc_name) and used a fixed, assigned value from the table for vfc_typenum for entries found in the table. Only do the "maxvfsconf++" when there isn't an entry for the fstype in the table. (VFS_GENERIC can be set to the size of the table. That's what happened in the bad old days when vfsconf was a table built at kernel config time.) If you guys think (B) is preferred, I could come up with a patch. I don't know enough about ZFS to do (A). rick
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1565511281.69213.1313764157732.JavaMail.root>