From owner-freebsd-current@FreeBSD.ORG Fri Aug 19 13:43:29 2011 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 056EC106564A; Fri, 19 Aug 2011 13:43:29 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id 030C38FC0A; Fri, 19 Aug 2011 13:43:27 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id p7JDhF7D052214; Fri, 19 Aug 2011 22:43:25 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id p7JDhDF3084027; Fri, 19 Aug 2011 22:43:15 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Fri, 19 Aug 2011 22:43:10 +0900 (JST) Message-Id: <20110819.224310.740411147168584392.hrs@allbsd.org> To: current@FreeBSD.org From: Hiroki Sato In-Reply-To: <20110819.002046.908756241495481148.hrs@allbsd.org> References: <20110819.002046.908756241495481148.hrs@allbsd.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.3 on Emacs 23.1 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Fri_Aug_19_22_43_10_2011_991)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Fri, 19 Aug 2011 22:43:26 +0900 (JST) X-Spam-Status: No, score=-104.1 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT, FAKEDWORD_ZERO, RDNS_NONE, SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: pjd@FreeBSD.org Subject: Re: fsid change of ZFS? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 13:43:29 -0000 ----Security_Multipart(Fri_Aug_19_22_43_10_2011_991)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hiroki Sato wrote in <20110819.002046.908756241495481148.hrs@allbsd.org>: hr> Hi, hr> hr> I have experienced "Stale NFS file handle" issue when switching hr> between oldnfs and newnfs on a CURRENT box (NFS server exporting ZFS hr> mountpoints). The cause was that fsid was changed in the following hr> conditions and not in the NFS subsystem itself, but I am wondering if hr> these are expected behavior... hr> hr> First, I tried the following configurations of NFS and ZFS, and saw hr> if fsid of the same mountpoint (a mounted ZFS dataset) changed or hr> not by using statfs(2): hr> hr> compile opts kld module fsid[0:1] kld loaded by hr> ---------------------------------------------------------------------------- hr> NFSSERVER+NFSCLIENT zfs 865798fa:8346ef02 loader hr> hr> NFSSERVER+NFSCLIENT zfs 865798fa:8346ef07 kldload(8) hr> hr> NFSSERVER+NFSCLIENT+ hr> NFSD+NFSCL zfs 865798fa:8346ef03 loader hr> hr> NFSSERVER+NFSCLIENT+ hr> NFSD+NFSCL zfs 865798fa:8346ef08 kldload(8) hr> hr> NFSSERVER+NFSCLIENT nfsd+nfscl+zfs 865798fa:8346ef08 loader hr> ---------------------------------------------------------------------------- Ah, I found why this happened: /* * The fsid is 64 bits, composed of an 8-bit fs type, which * separates our fsid from any other filesystem types, and a * 56-bit objset unique ID. The objset unique ID is unique to * all objsets open on this system, provided by unique_create(). * The 8-bit fs type must be put in the low bits of fsid[1] * because that's where other Solaris filesystems put it. */ fsid_guid = dmu_objset_fsid_guid(zfsvfs->z_os); ASSERT((fsid_guid & ~((1ULL<<56)-1)) == 0); vfsp->vfs_fsid.val[0] = fsid_guid; vfsp->vfs_fsid.val[1] = ((fsid_guid>>32) << 8) | vfsp->mnt_vfc->vfc_typenum & 0xFF; Since the vfc_typenum variable is incremented every time a new vfs is installed, loading order of modules that call vfs_register() affects ZFS's fsid. Anyway, possibility of fsid change is troublesome especially for an NFS server with a lot of clients running. Can zeroing or setting a fixed value to the lowest 8-bit of vfs_fsid.val[1] be harmful? -- Hiroki ----Security_Multipart(Fri_Aug_19_22_43_10_2011_991)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk5OaG4ACgkQTyzT2CeTzy0QsQCg22UdyvrWFV2QW/VxO4oqAz7v uNIAoIxRDfUCNBcPGrCWC8pj8dma7rHy =kBne -----END PGP SIGNATURE----- ----Security_Multipart(Fri_Aug_19_22_43_10_2011_991)----