Date: Tue, 10 Dec 2013 19:21:34 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: Jason Keltz <jas@cse.yorku.ca> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org>, Steve Dickson <SteveD@redhat.com> Subject: Re: mount ZFS snapshot on Linux system Message-ID: <1094466847.28925085.1386721294944.JavaMail.root@uoguelph.ca> In-Reply-To: <52A78496.3030803@cse.yorku.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Jason Keltz wrote: > I'm running FreeBSD 9.2 with various ZFS datasets. > I export a dataset to a Linux system (RHEL64), and mount it. It > works > fine... > When I try to access the ZFS snapshot directory on the Linux NFS > client, > things go weird. > > With NFSv4: > > [jas@archive /]# cd /mnt/.zfs/snapshot > [jas@archive snapshot]# ls > 20131203 20131205 20131206 20131207 20131208 20131209 20131210 > [jas@archive snapshot]# cd 20131210 > 20131210: Not a directory. > > huh? > > [jas@archive snapshot]# ls -al > total 77 > dr-xr-xr-x 9 root root 9 Dec 10 11:20 . > dr-xr-xr-x 4 root root 4 Nov 28 15:42 .. > drwxr-xr-x 380 root root 380 Dec 2 15:56 20131203 > drwxr-xr-x 381 root root 381 Dec 3 11:24 20131205 > drwxr-xr-x 381 root root 381 Dec 3 11:24 20131206 > drwxr-xr-x 381 root root 381 Dec 3 11:24 20131207 > drwxr-xr-x 381 root root 381 Dec 3 11:24 20131208 > drwxr-xr-x 381 root root 381 Dec 3 11:24 20131209 > drwxr-xr-x 381 root root 381 Dec 3 11:24 20131210 > [jas@archive snapshot]# stat * > [jas@archive snapshot]# ls -al > total 292 > dr-xr-xr-x 9 root root 9 Dec 10 11:20 . > dr-xr-xr-x 4 root root 4 Nov 28 15:42 .. > -rw-r--r-- 1 uax guest 137647 Mar 17 2010 20131203 > -rw-r--r-- 1 uax guest 865 Jul 31 2009 20131205 > -rw-r--r-- 1 uax guest 137647 Mar 17 2010 20131206 > -rw-r--r-- 1 uax guest 771 Jul 31 2009 20131207 > -rw-r--r-- 1 uax guest 778 Jul 31 2009 20131208 > -rw-r--r-- 1 uax guest 5281 Jul 31 2009 20131209 > -rw------- 1 btx faculty 893 Jul 13 20:21 20131210 > > But it gets even more fun.. > > # ls -ali > total 205 > 2 dr-xr-xr-x 9 root root 9 Dec 10 11:20 . > 1 dr-xr-xr-x 4 root root 4 Nov 28 15:42 .. > 863 -rw-r--r-- 1 uax guest 137647 Mar 17 2010 20131203 > 4 drwxr-xr-x 381 root root 381 Dec 3 11:24 20131205 > 4 drwxr-xr-x 381 root root 381 Dec 3 11:24 20131206 > 4 drwxr-xr-x 381 root root 381 Dec 3 11:24 20131207 > 4 drwxr-xr-x 381 root root 381 Dec 3 11:24 20131208 > 4 drwxr-xr-x 381 root root 381 Dec 3 11:24 20131209 > 4 drwxr-xr-x 381 root root 381 Dec 3 11:24 20131210 > > This is not a user id mapping issue because all the files in /mnt > have > the proper owner/groups, and I can access them there fine. > > I also tried explicitly exporting .zfs/snapshot. The result isn't > any > different. > > If I use nfs v3 it "works", but I'm seeing a whole lot of errors like > these in syslog: > > Dec 10 12:32:28 jungle mountd[49579]: can't delete exports for > /local/backup/home9/.zfs/snapshot/20131203: Invalid argument > Dec 10 12:32:28 jungle mountd[49579]: can't delete exports for > /local/backup/home9/.zfs/snapshot/20131209: Invalid argument > Dec 10 12:32:28 jungle mountd[49579]: can't delete exports for > /local/backup/home9/.zfs/snapshot/20131210: Invalid argument > Dec 10 12:32:28 jungle mountd[49579]: can't delete exports for > /local/backup/home9/.zfs/snapshot/20131207: Invalid argument > > It's not clear to me why this doesn't just "work". > > Can anyone provide any advice on debugging this? > As I think you already know, I know nothing about ZFS and never use it. Having said that, I suspect that there are filenos (i-node #s) that are the same in the snapshot as in the parent file system tree. The basic assumptions are: - within a file system, all i-node# are unique (represent one file object only) and all file objects have the same fsid - when the fsid changes, that indicates a file system boundary and fileno (i-node#s) can be reused in the subtree with a different fsid For NFSv3, the server should export single volumes only (all objects have the same fsid and the filenos are unique). This is indicated to the VFS by the use of the NOCROSSMOUNT flag on VOP_LOOKUP() and friends. For NFSv4, the server does export multiple volumes and the boundary is indicated by a change in fsid value. I suspect ZFS snaphots don't obey the above in some way, but that is just a hunch. Now, how to narrow this down... - Do the above tests (both NFSv4 and NFSv3) and capture the packets, then look at them in wireshark. In particular, look at the fileid numbers and fsid values for the various directories under .zfs. - Try mounting the individual snapshot directory, like .zfs/snapshot/20131209 and see if that works (for both NFSv3 and NFSv4). - Try doing the mounts with a FreeBSD client and see if you get the same behaviour? rick > Thanks, > > Jason. > > -- > Jason Keltz > Manager of Development > Department of Electrical Engineering and Computer Science > York University, Toronto, Canada > Tel: 416-736-2100 x. 33570 > Fax: 416-736-5872 > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1094466847.28925085.1386721294944.JavaMail.root>