Date: Wed, 25 Nov 2009 15:43:10 -0800 From: "Mahlon E. Smith" <mahlon@martini.nu> To: questions@freebsd.org Cc: ryan@skylab.org, ged@skylab.org Subject: ZFS and jailed environments -- best practice? Message-ID: <20091125234310.GA23504@martini.nu>
next in thread | raw e-mail | index | archive | help
--+vcRm3WFmV0Q/ShD
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
I've been playing with mixing up ZFS and jailed environments under
8.0RC, and I've hit a point where I'm just kind of wondering how
everyone else is doing it. I wanted to do this to take advantage of
delegated administration -- I want users inside a jail to be able to
control snapshot/rollback in their own homedir.
I'll break this up into what I did to get it working (I can't seem to
find a good step by step out there yet), and where I think I'm running
into what could be potential trouble.
First off, sysctl variables.
security.jail.enforce_statfs=3D0
security.jail.mount_allowed=3D1
vfs.usermount=3D1
I've always run jails with with enforce_statfs=3D1 or enforce_statfs=3D2. I
honestly don't see why that wouldn't work for ZFS stuff too, but in the
interests of following instructions (the zfs man page), I set it to 0.
Next, the 'zfs' dev node needs to be accessible from inside the jail.
So I created an /etc/devfs.rules file with the following:
host# cat /etc/devfs.rules=20
[zfsenable=3D10]
add path 'zfs' unhide=20
=2E..and added the ruleset to the jail config in rc.conf:
jail_zfstest_devfs_ruleset=3D"zfsenable"
So far so good, the jail gets a /dev/zfs, and I can issue zfs commands.
I get 'no datasets available' from within the jail, which is exactly
what I'd expect.
So, tank/jails/jail1 is a ZFS volume, and I want tank/jails/jail1/home
to be under the control of the jail, and mounted at /home inside of it.
I stop the jail and unmount the home volume.
host# zfs umount tank/jails/jail1/home
Then enabled 'jailed mode' on the volume, and start the jail back up.
host# zfs set jailed=3Don tank/jails/jail1/home
In the host, lets just say the JID is 8.
host# /sbin/zfs jail 8 tank/jails/jail1/home
=46rom that point, it appears that the host thinks that volume is not
under its own control. (good!)
host# zfs mount tank/jails/jail1/home
cannot mount 'tank/jails/jail1/home': dataset is exported to a local zo=
ne
Whew, okay. Back into the jail.
jail# zfs set mountpoint=3D/home tank/jails/jail1/home
jail# zfs mount -a
jail# zfs allow -s @homedir create,clone,mount,rollback,snapshot,send,r=
eceive,compression,checksum,quota,readonly,destroy tank/jails/jail1/home
jail# zfs allow -u user1 @homedir tank/jails/jail1/home/user1
=2E.. and by god, it works. Yay!
Here are the weird parts, or parts that make me feel like I'm not doing
something correctly.
1) From the host now -- I've got two /home partitions mounted when
displaying a 'df'. They -appear- to do the right thing... /home on the
host is correct when getting a listing, and /home in the jail is also
correct. But I can't help but feel like this is asking for trouble, or
will eat the delicious data at some point.
2) What the heck is the procedure for automating this on boot? Roll
your own? The JID shuffles, of course. I could easily whip up some
zfs jail `jls | awk '/jail1/ { print $2 }' ... junk, but where would
I put something like that? jail_afterstart0=3D"" seems to load things
in the context of the jail, not the host. And then I'd have to set
canmount=3Dnoauto on that home volume, and mount it manually from within
the jail via some startup script? Seems... like a pain in the ass for
what is otherwise a pretty blissful setup.
Really, I'm not sure what's right, what's stable, and what won't make me
totally regret doing this later. :)
Advice, discussion, or pointers elsewhere are all appreciated!
-Mahlon
--
Mahlon E. Smith =20
http://www.martini.nu/contact.html
--+vcRm3WFmV0Q/ShD
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
iD8DBQFLDcEO1bsjBDapbeMRApDQAJ4oKi3g0U7hQ7Xyy3ZBBkiUDcAiXACcDiCt
Yg6X1h5kTHDYaOfy9iUoym4=
=89yE
-----END PGP SIGNATURE-----
--+vcRm3WFmV0Q/ShD--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091125234310.GA23504>