From owner-freebsd-questions@FreeBSD.ORG Thu Nov 26 00:09:53 2009 Return-Path: Delivered-To: questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04DEA10656AD for ; Thu, 26 Nov 2009 00:09:53 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: from martini.nu (martini.nu [198.145.180.83]) by mx1.freebsd.org (Postfix) with SMTP id D50AB8FC17 for ; Thu, 26 Nov 2009 00:09:52 +0000 (UTC) Received: (qmail 18566 invoked by uid 1000); 25 Nov 2009 23:43:10 -0000 Date: Wed, 25 Nov 2009 15:43:10 -0800 From: "Mahlon E. Smith" To: questions@freebsd.org Message-ID: <20091125234310.GA23504@martini.nu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+vcRm3WFmV0Q/ShD" Content-Disposition: inline X-GPG-Fingerprint: 19B8 DDB3 0156 3A03 FA80 8278 C0BE 6BFB 3606 B267 X-Sysinfo: FreeBSD 7.0-RELEASE-p1 i386 User-Agent: Mutt/1.5.17 (2007-11-01) Cc: ryan@skylab.org, ged@skylab.org Subject: ZFS and jailed environments -- best practice? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Nov 2009 00:09:53 -0000 --+vcRm3WFmV0Q/ShD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've been playing with mixing up ZFS and jailed environments under 8.0RC, and I've hit a point where I'm just kind of wondering how everyone else is doing it. I wanted to do this to take advantage of delegated administration -- I want users inside a jail to be able to control snapshot/rollback in their own homedir. I'll break this up into what I did to get it working (I can't seem to find a good step by step out there yet), and where I think I'm running into what could be potential trouble. First off, sysctl variables. security.jail.enforce_statfs=3D0 security.jail.mount_allowed=3D1 vfs.usermount=3D1 I've always run jails with with enforce_statfs=3D1 or enforce_statfs=3D2. I honestly don't see why that wouldn't work for ZFS stuff too, but in the interests of following instructions (the zfs man page), I set it to 0. Next, the 'zfs' dev node needs to be accessible from inside the jail. So I created an /etc/devfs.rules file with the following: host# cat /etc/devfs.rules=20 [zfsenable=3D10] add path 'zfs' unhide=20 =2E..and added the ruleset to the jail config in rc.conf: jail_zfstest_devfs_ruleset=3D"zfsenable" So far so good, the jail gets a /dev/zfs, and I can issue zfs commands. I get 'no datasets available' from within the jail, which is exactly what I'd expect. So, tank/jails/jail1 is a ZFS volume, and I want tank/jails/jail1/home to be under the control of the jail, and mounted at /home inside of it. I stop the jail and unmount the home volume. host# zfs umount tank/jails/jail1/home Then enabled 'jailed mode' on the volume, and start the jail back up. host# zfs set jailed=3Don tank/jails/jail1/home In the host, lets just say the JID is 8. host# /sbin/zfs jail 8 tank/jails/jail1/home =46rom that point, it appears that the host thinks that volume is not under its own control. (good!) host# zfs mount tank/jails/jail1/home cannot mount 'tank/jails/jail1/home': dataset is exported to a local zo= ne Whew, okay. Back into the jail. jail# zfs set mountpoint=3D/home tank/jails/jail1/home jail# zfs mount -a jail# zfs allow -s @homedir create,clone,mount,rollback,snapshot,send,r= eceive,compression,checksum,quota,readonly,destroy tank/jails/jail1/home jail# zfs allow -u user1 @homedir tank/jails/jail1/home/user1 =2E.. and by god, it works. Yay! Here are the weird parts, or parts that make me feel like I'm not doing something correctly. 1) From the host now -- I've got two /home partitions mounted when displaying a 'df'. They -appear- to do the right thing... /home on the host is correct when getting a listing, and /home in the jail is also correct. But I can't help but feel like this is asking for trouble, or will eat the delicious data at some point. 2) What the heck is the procedure for automating this on boot? Roll your own? The JID shuffles, of course. I could easily whip up some zfs jail `jls | awk '/jail1/ { print $2 }' ... junk, but where would I put something like that? jail_afterstart0=3D"" seems to load things in the context of the jail, not the host. And then I'd have to set canmount=3Dnoauto on that home volume, and mount it manually from within the jail via some startup script? Seems... like a pain in the ass for what is otherwise a pretty blissful setup. Really, I'm not sure what's right, what's stable, and what won't make me totally regret doing this later. :) Advice, discussion, or pointers elsewhere are all appreciated! -Mahlon -- Mahlon E. Smith =20 http://www.martini.nu/contact.html --+vcRm3WFmV0Q/ShD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFLDcEO1bsjBDapbeMRApDQAJ4oKi3g0U7hQ7Xyy3ZBBkiUDcAiXACcDiCt Yg6X1h5kTHDYaOfy9iUoym4= =89yE -----END PGP SIGNATURE----- --+vcRm3WFmV0Q/ShD--