From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 7 17:15:03 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1144106566C for ; Mon, 7 Dec 2009 17:15:03 +0000 (UTC) (envelope-from mahlon@martini.nu) Received: from martini.nu (martini.nu [198.145.180.83]) by mx1.freebsd.org (Postfix) with SMTP id 80D5E8FC16 for ; Mon, 7 Dec 2009 17:15:03 +0000 (UTC) Received: (qmail 86904 invoked by uid 1000); 7 Dec 2009 16:48:22 -0000 Date: Mon, 7 Dec 2009 08:48:22 -0800 From: "Mahlon E. Smith" To: freebsd-hackers@freebsd.org Message-ID: <20091207164821.GA22924@martini.nu> Mail-Followup-To: "Mahlon E. Smith" , freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7AUc2qLy4jB3hD7Z" Content-Disposition: inline X-GPG-Fingerprint: 19B8 DDB3 0156 3A03 FA80 8278 C0BE 6BFB 3606 B267 X-Sysinfo: FreeBSD 7.0-RELEASE-p1 i386 User-Agent: Mutt/1.5.17 (2007-11-01) Subject: Fwd: ZFS and jailed environments -- best practice? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2009 17:15:03 -0000 --7AUc2qLy4jB3hD7Z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [ repost from -questions in the hopes of generating some discussion! ] ----- Forwarded message from "Mahlon E. Smith" ----- Date: Wed, 25 Nov 2009 15:43:10 -0800 =46rom: "Mahlon E. Smith" To: questions@freebsd.org Subject: ZFS and jailed environments -- best practice? I've been playing with mixing up ZFS and jailed environments under 8.0RC, and I've hit a point where I'm just kind of wondering how everyone else is doing it. I wanted to do this to take advantage of delegated administration -- I want users inside a jail to be able to control snapshot/rollback in their own homedir. I'll break this up into what I did to get it working (I can't seem to find a good step by step out there yet), and where I think I'm running into what could be potential trouble. First off, sysctl variables. security.jail.enforce_statfs=3D0 security.jail.mount_allowed=3D1 vfs.usermount=3D1 I've always run jails with with enforce_statfs=3D1 or enforce_statfs=3D2. I honestly don't see why that wouldn't work for ZFS stuff too, but in the interests of following instructions (the zfs man page), I set it to 0. Next, the 'zfs' dev node needs to be accessible from inside the jail. So I created an /etc/devfs.rules file with the following: host# cat /etc/devfs.rules=20 [zfsenable=3D10] add path 'zfs' unhide=20 =2E..and added the ruleset to the jail config in rc.conf: jail_zfstest_devfs_ruleset=3D"zfsenable" So far so good, the jail gets a /dev/zfs, and I can issue zfs commands. I get 'no datasets available' from within the jail, which is exactly what I'd expect. So, tank/jails/jail1 is a ZFS volume, and I want tank/jails/jail1/home to be under the control of the jail, and mounted at /home inside of it. I stop the jail and unmount the home volume. host# zfs umount tank/jails/jail1/home Then enabled 'jailed mode' on the volume, and start the jail back up. host# zfs set jailed=3Don tank/jails/jail1/home In the host, lets just say the JID is 8. host# /sbin/zfs jail 8 tank/jails/jail1/home =46rom that point, it appears that the host thinks that volume is not under its own control. (good!) host# zfs mount tank/jails/jail1/home cannot mount 'tank/jails/jail1/home': dataset is exported to a local zo= ne Whew, okay. Back into the jail. jail# zfs set mountpoint=3D/home tank/jails/jail1/home jail# zfs mount -a jail# zfs allow -s @homedir create,clone,mount,rollback,snapshot,send,r= eceive,compression,checksum,quota,readonly,destroy tank/jails/jail1/home jail# zfs allow -u user1 @homedir tank/jails/jail1/home/user1 =2E.. and by god, it works. Yay! Here are the weird parts, or parts that make me feel like I'm not doing something correctly. 1) From the host now -- I've got two /home partitions mounted when displaying a 'df'. They -appear- to do the right thing... /home on the host is correct when getting a listing, and /home in the jail is also correct. But I can't help but feel like this is asking for trouble, or will eat the delicious data at some point. 2) What the heck is the procedure for automating this on boot? Roll your own? The JID shuffles, of course. I could easily whip up some zfs jail `jls | awk '/jail1/ { print $2 }' ... junk, but where would I put something like that? jail_afterstart0=3D"" seems to load things in the context of the jail, not the host. And then I'd have to set canmount=3Dnoauto on that home volume, and mount it manually from within the jail via some startup script? Seems... like a pain in the ass for what is otherwise a pretty blissful setup. Really, I'm not sure what's right, what's stable, and what won't make me totally regret doing this later. :) Advice, discussion, or pointers elsewhere are all appreciated! -Mahlon -- Mahlon E. Smith =20 http://www.martini.nu/contact.html ----- End forwarded message ----- --7AUc2qLy4jB3hD7Z Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFLHTHV1bsjBDapbeMRAmQwAKCMXZIIX9uDOq1O6rigQTNEGAMq9gCgpW5S y5yUISqOFm1mHzVcSDzF+sI= =nj8S -----END PGP SIGNATURE----- --7AUc2qLy4jB3hD7Z--