From owner-freebsd-jail@FreeBSD.ORG Mon Jun 20 11:07:05 2011 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 498B71065676 for ; Mon, 20 Jun 2011 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 389978FC17 for ; Mon, 20 Jun 2011 11:07:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p5KB75tj098158 for ; Mon, 20 Jun 2011 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p5KB74LN098156 for freebsd-jail@FreeBSD.org; Mon, 20 Jun 2011 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 20 Jun 2011 11:07:04 GMT Message-Id: <201106201107.p5KB74LN098156@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-jail@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 11:07:05 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/156111 jail [jail] procstat -b not supported in jail o misc/155765 jail [patch] `buildworld' does not honors WITHOUT_JAIL o conf/154246 jail [jail] [patch] Bad symlink created if devfs mount poin o conf/149050 jail [jail] rcorder ``nojail'' too coarse for Jail+VNET s conf/142972 jail [jail] [patch] Support JAILv2 and vnet in rc.d/jail o conf/141317 jail [patch] uncorrect jail stop in /etc/rc.d/jail o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid o bin/32828 jail [jail] w(1) incorrectly handles stale utmp slots with 10 problems total. From owner-freebsd-jail@FreeBSD.ORG Tue Jun 21 01:36:06 2011 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CB141065675 for ; Tue, 21 Jun 2011 01:36:06 +0000 (UTC) (envelope-from lars@oddbit.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5AC4E8FC13 for ; Tue, 21 Jun 2011 01:36:06 +0000 (UTC) Received: by iwr19 with SMTP id 19so4841281iwr.13 for ; Mon, 20 Jun 2011 18:36:06 -0700 (PDT) Received: by 10.231.29.132 with SMTP id q4mr5612266ibc.169.1308618586310; Mon, 20 Jun 2011 18:09:46 -0700 (PDT) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx.google.com with ESMTPS id p15sm3507667ibh.63.2011.06.20.18.09.44 (version=SSLv3 cipher=OTHER); Mon, 20 Jun 2011 18:09:44 -0700 (PDT) Received: by iwr19 with SMTP id 19so4823349iwr.13 for ; Mon, 20 Jun 2011 18:09:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.42.162.194 with SMTP id z2mr4415193icx.79.1308618584364; Mon, 20 Jun 2011 18:09:44 -0700 (PDT) Received: by 10.231.39.137 with HTTP; Mon, 20 Jun 2011 18:09:44 -0700 (PDT) In-Reply-To: <4DFC7B0C.6040205@freenas.org> References: <4DFC7B0C.6040205@freenas.org> Date: Mon, 20 Jun 2011 21:09:44 -0400 Message-ID: From: Lars Kellogg-Stedman To: Christian Degen Content-Type: text/plain; charset=UTF-8 Cc: freebsd-jail@freebsd.org Subject: Re: Exposing a hierarchy of ZFS datasets inside multiple jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2011 01:36:06 -0000 Christian, Thank you for your answer. I've read through your email a few times, and I think that there is a problem with your suggestion. Please let me know if I've misunderstood something. If I create a ZFS dataset: zfs create tank/nullfs And then mount this somewhere else via nullfs: mount_nullfs /tank/nullfs /mnt/nfs_home And then create a new ZFS dataset below tank/nullfs: zfs create tank/nullfs/user1 And then create some files in that dataset: $ touch /tank/nullfs/user1/file1 $ touch /tank/nullfs/user1/file2 $ find /tank/nullfs/user1 /tank/nullfs/user1 /tank/nullfs/user1/file1 /tank/nullfs/user1/file2 The only thing I will find in /mnt/nfs_home is an empty directory named "user1": $ find /mnt/nfs_home /mnt/nfs_home/ /mnt/nfs_home/user1 The nullfs mount of /tank/nullfs to /mnt/nfs_home only exposes files and directories contained in the "nullfs" dataset, but not in any subordinate datasets. This is exactly my original problem (otherwise I would simply have nullfs mounted /home inside my jails). > teufelchen# zfs create tank/nullfs/dataset1 > teufelchen# touch /mnt/tank/nfs_home/dataset1/newfile > teufelchen# jexec 14 ls /mnt/nfs_home/dataset1/ > newfile > > Is this what you are trying todo? I think that there may be a problem with your example here. When you run: teufelchen# touch /mnt/tank/nfs_home/dataset1/newfile ...I don't think you're creating the file where you think you are. Take a look at /mnt/tank/nullfs/dataset1; I suspect you won't find it there. What you've accomplished is to use the "nfs_home" dataset exclusively, ignoring any subordinate datasets. That is, you're treating it like you do this: zfs create tank/nullfs mkdir /mnt/tank/nullfs/dataset1 Rather than: zfs create tank/nullfs zfs create tank/nullfs/dataset1 The difference is crucial to this problem. Cheers, -- Lars From owner-freebsd-jail@FreeBSD.ORG Tue Jun 21 18:51:29 2011 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C2DD1065690 for ; Tue, 21 Jun 2011 18:51:29 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 2E06F8FC1D for ; Tue, 21 Jun 2011 18:51:28 +0000 (UTC) Received: from mx1.psconsult.nl ([80.89.238.138]) by mx1.psconsult.nl (8.14.4/8.14.4) with ESMTP id p5LIpMVv014074 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 21 Jun 2011 20:51:27 +0200 (CEST) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.4/8.14.4/Submit) id p5LIpMCh014073 for freebsd-jail@freebsd.org; Tue, 21 Jun 2011 20:51:22 +0200 (CEST) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Tue, 21 Jun 2011 20:51:22 +0200 From: Paul Schenkeveld To: freebsd-jail@freebsd.org Message-ID: <20110621185122.GA13459@psconsult.nl> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: Exposing a hierarchy of ZFS datasets inside multiple jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2011 18:51:29 -0000 Hi, On Fri, Jun 17, 2011 at 02:46:59PM -0400, Lars Kellogg-Stedman wrote: > Hello all, > > Hi there, > > I am trying to expose a hierarchy of home directories to a number of > FreeBSD jails. The home directories are configured such that each is a > unique ZFS dataset. The jails are used for development work and hence > are created and destroyed on a regular basis. > > My first thought was simply to use nullfs to mount /home inside the > jail, but nullfs doesn't provide any way to access subordinate > filesystems. > > My second thought was to export the directories via NFS and then run > the automounter daemon (amd) inside each jail. This would have Just > Worked...if it were possible to perform NFS mounts inside a jail. But > it's not. > > My third thought was to run amd on the host and provision nullfs > mounts into the jails...but amd support for nullfs doesn't exist. > > My fourth thought was to go back to exporting the directories using > NFS, because of course amd works with NFS, right? Unfortunately, > rather than mounting a directory on the target mountpoint, amd likes > to mount things in a temporary location (/.amd_mnt/...) and then > create a symlink...which, of course, is useless inside the jail > environment.t > > So maybe you could use nullfs to expose a subdirectory of /.amd_mnt to > the jail? No! This brings us back to my first attempt, in which we > find that there is no way to access subordinate filesystems using > nullfs. > > And then my head exploded. > > Is there a good solution for what I'm trying to do? A bad solution > would be to run a script after booting the jail that would create > multiple nullfs mountpoints for all the home directories, but this is > pretty clunky -- it would need to be run periodically to take into > account new directories or removed directories. So basically I would > have to write a poorly designed automounter. > > There must be a better way. How are other folks solving this? > > It looks like there are discussions going back several years about > setting the VFCF_JAIL on NFS filesystems, but it these haven't > resulted in any changes to the released code. Is this the best way to > go? In theory, if I build a kernel under which NFS is jail friendly I > can go ahead and run amd inside the jail Probably not a good solution but to stir the pool of thoughts a bit... Nullfs mounts and NFS mounts operate on filesystems (or datasets) and do not include subordinates. Smbfs operates on directory (sub)trees so have /home and /home/user[123...] datasets outside the jails, run samba there with a share called [home] (not to be confused with the [homes] share that comes with smb.conf.sample) and mount this share using mount_smbfs inside every jail (from fstab.). Just my $.02 Regards, Paul Schenkeveld From owner-freebsd-jail@FreeBSD.ORG Tue Jun 21 20:21:11 2011 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A2931065670 for ; Tue, 21 Jun 2011 20:21:11 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id EB0028FC17 for ; Tue, 21 Jun 2011 20:21:10 +0000 (UTC) Received: by iwr19 with SMTP id 19so165125iwr.13 for ; Tue, 21 Jun 2011 13:21:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to; bh=gvUu3gg98KSnFkBX0XS15QP5Q/5mvfHwUix8mBIXiC4=; b=Dnn9IXQiOeZmk8shnrEf+rnJEZ5UQmltMbCVX2HkDLzmwRs7c1kuO+s5ldRY3B7X3m JrF6WWq9uRjhYCtAFmkgSw1+/o5W0Hrbdhpd5oHlGrwgzWVIT9yfnZdX3mjLrxqrcLBf 6Dm9mnBczQMf2lMxAkJw65CTrm5lPNnFhTRTQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; b=S0nan9mt8tBK+GTN6MTBICLnwyjxto6JwXAzuILYsopYQOC8RYjrceLZPWvWkYBBIc ecDPxJqnNl96SXGaYgseknN+mlefbYkPQ5+02U+BKfTdDlzxBsVUZXKnSdUB5Ase/F5O 0aUe4ANbrFXaOgbYgM33zItiSFEHl9J0jGBvk= Received: by 10.42.29.131 with SMTP id r3mr7778774icc.377.1308686353212; Tue, 21 Jun 2011 12:59:13 -0700 (PDT) Received: from disbatch.dataix.local (adsl-99-19-44-84.dsl.klmzmi.sbcglobal.net [99.19.44.84]) by mx.google.com with ESMTPS id v16sm3963770ibe.34.2011.06.21.12.59.10 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 21 Jun 2011 12:59:11 -0700 (PDT) Sender: "J. Hellenthal" Received: from disbatch.dataix.local (localhost [127.0.0.1]) by disbatch.dataix.local (8.14.5/8.14.5) with ESMTP id p5LJx7uq048268 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jun 2011 15:59:08 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by disbatch.dataix.local (8.14.5/8.14.5/Submit) id p5LJx7Xl048267; Tue, 21 Jun 2011 15:59:07 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Tue, 21 Jun 2011 15:59:06 -0400 From: jhell To: Paul Schenkeveld Message-ID: <20110621195906.GA62312@DataIX.net> References: <20110621185122.GA13459@psconsult.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110621185122.GA13459@psconsult.nl> Cc: freebsd-jail@freebsd.org Subject: Re: Exposing a hierarchy of ZFS datasets inside multiple jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2011 20:21:11 -0000 On Tue, Jun 21, 2011 at 08:51:22PM +0200, Paul Schenkeveld wrote: > Hi, > > On Fri, Jun 17, 2011 at 02:46:59PM -0400, Lars Kellogg-Stedman wrote: > > Hello all, > > > > Hi there, > > > > I am trying to expose a hierarchy of home directories to a number of > > FreeBSD jails. The home directories are configured such that each is a > > unique ZFS dataset. The jails are used for development work and hence > > are created and destroyed on a regular basis. > > > > My first thought was simply to use nullfs to mount /home inside the > > jail, but nullfs doesn't provide any way to access subordinate > > filesystems. > > > > My second thought was to export the directories via NFS and then run > > the automounter daemon (amd) inside each jail. This would have Just > > Worked...if it were possible to perform NFS mounts inside a jail. But > > it's not. > > > > My third thought was to run amd on the host and provision nullfs > > mounts into the jails...but amd support for nullfs doesn't exist. > > > > My fourth thought was to go back to exporting the directories using > > NFS, because of course amd works with NFS, right? Unfortunately, > > rather than mounting a directory on the target mountpoint, amd likes > > to mount things in a temporary location (/.amd_mnt/...) and then > > create a symlink...which, of course, is useless inside the jail > > environment.t > > > > So maybe you could use nullfs to expose a subdirectory of /.amd_mnt to > > the jail? No! This brings us back to my first attempt, in which we > > find that there is no way to access subordinate filesystems using > > nullfs. > > > > And then my head exploded. > > > > Is there a good solution for what I'm trying to do? A bad solution > > would be to run a script after booting the jail that would create > > multiple nullfs mountpoints for all the home directories, but this is > > pretty clunky -- it would need to be run periodically to take into > > account new directories or removed directories. So basically I would > > have to write a poorly designed automounter. > > > > There must be a better way. How are other folks solving this? > > > > It looks like there are discussions going back several years about > > setting the VFCF_JAIL on NFS filesystems, but it these haven't > > resulted in any changes to the released code. Is this the best way to > > go? In theory, if I build a kernel under which NFS is jail friendly I > > can go ahead and run amd inside the jail > > Probably not a good solution but to stir the pool of thoughts a bit... > > Nullfs mounts and NFS mounts operate on filesystems (or datasets) and > do not include subordinates. Smbfs operates on directory (sub)trees > so have /home and /home/user[123...] datasets outside the jails, run > samba there with a share called [home] (not to be confused with the > [homes] share that comes with smb.conf.sample) and mount this share > using mount_smbfs inside every jail (from fstab.). > mount_smbfs needing rpcbind for the host that jails are being hosted on and also rpcbind for each jail, your going to run into a problem here due to the fact that rpcbind listens on the *:N without going in and making some specicial modifications for each jail and the host itself by editing code and recompiling for each jail. This is a known problem with RPC type services and jails and will only escalate by the number of jails that you would have to implement this on. Instead of making each userdir a dataset I would suggest just creating them as regular directories within a base dataset if this is the approach you would like to take with nullfs.