Date: Tue, 21 Jun 2011 15:59:06 -0400 From: jhell <jhell@DataIX.net> To: Paul Schenkeveld <freebsd@psconsult.nl> Cc: freebsd-jail@freebsd.org Subject: Re: Exposing a hierarchy of ZFS datasets inside multiple jails Message-ID: <20110621195906.GA62312@DataIX.net> In-Reply-To: <20110621185122.GA13459@psconsult.nl> References: <BANLkTikrWYnBAnQsXZ535OdX5tVp9eOrNQ@mail.gmail.com> <20110621185122.GA13459@psconsult.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jun 21, 2011 at 08:51:22PM +0200, Paul Schenkeveld wrote: > Hi, > > On Fri, Jun 17, 2011 at 02:46:59PM -0400, Lars Kellogg-Stedman wrote: > > Hello all, > > > > Hi there, > > > > I am trying to expose a hierarchy of home directories to a number of > > FreeBSD jails. The home directories are configured such that each is a > > unique ZFS dataset. The jails are used for development work and hence > > are created and destroyed on a regular basis. > > > > My first thought was simply to use nullfs to mount /home inside the > > jail, but nullfs doesn't provide any way to access subordinate > > filesystems. > > > > My second thought was to export the directories via NFS and then run > > the automounter daemon (amd) inside each jail. This would have Just > > Worked...if it were possible to perform NFS mounts inside a jail. But > > it's not. > > > > My third thought was to run amd on the host and provision nullfs > > mounts into the jails...but amd support for nullfs doesn't exist. > > > > My fourth thought was to go back to exporting the directories using > > NFS, because of course amd works with NFS, right? Unfortunately, > > rather than mounting a directory on the target mountpoint, amd likes > > to mount things in a temporary location (/.amd_mnt/...) and then > > create a symlink...which, of course, is useless inside the jail > > environment.t > > > > So maybe you could use nullfs to expose a subdirectory of /.amd_mnt to > > the jail? No! This brings us back to my first attempt, in which we > > find that there is no way to access subordinate filesystems using > > nullfs. > > > > And then my head exploded. > > > > Is there a good solution for what I'm trying to do? A bad solution > > would be to run a script after booting the jail that would create > > multiple nullfs mountpoints for all the home directories, but this is > > pretty clunky -- it would need to be run periodically to take into > > account new directories or removed directories. So basically I would > > have to write a poorly designed automounter. > > > > There must be a better way. How are other folks solving this? > > > > It looks like there are discussions going back several years about > > setting the VFCF_JAIL on NFS filesystems, but it these haven't > > resulted in any changes to the released code. Is this the best way to > > go? In theory, if I build a kernel under which NFS is jail friendly I > > can go ahead and run amd inside the jail > > Probably not a good solution but to stir the pool of thoughts a bit... > > Nullfs mounts and NFS mounts operate on filesystems (or datasets) and > do not include subordinates. Smbfs operates on directory (sub)trees > so have /home and /home/user[123...] datasets outside the jails, run > samba there with a share called [home] (not to be confused with the > [homes] share that comes with smb.conf.sample) and mount this share > using mount_smbfs inside every jail (from fstab.<jailname>). > mount_smbfs needing rpcbind for the host that jails are being hosted on and also rpcbind for each jail, your going to run into a problem here due to the fact that rpcbind listens on the <anyaddress>*:N without going in and making some specicial modifications for each jail and the host itself by editing code and recompiling for each jail. This is a known problem with RPC type services and jails and will only escalate by the number of jails that you would have to implement this on. Instead of making each userdir a dataset I would suggest just creating them as regular directories within a base dataset if this is the approach you would like to take with nullfs.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110621195906.GA62312>