Date: Thu, 29 Aug 2019 14:26:38 +0200 From: "O. Hartmann" <ohartmann@walstatt.org> To: Alexander Leidinger <Alexander@leidinger.net> Cc: freebsd-current@freebsd.org, Michael Zhilin <mizhka@gmail.com> Subject: Re: jails, ZFS, deprecated jail variables and poudriere problems Message-ID: <20190829142632.6b93892b@freyja> In-Reply-To: <20190828135700.Horde.m2EjpS9j67KYn2E1oSeoK8f@webmail.leidinger.net> References: <20190827101149.1efcb946@freyja> <20190828135700.Horde.m2EjpS9j67KYn2E1oSeoK8f@webmail.leidinger.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 28 Aug 2019 13:57:00 +0200
Alexander Leidinger <Alexander@leidinger.net> wrote:
> Quoting "O. Hartmann" <ohartmann@walstatt.org> (from Tue, 27 Aug 2019
> 10:11:54 +0200):
>
> > We have a single ZFS pool (raidz), call it pool00 and this pool00 cona=
tins a
> > ZFS dataset pool00/poudriere which we want to exclusively attach to a =
jail.
> > pool00/poudriere contains a complete clone of a former, now decomissio=
ned
> > machine and is usable by the host bearing the jails. The jail, named
> > poudriere,
> > has these config parameters set in /etc/jail.conf as recommended:
> >
> > enforce_statfs=3D "0";
now set to
enforce_statfs=3D "1";
> >
> > allow.raw_sockets=3D "1";
> >
> > allow.mount=3D "1";
> > allow.mount.zfs=3D "1";
>
> The line above is what is needed, and what is replacing the sysctl
> you've found.
>
> > allow.mount.devfs=3D "1";
> > allow.mount.fdescfs=3D "1";
> > allow.mount.procfs=3D "1";
> > allow.mount.nullfs=3D "1";
> > allow.mount.fusefs=3D "1";
... and those extended with these
allow.mount.tmpfs=3D "1";
allow.mount.linprocfs=3D "1";
> >
> > Here I find the first confusing observation. I can't interact with
> > the dataset
> > and its content within the jail. I've set the "jailed" property of
> > pool00/poudriere via "zfs set jailed=3Don pool00/poudriere" and I also=
have to
> > attach the jailed dataset manually via "zfs jail poudriere
> > pool00/poudriere" to
> > the (running) jail. But within the jail, listing ZFS's mountpoints rev=
eal:
> >
> > NAME USED AVAIL REFER MOUNTPOINT
> > pool00 124G 8.62T 34.9K /pool00
> > pool00/poudriere 34.9K 8.62T 34.9K /pool/poudriere
> >
> > but nothing below /pool/poudriere is visible to the jail. Being confus=
ed I
Since we use ezjail-admin for jail a rudimentary jail administration (just
creating and/or deleting the jail, maintenance is done manually), jails ar=
e
rooting at
pool00 /pool00
pool00/ezjail/ /pool/jails
pool00/ezjail/pulverfass /pool/jails/pulverfass
"pulverfass" is the jail supposed to do the poudriere's job.
Since I got confused about the orientation of the "directory tree" - the r=
oot
is toplevel instead of downlevel - I corrected the ZFS dataset holding the
poudriere stuff that way:
pool00/ezjail/poudriere /pool/poudriere
The jail "pulverfass" now is supposed to mount the dataset at
/pool/jails/pulverfass/pool/poudriere
>
> Please be more verbose what you mean by "interact" and "is visible".
>
> Do zfs commands on the dataset work?
After I corrected my mistake by not respecting the mountpoint according to
statfs, with the changes explained above I'm able to mount /pool/poudriere
within the jail "pulverfass", but I still have problems with the way how I=
have
to mount this dataset. When zfs-mounted (zfs mount -a), I'm able to use th=
e
dataset with poudriere as expected! But after rebooting the host and after=
all
jails has been restarted as well, I have to first make the dataset
/pool/poudriere available to the jail via the command "zfs jail pulverfass
ool00/ezjail/pulverfass" - which seems not to be done automatically by the
startup process - and then from within the jail "pulverfass" I can mount t=
he
dataset as desribed above. This seems to be a big step ahead for me.
>
> Note, I don't remember if you can manage the root of the jail, but at
> least subsequent jails should be possible to manage. I don't have a
> jail where the root is managed in the jail, just additional ones.
> Those need to have set a mountpoint after the initial jailing and then
> maybe even be mounted for the first time.
>
> Please also check /etc/defaults/devfs.rules if the jail rule contains
> an unhide entry for zfs.
Within /etc/jail.conf
devfs_ruleset=3D "4";
is configured as a common rulesset for all jails (in the common portion of
/etc/jail.conf).
There is no custom devfs.rules in /etc/, so /etc/defaults/devfs.rules shou=
ld
apply and as far as I can see, there is an "unhide" applied to zfs:
[... /etc/defaults/devfs.rulse ...]
# Devices usually found in a jail.
#
[devfsrules_jail=3D4]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path fuse unhide
add path zfs unhide
[...]
So, I guess everything is all right from this perspective, isn't it?
Is there a way to automatically provide the ZFS dataset of choice to the
propper jail or do i have to either
issue manually "zfs jail jailid/jailname pool/dataset" or put such a comma=
nd as
script-command in the jail's definition portion as
exec.prestart+=3D "zfs jail ${name} pool00/ezjail/poudriere";
?
>
> Bye,
> Alexander.
>
Thank you very much and kind regards,
oh
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190829142632.6b93892b>