Date: Fri, 30 Aug 2019 14:17:20 +0200 From: Alexander Leidinger <Alexander@leidinger.net> To: "O. Hartmann" <ohartmann@walstatt.org> Cc: freebsd-current@freebsd.org, Michael Zhilin <mizhka@gmail.com> Subject: Re: jails, ZFS, deprecated jail variables and poudriere problems Message-ID: <20190830141720.Horde.yKnzT-Dd2OP1_BMC2-svTaC@webmail.leidinger.net> In-Reply-To: <20190829142632.6b93892b@freyja> References: <20190827101149.1efcb946@freyja> <20190828135700.Horde.m2EjpS9j67KYn2E1oSeoK8f@webmail.leidinger.net> <20190829142632.6b93892b@freyja>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Quoting "O. Hartmann" <ohartmann@walstatt.org> (from Thu, 29 Aug 2019
14:26:38 +0200):
> On Wed, 28 Aug 2019 13:57:00 +0200
> Alexander Leidinger <Alexander@leidinger.net> wrote:
>
>> Quoting "O. Hartmann" <ohartmann@walstatt.org> (from Tue, 27 Aug 2019
>> 10:11:54 +0200):
>>
>> > We have a single ZFS pool (raidz), call it pool00 and this pool00
>> conatins a
>> > ZFS dataset pool00/poudriere which we want to exclusively attach
>> to a jail.
>> > pool00/poudriere contains a complete clone of a former, now decomissioned
>> > machine and is usable by the host bearing the jails. The jail, named
>> > poudriere,
>> > has these config parameters set in /etc/jail.conf as recommended:
>> >
>> > enforce_statfs= "0";
>
> now set to
> enforce_statfs= "1";
From a security point of view this is off course better, but "0"
should have been OK for your problem case.
[...]
>> > Here I find the first confusing observation. I can't interact with
>> > the dataset
>> > and its content within the jail. I've set the "jailed" property of
>> > pool00/poudriere via "zfs set jailed=on pool00/poudriere" and I
>> also have to
>> > attach the jailed dataset manually via "zfs jail poudriere
>> > pool00/poudriere" to
>> > the (running) jail. But within the jail, listing ZFS's mountpoints reveal:
>> >
>> > NAME USED AVAIL REFER MOUNTPOINT
>> > pool00 124G 8.62T 34.9K /pool00
>> > pool00/poudriere 34.9K 8.62T 34.9K /pool/poudriere
>> >
>> > but nothing below /pool/poudriere is visible to the jail. Being confused I
>
> Since we use ezjail-admin for jail a rudimentary jail administration (just
> creating and/or deleting the jail, maintenance is done manually), jails are
> rooting at
>
> pool00 /pool00
> pool00/ezjail/ /pool/jails
> pool00/ezjail/pulverfass /pool/jails/pulverfass
>
> "pulverfass" is the jail supposed to do the poudriere's job.
>
> Since I got confused about the orientation of the "directory tree" - the root
> is toplevel instead of downlevel - I corrected the ZFS dataset holding the
> poudriere stuff that way:
>
> pool00/ezjail/poudriere /pool/poudriere
>
> The jail "pulverfass" now is supposed to mount the dataset at
>
> /pool/jails/pulverfass/pool/poudriere
>
>>
>> Please be more verbose what you mean by "interact" and "is visible".
>>
>> Do zfs commands on the dataset work?
>
> After I corrected my mistake by not respecting the mountpoint according to
> statfs, with the changes explained above I'm able to mount /pool/poudriere
> within the jail "pulverfass", but I still have problems with the way
> how I have
> to mount this dataset. When zfs-mounted (zfs mount -a), I'm able to use the
> dataset with poudriere as expected! But after rebooting the host and
> after all
> jails has been restarted as well, I have to first make the dataset
> /pool/poudriere available to the jail via the command "zfs jail pulverfass
> ool00/ezjail/pulverfass" - which seems not to be done automatically by the
> startup process - and then from within the jail "pulverfass" I can mount the
> dataset as desribed above. This seems to be a big step ahead for me.
Great.
>> Note, I don't remember if you can manage the root of the jail, but at
>> least subsequent jails should be possible to manage. I don't have a
>> jail where the root is managed in the jail, just additional ones.
>> Those need to have set a mountpoint after the initial jailing and then
>> maybe even be mounted for the first time.
>>
>> Please also check /etc/defaults/devfs.rules if the jail rule contains
>> an unhide entry for zfs.
>
> Within /etc/jail.conf
>
> devfs_ruleset= "4";
>
> is configured as a common rulesset for all jails (in the common portion of
> /etc/jail.conf).
> There is no custom devfs.rules in /etc/, so /etc/defaults/devfs.rules should
> apply and as far as I can see, there is an "unhide" applied to zfs:
>
> [... /etc/defaults/devfs.rulse ...]
>
> # Devices usually found in a jail.
> #
> [devfsrules_jail=4]
> add include $devfsrules_hide_all
> add include $devfsrules_unhide_basic
> add include $devfsrules_unhide_login
> add path fuse unhide
> add path zfs unhide
> [...]
>
> So, I guess everything is all right from this perspective, isn't it?
Yes. And as you was able to do the zfs mount, you even had the proof.
> Is there a way to automatically provide the ZFS dataset of choice to the
> propper jail or do i have to either
>
> issue manually "zfs jail jailid/jailname pool/dataset" or put such a
> command as
> script-command in the jail's definition portion as
>
> exec.prestart+= "zfs jail ${name} pool00/ezjail/poudriere";
> ?
I never tried that with plain jails. ezjail has the
jail_xxx_zfs_datasets variable in the config directory. Unfortunately
the attachment of the datasets to the jail is not early enough to be
picked up by the start scripts (so probably with exec.poststart, if
you use that you can also use it to launch "jexec ... service zfs
start" inside the jail). iocage works much better in this regard.
Bye,
Alexander.
--
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJdaRPQAAoJEBINsJsD+NiGHs4P/ice82BNIeRlfwbQxzXjdVKL
gH8HyrPPH9glfe20Pq/ByawoZhU+lEbayQUI0zzxVsC7VqCYYYI/DAWWEVZPVCkI
XiKTdVt6+fjrtd2cnFcf6c1SWncxIKmO98Qe8fXBmPPp4Q+OcT9XhWEtlM/3SGxa
vipXQvTVNEVaiuexKH7pa9QXEsyQ30YzQ58VTX12UNOue3LzoUsOu3QRgJGBdCzh
9Fd+G9+KOHurTfzLm9I1PAFmxQ7flwKT9pgh40q64I0/3M8hKDSfX/pqS/A+bz9O
9bdHnmbPlHQTrZ5YNbC/aWyg49pmGZWKbd6z1EGl+NYdQRjv61Xnuf7+d0WaOJ/6
Ulp5OIQ1VjLX6NxuMSvk6APWn4n5HwboO316WRTpIyDO+qf2KBztEi7Z1grvHIm5
qcI3D/YaDSjfOH90qYKp7pFwhP66ZHxcg2/1gI9HMBrunt/xcNpsibxsRf03fMX6
ECxKTfXgR1yxEQNuczLtllBMeWER6K0d6UWqoaXtRFYlGsgAHkUb4OneqavRiFhp
TGRgb22BdVKlwCMdt0TtY+zn8YEUxIqUBNe7LhySM6j9rhRnh0mzQKEW85HxY75c
AVHAd0pJJv1hB0DDfYmEj4GLWqwa5Yzk0m1krSlr/SmPUi+EG4Xaq3hAsG+dQ10V
qu2cceo050kGtJUzG8wK
=w+JP
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190830141720.Horde.yKnzT-Dd2OP1_BMC2-svTaC>