Date: Fri, 30 Aug 2019 14:17:20 +0200 From: Alexander Leidinger <Alexander@leidinger.net> To: "O. Hartmann" <ohartmann@walstatt.org> Cc: freebsd-current@freebsd.org, Michael Zhilin <mizhka@gmail.com> Subject: Re: jails, ZFS, deprecated jail variables and poudriere problems Message-ID: <20190830141720.Horde.yKnzT-Dd2OP1_BMC2-svTaC@webmail.leidinger.net> In-Reply-To: <20190829142632.6b93892b@freyja> References: <20190827101149.1efcb946@freyja> <20190828135700.Horde.m2EjpS9j67KYn2E1oSeoK8f@webmail.leidinger.net> <20190829142632.6b93892b@freyja>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format and has been PGP signed.
--=_6KdtFXi7O3zedMNg8yM4xn9
Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Quoting "O. Hartmann" <ohartmann@walstatt.org> (from Thu, 29 Aug 2019=20=20
14:26:38=20+0200):
> On Wed, 28 Aug 2019 13:57:00 +0200
> Alexander Leidinger <Alexander@leidinger.net> wrote:
>
>> Quoting "O. Hartmann" <ohartmann@walstatt.org> (from Tue, 27 Aug 2019
>> 10:11:54 +0200):
>>
>> > We have a single ZFS pool (raidz), call it pool00 and this pool00=20=
=20
>>=20conatins a
>> > ZFS dataset pool00/poudriere which we want to exclusively attach=20=20
>>=20to a jail.
>> > pool00/poudriere contains a complete clone of a former, now decomissio=
ned
>> > machine and is usable by the host bearing the jails. The jail, named
>> > poudriere,
>> > has these config parameters set in /etc/jail.conf as recommended:
>> >
>> > enforce_statfs=3D "0";
>
> now set to
> enforce_statfs=3D "1";
From a security point of view this is off course better, but "0"=20=20
should=20have been OK for your problem case.
[...]
>> > Here I find the first confusing observation. I can't interact with
>> > the dataset
>> > and its content within the jail. I've set the "jailed" property of
>> > pool00/poudriere via "zfs set jailed=3Don pool00/poudriere" and I=20=
=20
>>=20also have to
>> > attach the jailed dataset manually via "zfs jail poudriere
>> > pool00/poudriere" to
>> > the (running) jail. But within the jail, listing ZFS's mountpoints rev=
eal:
>> >
>> > NAME USED AVAIL REFER MOUNTPOINT
>> > pool00 124G 8.62T 34.9K /pool00
>> > pool00/poudriere 34.9K 8.62T 34.9K /pool/poudriere
>> >
>> > but nothing below /pool/poudriere is visible to the jail. Being confus=
ed I
>
> Since we use ezjail-admin for jail a rudimentary jail administration (jus=
t
> creating and/or deleting the jail, maintenance is done manually), jails a=
re
> rooting at
>
> pool00 /pool00
> pool00/ezjail/ /pool/jails
> pool00/ezjail/pulverfass /pool/jails/pulverfass
>
> "pulverfass" is the jail supposed to do the poudriere's job.
>
> Since I got confused about the orientation of the "directory tree" - the =
root
> is toplevel instead of downlevel - I corrected the ZFS dataset holding th=
e
> poudriere stuff that way:
>
> pool00/ezjail/poudriere /pool/poudriere
>
> The jail "pulverfass" now is supposed to mount the dataset at
>
> /pool/jails/pulverfass/pool/poudriere
>
>>
>> Please be more verbose what you mean by "interact" and "is visible".
>>
>> Do zfs commands on the dataset work?
>
> After I corrected my mistake by not respecting the mountpoint according t=
o
> statfs, with the changes explained above I'm able to mount /pool/poudrier=
e
> within the jail "pulverfass", but I still have problems with the way=20=
=20
>=20how I have
> to mount this dataset. When zfs-mounted (zfs mount -a), I'm able to use t=
he
> dataset with poudriere as expected! But after rebooting the host and=20=
=20
>=20after all
> jails has been restarted as well, I have to first make the dataset
> /pool/poudriere available to the jail via the command "zfs jail pulverfas=
s
> ool00/ezjail/pulverfass" - which seems not to be done automatically by th=
e
> startup process - and then from within the jail "pulverfass" I can mount =
the
> dataset as desribed above. This seems to be a big step ahead for me.
Great.
>> Note, I don't remember if you can manage the root of the jail, but at
>> least subsequent jails should be possible to manage. I don't have a
>> jail where the root is managed in the jail, just additional ones.
>> Those need to have set a mountpoint after the initial jailing and then
>> maybe even be mounted for the first time.
>>
>> Please also check /etc/defaults/devfs.rules if the jail rule contains
>> an unhide entry for zfs.
>
> Within /etc/jail.conf
>
> devfs_ruleset=3D "4";
>
> is configured as a common rulesset for all jails (in the common portion o=
f
> /etc/jail.conf).
> There is no custom devfs.rules in /etc/, so /etc/defaults/devfs.rules sho=
uld
> apply and as far as I can see, there is an "unhide" applied to zfs:
>
> [... /etc/defaults/devfs.rulse ...]
>
> # Devices usually found in a jail.
> #
> [devfsrules_jail=3D4]
> add include $devfsrules_hide_all
> add include $devfsrules_unhide_basic
> add include $devfsrules_unhide_login
> add path fuse unhide
> add path zfs unhide
> [...]
>
> So, I guess everything is all right from this perspective, isn't it?
Yes. And as you was able to do the zfs mount, you even had the proof.
> Is there a way to automatically provide the ZFS dataset of choice to the
> propper jail or do i have to either
>
> issue manually "zfs jail jailid/jailname pool/dataset" or put such a=20=
=20
>=20command as
> script-command in the jail's definition portion as
>
> exec.prestart+=3D "zfs jail ${name} pool00/ezjail/poudriere";
> ?
I never tried that with plain jails. ezjail has the=20=20
jail_xxx_zfs_datasets=20variable in the config directory. Unfortunately=20=
=20
the=20attachment of the datasets to the jail is not early enough to be=20=
=20
picked=20up by the start scripts (so probably with exec.poststart, if=20=20
you=20use that you can also use it to launch "jexec ... service zfs=20=20
start"=20inside the jail). iocage works much better in this regard.
Bye,
Alexander.
--=20
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF
--=_6KdtFXi7O3zedMNg8yM4xn9
Content-Type: application/pgp-signature
Content-Description: Digitale PGP-Signatur
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJdaRPQAAoJEBINsJsD+NiGHs4P/ice82BNIeRlfwbQxzXjdVKL
gH8HyrPPH9glfe20Pq/ByawoZhU+lEbayQUI0zzxVsC7VqCYYYI/DAWWEVZPVCkI
XiKTdVt6+fjrtd2cnFcf6c1SWncxIKmO98Qe8fXBmPPp4Q+OcT9XhWEtlM/3SGxa
vipXQvTVNEVaiuexKH7pa9QXEsyQ30YzQ58VTX12UNOue3LzoUsOu3QRgJGBdCzh
9Fd+G9+KOHurTfzLm9I1PAFmxQ7flwKT9pgh40q64I0/3M8hKDSfX/pqS/A+bz9O
9bdHnmbPlHQTrZ5YNbC/aWyg49pmGZWKbd6z1EGl+NYdQRjv61Xnuf7+d0WaOJ/6
Ulp5OIQ1VjLX6NxuMSvk6APWn4n5HwboO316WRTpIyDO+qf2KBztEi7Z1grvHIm5
qcI3D/YaDSjfOH90qYKp7pFwhP66ZHxcg2/1gI9HMBrunt/xcNpsibxsRf03fMX6
ECxKTfXgR1yxEQNuczLtllBMeWER6K0d6UWqoaXtRFYlGsgAHkUb4OneqavRiFhp
TGRgb22BdVKlwCMdt0TtY+zn8YEUxIqUBNe7LhySM6j9rhRnh0mzQKEW85HxY75c
AVHAd0pJJv1hB0DDfYmEj4GLWqwa5Yzk0m1krSlr/SmPUi+EG4Xaq3hAsG+dQ10V
qu2cceo050kGtJUzG8wK
=w+JP
-----END PGP SIGNATURE-----
--=_6KdtFXi7O3zedMNg8yM4xn9--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190830141720.Horde.yKnzT-Dd2OP1_BMC2-svTaC>