Date: Mon, 06 Nov 2023 20:35:06 +0000 From: DtxdF <DtxdF@disroot.org> To: freebsd-jail@freebsd.org Subject: =?US-ASCII?Q?Re=3A_I_can_get_zfs_snapshot/rollback_in_a_jail_to_work_?= =?US-ASCII?Q?99=25_but_it_isn=27t_quite_100=25?= =?US-ASCII?Q?_working=2E_What_am_I_missing=3F?= Message-ID: <A696936B-D83F-4B7B-B037-F51CF47C44A9@disroot.org> In-Reply-To: <CAHnbxSRuwGwdB0PULA4LEKTWTJw6oA6VeX_7PL4TMgAv_YS=vw@mail.gmail.com> References: <CAHnbxSRuwGwdB0PULA4LEKTWTJw6oA6VeX_7PL4TMgAv_YS=vw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------BSOZDAK4XRKBNCKWL0CKWPTSAUYJFS
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi Chris,
Maybe your dataset is not mounted inside the jail=2E I thought that simply=
enabling `/etc/rc=2Ed/zfs` was fine, but no, it just doesn't work=2E I don=
't know if this behavior is a bug or something else, but at the moment I do=
n't have time to investigate=2E
I have a similar setup for a jail with a delegated dataset=2E I use AppJai=
l, but the steps are similar to other tools:
```
# zfs create -o jailed=3Don -o mountpoint=3D/jailed zroot/jailed
# appjail quick jtest \
mount_devfs \
device=3D'include $devfsrules_hide_all' \
device=3D'include $devfsrules_unhide_basic' \
device=3D'include $devfsrules_unhide_login' \
device=3D'path zfs unhide' \
template=3Dtemplate=2Econf \
overwrite=3Dforce \
start
```
In AppJail, a template configuration file is similar to `jail=2Econf(5)`:
```
# cat template=2Econf
exec=2Estart: "/bin/sh /etc/rc"
exec=2Estop: "/bin/sh /etc/rc=2Eshutdown jail"
allow=2Emount
allow=2Emount=2Ezfs
enforce_statfs: 1
exec=2Epoststart: "zfs jail ${name} zroot/jailed"
exec=2Epoststart+: "appjail cmd jexec ${name} zfs mount zroot/jailed"
exec=2Eprestop: "appjail cmd jexec ${name} zfs umount zroot/jailed"
exec=2Eprestop+: "zfs unjail ${name} zroot/jailed"
```
As you can see, the dataset is mounted after running `zfs-jail(8)`=2E The =
steps are similar when the jail is stopped, but the dataset is unmounted an=
d `zfs-unjail(8)` is executed=2E
Inside the jail I can see the mounted datasets:
```
# appjail cmd jexec jtest zfs list -r
NAME USED AVAIL REFER MOUNTPOINT
zroot 34=2E1G 249G 96K /zroot
zroot/jailed 96K 249G 96K /jailed
# appjail cmd jexec jtest mount -t zfs
zroot/appjail/jails/jtest/jail on / (zfs, local, noatime, nfsv4acls)
zroot/jailed on /jailed (zfs, local, noatime, nfsv4acls)
# appjail cmd jexec jtest ls /jailed
index=2Etxt
# appjail cmd jexec jtest cat /jailed/index=2Etxt
Hi!
```
And I can use `zfs-rollback(8)` just fine:
```
# appjail cmd jexec jtest zfs snapshot zroot/jailed@guard
# appjail cmd jexec jtest zfs list -t snapshot zroot/jailed
NAME USED AVAIL REFER MOUNTPOINT
zroot/jailed@guard 0B - 96K -
# appjail cmd jexec jtest dd if=3D/dev/random of=3D/jailed/index=2Etxt bs=
=3D16 count=3D1
1+0 records in
1+0 records out
16 bytes transferred in 0=2E000102 secs (157318 bytes/sec)
# appjail cmd jexec jtest hd /jailed/index=2Etxt
00000000 a1 26 2a 9c f5 96 7b 81 90 8d ba 36 d6 f9 4d 93 |=2E&*=2E=2E=
=2E{=2E=2E=2E=2E6=2E=2EM=2E|
00000010
# appjail cmd jexec jtest zfs list -t snapshot zroot/jailed
NAME USED AVAIL REFER MOUNTPOINT
zroot/jailed@guard 56K - 96K -
# appjail cmd jexec jtest zfs rollback zroot/jailed@guard
# appjail cmd jexec jtest hd /jailed/index=2Etxt
00000000 48 69 21 0a |Hi!=2E|
00000004
```
I hope this can help you=2E
~ DtxdF
El 6 de noviembre de 2023 6:07:06 p=2E m=2E UTC, Chris Watson <bsdunix44@g=
mail=2Ecom> escribi=C3=B3:
>I've been trying to get a zfs dataset delegated into a jail (to run PG on=
),
>and allow snapshots and rollback to take place inside the jail=2E I can g=
et
>the dataset mounted into the jail, I can get zfs to take the snapshot, li=
st
>the snapshot, but when I rollback or try to ls -la the directory to see t=
he
>'=2Ezfs' dir it isn't there and the zfs rollback completes but it doesn't
>actually rollback=2E I'm so close to getting this to work! I'm just missi=
ng
>*something* in the sauce=2E When I do the zfs rollback zfs looks like it
>completes the rollback and goes back to a shell prompt but the files I
>remove before the rollback are not in the /var/db/postgres/data16 directo=
ry
>nor is "=2Ezfs" shown in ls -la=2E So something is wonky on my end=2E I'm=
so
>close, it's halfway there, it looks like it takes a snapshot, the snapsho=
t
>shows up in a zfs list -t snapshot, but it's also not really there=2E I'm
>doing something just slightly wrong here=2E I just cant figure out what I
>have wrong=2E
>
>Below are the configs:
># The jail's config
>https://bsd=2Eto/P176
># zfs list from inside the jail
>https://bsd=2Eto/mPde
># zfs list -t snapshot from inside the jail
>https://bsd=2Eto/R8dw
># ls -la /var/db/postgres/data16 output from inside the jail
>https://bsd=2Eto/1di2
># rc=2Econf of the jail
>https://bsd=2Eto/JcnH
>
>The jail is running 13=2E2-P4=2E
>Using bastillebsd 0=2E10=2E20231013 for creation/management=2E
>
>Thanks!
>Chris
------BSOZDAK4XRKBNCKWL0CKWPTSAUYJFS
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div dir=3D"auto">Hi Chris,<br><br>Maybe your data=
set is not mounted inside the jail=2E I thought that simply enabling `/etc/=
rc=2Ed/zfs` was fine, but no, it just doesn't work=2E I don't know if this =
behavior is a bug or something else, but at the moment I don't have time to=
investigate=2E<br><br>I have a similar setup for a jail with a delegated d=
ataset=2E I use AppJail, but the steps are similar to other tools:<br><br>`=
``<br># zfs create -o jailed=3Don -o mountpoint=3D/jailed zroot/jailed<br>#=
appjail quick jtest \<br> mount_devfs \<br> device=3D'include $devfsrules_=
hide_all' \<br> device=3D'include $devfsrules_unhide_basic' \<br> device=3D=
'include $devfsrules_unhide_login' \<br> device=3D'path zfs unhide' \<br> t=
emplate=3Dtemplate=2Econf \<br> overwrite=3Dforce \<br> start<br>```<br><br=
>In AppJail, a template configuration file is similar to `jail=2Econf(5)`:<=
br><br>```<br># cat template=2Econf<br>exec=2Estart: "/bin/sh /etc/rc"<br>e=
xec=2Estop: "/bin/sh /etc/rc=2Eshutdown jail"<br>allow=2Emount<br>allow=2Em=
ount=2Ezfs<br>enforce_statfs: 1<br>exec=2Epoststart: "zfs jail ${name} zroo=
t/jailed"<br>exec=2Epoststart+: "appjail cmd jexec ${name} zfs mount zroot/=
jailed"<br>exec=2Eprestop: "appjail cmd jexec ${name} zfs umount zroot/jail=
ed"<br>exec=2Eprestop+: "zfs unjail ${name} zroot/jailed"<br>```<br><br>As =
you can see, the dataset is mounted after running `zfs-jail(8)`=2E The step=
s are similar when the jail is stopped, but the dataset is unmounted and `z=
fs-unjail(8)` is executed=2E<br><br>Inside the jail I can see the mounted d=
atasets:<br><br>```<br># appjail cmd jexec jtest zfs list -r<br>NAME=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 USED=C2=A0 AVAIL=C2=
=A0=C2=A0=C2=A0=C2=A0 REFER=C2=A0 MOUNTPOINT<br>zroot=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 34=2E1G=C2=A0=C2=A0 249G=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 96K=C2=A0 /zroot<br>zroot/jailed=C2=A0=C2=A0=C2=A0 96K=C2=
=A0=C2=A0 249G=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 96K=C2=A0 /jailed<br># a=
ppjail cmd jexec jtest mount -t zfs<br>zroot/appjail/jails/jtest/jail on / =
(zfs, local, noatime, nfsv4acls)<br>zroot/jailed on /jailed (zfs, local, no=
atime, nfsv4acls)<br># appjail cmd jexec jtest ls /jailed<br>index=2Etxt<br=
># appjail cmd jexec jtest cat /jailed/index=2Etxt<br>Hi!<br>```<br><br>And=
I can use `zfs-rollback(8)` just fine:<br><br>```<br># appjail cmd jexec j=
test zfs snapshot zroot/jailed@guard<br># appjail cmd jexec jtest zfs list =
-t snapshot zroot/jailed<br>NAME=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 USED=C2=A0 AVAIL=C2=
=A0=C2=A0=C2=A0=C2=A0 REFER=C2=A0 MOUNTPOINT<br>zroot/jailed@guard=C2=A0=C2=
=A0=C2=A0=C2=A0 0B=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 96K=C2=A0 -<br># appjail cmd jexec jtest dd if=3D/dev/random o=
f=3D/jailed/index=2Etxt bs=3D16 count=3D1<br>1+0 records in<br>1+0 records =
out<br>16 bytes transferred in 0=2E000102 secs (157318 bytes/sec)<br># appj=
ail cmd jexec jtest hd /jailed/index=2Etxt<br>00000000=C2=A0 a1 26 2a 9c f5=
96 7b 81=C2=A0 90 8d ba 36 d6 f9 4d 93=C2=A0 |=2E&*=2E=2E=2E{=2E=2E=2E=
=2E6=2E=2EM=2E|<br>00000010<br># appjail cmd jexec jtest zfs list -t snapsh=
ot zroot/jailed<br>NAME=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 USED=C2=A0 AVAIL=C2=A0=C2=A0=
=C2=A0=C2=A0 REFER=C2=A0 MOUNTPOINT<br>zroot/jailed@guard=C2=A0=C2=A0=C2=A0=
56K=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 96=
K=C2=A0 -<br># appjail cmd jexec jtest zfs rollback zroot/jailed@guard<br>#=
appjail cmd jexec jtest hd /jailed/index=2Etxt<br>00000000=C2=A0 48 69 21 =
0a=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 |Hi!=2E|<br>00000004<br>```<br><br>I hope this can help you=2E<br><b=
r>~ DtxdF</div><br><br><div class=3D"gmail_quote"><div dir=3D"auto">El 6 de=
noviembre de 2023 6:07:06 p=2E m=2E UTC, Chris Watson <bsdunix44@gmail=
=2Ecom> escribi=C3=B3:</div><blockquote class=3D"gmail_quote" style=3D"m=
argin: 0pt 0pt 0pt 0=2E8ex; border-left: 1px solid rgb(204, 204, 204); padd=
ing-left: 1ex;">
<div dir=3D"ltr">I've been trying to get a zfs dataset delegated into a ja=
il (to run PG on), and allow snapshots and rollback to take place inside th=
e jail=2E I can get the dataset mounted into the jail, I can get zfs to tak=
e the snapshot, list the snapshot, but when I rollback or try to ls -la the=
directory to see the '=2Ezfs' dir it isn't there and the zfs rollback comp=
letes but it doesn't actually rollback=2E I'm so close to getting this to w=
ork! I'm just missing *something* in the sauce=2E When I do the zfs rollbac=
k zfs looks like it completes the rollback and goes back to a shell prompt =
but the files I remove before the rollback are not in the /var/db/postgres/=
data16 directory nor is "=2Ezfs" shown in ls -la=2E So something is wonky o=
n my end=2E I'm so close, it's halfway there, it looks like it takes a snap=
shot, the snapshot shows up in a zfs list -t snapshot, but it's also not re=
ally there=2E I'm doing something just slightly wrong here=2E I just cant f=
igure out what I have wrong=2E<div><br></div><div>Below are the configs:</d=
iv><div># The jail's config</div><div><a href=3D"https://bsd=2Eto/P176">htt=
ps://bsd=2Eto/P176</a></div><div># zfs list from inside the jail</div><div>=
<a href=3D"https://bsd=2Eto/mPde">https://bsd=2Eto/mPde</a><br></div><div>#=
zfs list -t snapshot from inside the jail</div><div><a href=3D"https://bsd=
=2Eto/R8dw">https://bsd=2Eto/R8dw</a><br></div><div># ls -la /var/db/postgr=
es/data16 output from inside the jail</div><div><a href=3D"https://bsd=2Eto=
/1di2">https://bsd=2Eto/1di2</a><br></div><div># rc=2Econf of the jail</div=
><div><a href=3D"https://bsd=2Eto/JcnH">https://bsd=2Eto/JcnH</a><br></div>=
<div><br></div><div>The jail is running 13=2E2-P4=2E</div><div>Using bastil=
lebsd 0=2E10=2E20231013 for creation/management=2E</div><div><br></div><div=
>Thanks!</div><div>Chris</div>
</div>
</blockquote></div></body></html>
------BSOZDAK4XRKBNCKWL0CKWPTSAUYJFS--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A696936B-D83F-4B7B-B037-F51CF47C44A9>