Date: Sun, 17 Dec 2017 16:53:46 -0500 From: Dan Langille <dan@langille.org> To: Warner Losh <imp@bsdimp.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: cannot access pass device from within jail Message-ID: <E36E9900-FC65-4D97-9C9D-8D83FDF41FC0@langille.org> In-Reply-To: <CANCZdfr_VQE-WyZUfsYyWdyGRuPUh4qgn2Aqr7agGWQEH_Ypdg@mail.gmail.com> References: <E1314554-C8D0-4E8F-B8DB-E0B4D9DE325F@langille.org> <CANCZdfqWF1ckY58yp6sDGkJHxnwoyQB=nz1mnRKZe_mOM3H-gQ@mail.gmail.com> <C19AEFEA-1105-4891-ABDE-B7222147D396@langille.org> <CANCZdfr_VQE-WyZUfsYyWdyGRuPUh4qgn2Aqr7agGWQEH_Ypdg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Dec 17, 2017, at 4:48 PM, Warner Losh <imp@bsdimp.com> wrote:
>=20
> Sorry to top post. The problem did turn out to be securelevel. We =
found
> this by doing
>=20
> dtrace -n 'fbt::securelevel_gt:return {print(args[1];)}'
>=20
> Though you could also replace securelevel_gt with passopen to see it =
was in
> passopen too...
On the host we ran:
[root@r710-01:~] # sudo dtrace -n 'fbt::securelevel_gt:return =
{print(args[1]);}'
dtrace: description 'fbt::securelevel_gt:return ' matched 1 probe
CPU ID FUNCTION:NAME
21 50269 securelevel_gt:return int 0x1
In the jail: mtx -f /dev/pass7 status
Based on the dtrace output, I again checked securelevel in the jail:
[dan@bacula-sd-02] $ sysctl kern.securelevel
kern.securelevel: 2
WTF? I'd already checked that as seen at =
https://lists.freebsd.org/pipermail/freebsd-scsi/2017-December/007614.html=
Oh wait. The above URL refers to bacula-sd-01, not bacula-sd-02. Ouch. =
User error... me.
After going back into the host, I set:
$ sudo iocage set securelevel=3D1 bacula-sd-02
Property: securelevel has been updated to 1
After restarting the jail and getting back into it:
[root@bacula-sd-02 ~]# sysctl kern.securelevel
kern.securelevel: 1
[root@bacula-sd-02 ~]# mtx -f /dev/pass7 status=20
Storage Changer /dev/pass7:2 Drives, 47 Slots ( 0 Import/Export )
Data Transfer Element 0:Full (Storage Element 1 Loaded):VolumeTag =3D =
000001L4 =20
....
Success.
Sorry for the noise which would have been reduced if I'd gotten my =
sysctl on the right host.
Thank you all=10.
>=20
> Warner
>=20
> On Sun, Dec 17, 2017 at 2:08 PM, Dan Langille <dan@langille.org> =
wrote:
>=20
>>> On Dec 17, 2017, at 4:04 PM, Warner Losh <imp@bsdimp.com> wrote:
>>>=20
>>> What's the permissions of /dev/xpt0 in the jail? If it's not there I =
know
>>> at least camcontrol won't work. I've not used mtx, so I can't say if =
it's
>>> affected too or not.
>>=20
>> I have tried both with and without xpt0. When I tried, it was:
>>=20
>> # ls -l /dev/xpt0
>> crw------- 1 root operator 0x4c Dec 16 21:52 /dev/xpt0
>>=20
>>>=20
>>> However, looking at the truss output:
>>>=20
>>> openat(AT_FDCWD,"/dev/pass7",O_RDWR|O_EXCL,00) ERR#1 'Operation not
>>> permitted'
>>> suggests something other than the canonical xpt0 issue else is going =
on.
>> If
>>> we look at passopen in cam, I can see two exit paths:
>>>=20
>>> error =3D securelevel_gt(td->td_ucred, 1); if (error !=3D 0) =
{...
>>> return error; }
>>> securelevel_gt is just "return (cr->cr_prison->pr_securelevel > =
level ?
>>> EPERM : 0);" which might be possible. What's the securelevel of the =
jail?
>>> Maybe this is going on somehow?
>>=20
>>=20
>> On the host:
>>=20
>> $ sysctl kern.securelevel
>> kern.securelevel: -1
>>=20
>>=20
>> On the jail:
>>=20
>> $ sysctl kern.securelevel
>> kern.securelevel: -1
>>=20
>>>=20
>>> The second is basically
>>> if (((flags & FWRITE) =3D=3D 0) || ((flags & FREAD) =3D=3D 0)) =
{... return
>>> EPERM; }
>>> which isn't happening because of the O_RDWR in the truss output.
>>>=20
>>> The other possibility is that something above the pass driver is =
doing
>> the
>>> check. I've not looked at that code path yet, buy you can see if =
it's
>>> making it to passopen() with dtrace and checking its return value. I
>> don't
>>> see anything in how we register the device, though, that would =
suggest
>>> filtering it in jails.
>>>=20
>>> Warner
>>>=20
>>> On Sun, Dec 17, 2017 at 12:52 PM, Dan Langille <dan@langille.org> =
wrote:
>>>=20
>>>> Hello,
>>>>=20
>>>> What suggestions do you have for where I should look next? I'm =
happy to
>>>> start installing various builds of FreeBSD in order to track down =
which
>>>> commit caused this.
>>>>=20
>>>> I'm trying to access a tape library from within a jail running on a
>>>> FreeBSD 11.1 host. sa(4) devices are working (e.g. I can rewind =
nsa0).
>>>>=20
>>>> pass(4) devices (i.e. the tape changer ch0) are not working. This
>> morning
>>>> I posted to -scsi@: https://lists.freebsd.org/
>> pipermail/freebsd-scsi/2017-
>>>> December/007608.html
>>>>=20
>>>> The device appears in the jail and has appropriate permissions. =
This
>>>> access was granted
>>>> via /etc/devfs.rules using the same approach I used for FreeBSD =
10.3
>>>>=20
>>>> The permissions in the jail:
>>>>=20
>>>> [root@bacula-sd-02 ~]# ls -l /dev/pass7
>>>> crw------- 1 root operator 0x74 Dec 16 21:52 /dev/pass7
>>>>=20
>>>> The command in the jail:
>>>>=20
>>>> [root@bacula-sd-02 ~]# mtx -f /dev/pass7 status
>>>> cannot open SCSI device '/dev/pass7' - Operation not permitted
>>>>=20
>>>> Here is the truss output of the command in question:
>>>> https://gist.github.com/dlangille/b80ee804b8080e1cbf5b5ab67f0bdabe
>>>>=20
>>>> Thank you.
>>>>=20
>>>> --
>>>> Dan Langille - BSDCan / PGCon
>>>> dan@langille.org
>>>>=20
>>>>=20
>>>> _______________________________________________
>>>> freebsd-current@freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@
>> freebsd.org"
>>>>=20
>>> _______________________________________________
>>> freebsd-current@freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@
>> freebsd.org"
>>=20
>>=20
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to =
"freebsd-current-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E36E9900-FC65-4D97-9C9D-8D83FDF41FC0>