Date: Tue, 13 Jun 2017 01:04:46 +0200 From: Willem Jan Withagen <wjw@digiware.nl> To: Allan Jude <allanjude@freebsd.org>, freebsd-jail@freebsd.org Subject: Re: Passing a limited amount of disk devices to jails Message-ID: <95f91eaf-3883-60a1-dea6-efaf4d49ed9b@digiware.nl> In-Reply-To: <bd164b17-efed-cf2c-5472-76a0099f3efe@digiware.nl> References: <edc6787a-256f-1027-0008-6d5dfa10d651@digiware.nl> <109608ec-2fc2-1f04-3dcc-0056f243639c@multiplay.co.uk> <77f2e146-b0c8-00ba-117b-b4385f355be6@digiware.nl> <593AAEBC.7050701@quip.cz> <46f93966-655a-84ef-181d-6586c18d77c7@digiware.nl> <61b6dc73-5b73-7c5f-59d1-831d6336b4a2@freebsd.org> <bd164b17-efed-cf2c-5472-76a0099f3efe@digiware.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12-6-2017 11:48, Willem Jan Withagen wrote:
> On 11-6-2017 02:41, Allan Jude wrote:
>> On 06/10/2017 20:13, Willem Jan Withagen wrote:
>>> On 9-6-2017 16:20, Miroslav Lachman wrote:
>>>> Willem Jan Withagen wrote on 2017/06/09 15:48:
>>>>> On 9-6-2017 11:23, Steven Hartland wrote:
>>>>>> You could do effectively this by using dedicated zfs filesystems per
>>>>>> jail
>>>>>
>>>>> Hi Steven,
>>>>>
>>>>> That is how I'm going to do it, when nothing else works.
>>>>> But then I don't get to test the part of building the ceph-cluster from
>>>>> raw disk...
>>>>>
>>>>> I was more thinking along the lines of tinkering with the devd.conf or
>>>>> something. And would appreciate opinions on how to (not) do it.
>>>>
>>>> I totally skipped devd.conf in my mind in previous reply. So maybe you
>>>> can really use devd.conf to allow access to /dev/adaX devices or you can
>>>> use ZFS zvol if you have big pool and need some smaller devices to test
>>>> with.
>>>
>>> I want the jail to look as much as a normal system would, and then run
>>> ceph-tools on them. And they would like to see /dev/{disk}....
>>>
>>> Now I have found /sbin/devfs which allows to add/remove devices to an
>>> already existing devfs-mount.
>>>
>>> So I can 'rule add type disk unhide' and see the disks.
>>> Gpart can then list partitions.
>>> But any of the other commands is met with an unwilling system:
>>>
>>> root@ceph-1:/ # gpart delete -i 1 ada0
>>> gpart: No such file or directory
>>>
>>> So there is still some protection in place in the jail....
>>>
>>> However dd-ing to the device does overwrite some stuff.
>>> Since after the 'dd if=/dev/zero of=/dev/ada0' gpart reports a corrupt
>>> gpartition.
>>>
>>> But I don't see any sysctl options to toggle that on or off
>
>> To use GEOM tools like gpart, I think you'll need to unhide
>> /dev/geom.ctl in the jail
>>
>>
>
> Right, thanx, could very well be the case.
> I'll try and post back here.
>
> But I'll take a different approach and just enable all devices in /dev
> Since I'm not really needing security, but only need separate compute
> spaces. And jails have the advantage over bhyve that it is easy to
> modify files in the subdomains.
> Restricting afterwards might be an easier job.
>
> I'm also having trouble expanding /etc/{,defaults/}devfs.rules and have
> 'mount -t devfs -oruleset'
> pick up the changes.
> Even adding any extra ruleset to the /etc/defaults/devfs.rules does not
> get picked up, hence my toying with /sbin/devfs.
Right,
That will help.
Next challenge is to allow zfs to create a filesystem on a partition.
root@ceph-1:/ # gpart destroy -F ada8
ada8 destroyed
root@ceph-1:/ # gpart create -s GPT ada8
ada8 created
root@ceph-1:/ # gpart add -t freebsd-zfs -a 1M -l osd-disk-1 /dev/ada8
ada8p1 added
root@ceph-1:/ # zpool create -f osd.1 /dev/ada8p1
cannot create 'osd.1': permission denied
root@ceph-1:/ #
--WjW
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?95f91eaf-3883-60a1-dea6-efaf4d49ed9b>