From owner-freebsd-jail@freebsd.org Mon Jun 12 09:49:03 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9ABFBF832B for ; Mon, 12 Jun 2017 09:49:03 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9187C64AFE; Mon, 12 Jun 2017 09:49:03 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 3CDE343480; Mon, 12 Jun 2017 11:48:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p5GDiwEhvr0j; Mon, 12 Jun 2017 11:48:58 +0200 (CEST) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id AB1DD4347F; Mon, 12 Jun 2017 11:48:58 +0200 (CEST) Subject: Re: Passing a limited amount of disk devices to jails To: Allan Jude , freebsd-jail@freebsd.org References: <109608ec-2fc2-1f04-3dcc-0056f243639c@multiplay.co.uk> <77f2e146-b0c8-00ba-117b-b4385f355be6@digiware.nl> <593AAEBC.7050701@quip.cz> <46f93966-655a-84ef-181d-6586c18d77c7@digiware.nl> <61b6dc73-5b73-7c5f-59d1-831d6336b4a2@freebsd.org> From: Willem Jan Withagen Message-ID: Date: Mon, 12 Jun 2017 11:48:46 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <61b6dc73-5b73-7c5f-59d1-831d6336b4a2@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2017 09:49:03 -0000 On 11-6-2017 02:41, Allan Jude wrote: > On 06/10/2017 20:13, Willem Jan Withagen wrote: >> On 9-6-2017 16:20, Miroslav Lachman wrote: >>> Willem Jan Withagen wrote on 2017/06/09 15:48: >>>> On 9-6-2017 11:23, Steven Hartland wrote: >>>>> You could do effectively this by using dedicated zfs filesystems per >>>>> jail >>>> >>>> Hi Steven, >>>> >>>> That is how I'm going to do it, when nothing else works. >>>> But then I don't get to test the part of building the ceph-cluster from >>>> raw disk... >>>> >>>> I was more thinking along the lines of tinkering with the devd.conf or >>>> something. And would appreciate opinions on how to (not) do it. >>> >>> I totally skipped devd.conf in my mind in previous reply. So maybe you >>> can really use devd.conf to allow access to /dev/adaX devices or you can >>> use ZFS zvol if you have big pool and need some smaller devices to test >>> with. >> >> I want the jail to look as much as a normal system would, and then run >> ceph-tools on them. And they would like to see /dev/{disk}.... >> >> Now I have found /sbin/devfs which allows to add/remove devices to an >> already existing devfs-mount. >> >> So I can 'rule add type disk unhide' and see the disks. >> Gpart can then list partitions. >> But any of the other commands is met with an unwilling system: >> >> root@ceph-1:/ # gpart delete -i 1 ada0 >> gpart: No such file or directory >> >> So there is still some protection in place in the jail.... >> >> However dd-ing to the device does overwrite some stuff. >> Since after the 'dd if=/dev/zero of=/dev/ada0' gpart reports a corrupt >> gpartition. >> >> But I don't see any sysctl options to toggle that on or off > To use GEOM tools like gpart, I think you'll need to unhide > /dev/geom.ctl in the jail > > Right, thanx, could very well be the case. I'll try and post back here. But I'll take a different approach and just enable all devices in /dev Since I'm not really needing security, but only need separate compute spaces. And jails have the advantage over bhyve that it is easy to modify files in the subdomains. Restricting afterwards might be an easier job. I'm also having trouble expanding /etc/{,defaults/}devfs.rules and have 'mount -t devfs -oruleset' pick up the changes. Even adding any extra ruleset to the /etc/defaults/devfs.rules does not get picked up, hence my toying with /sbin/devfs. --WjW