From owner-freebsd-jail@freebsd.org Mon Feb 22 11:57:14 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E701AAF526 for ; Mon, 22 Feb 2016 11:57:14 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B69E11C9F for ; Mon, 22 Feb 2016 11:57:13 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 5FC9C28412; Mon, 22 Feb 2016 12:57:09 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 20B672840C; Mon, 22 Feb 2016 12:57:08 +0100 (CET) Message-ID: <56CAF793.2030104@quip.cz> Date: Mon, 22 Feb 2016 12:57:07 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Aristedes Maniatis CC: freebsd-jail Subject: Re: Jail management References: <0f5cae7e-7de3-2617-fcf6-3423d4caf13a@ish.com.au> <56CAE974.4050508@quip.cz> <0eaf61d4-43e6-265a-f773-820244fc8931@ish.com.au> In-Reply-To: <0eaf61d4-43e6-265a-f773-820244fc8931@ish.com.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2016 11:57:14 -0000 Aristedes Maniatis wrote on 02/22/2016 12:26: > On 22/02/2016 9:56pm, Miroslav Lachman wrote: >> I don't know your environment and your FreeBSD jails skills but it seems you think jails are something complex and "magic". It is not. > ... >> Just don't be afraid of writing simple shell scripts :) > > > You are right, and perhaps I should just bite the bullet. I am afraid of only two things. > > * upgrade the basejail with FreeBSD upgrades. I am sure this is a simple bit of chroot magic, but freebsd-update is a bit of a black box to me. I tried it few years ago and it had some problems that doesn't fit well in to my environment, then I moved all our servers to own buildserver with make buildkernel + buildworld and then installworld through NFS in to destionation. Faster, safer and predictable solution. (I had problems with freebsd-update even on bare metal systems, not in jails) > * nullfs. I've never used it before and need to play with it more Nullfs is easy. You can "mount" one directory to another. If you have /vol0/jail/_basejail and jails in /vol0/jail/alpha, /vol0/jail/beta Then you can do mkdir /vol0/jail/alpha/basejail mkdir /vol0/jail/beta/basejail mount -t nullfs /vol0/jail/_basejail /vol0/jail/alpha/basejail mount -t nullfs /vol0/jail/_basejail /vol0/jail/beta/basejail Your basejail contains # ls -1 /vol0/jail/_basejail UPDATED bin boot lib libexec rescue sbin usr an jails (alpha, beta and you new jail template) contains symlinks to these directories # ls -lg /vol0/jail/alpha/ -rw-r--r-- 1 root wheel 798 Jan 13 2015 .cshrc -rw-r--r-- 2 root wheel 265 Jan 13 2015 .profile -r--r--r-- 1 root wheel 6197 May 12 2015 COPYRIGHT drwxr-xr-x 9 root wheel 10 May 12 2015 basejail lrwxr-xr-x 1 root wheel 13 Jan 13 2015 bin -> /basejail/bin lrwxr-xr-x 1 root wheel 14 Jan 13 2015 boot -> /basejail/boot dr-xr-xr-x 7 root wheel 512 Oct 18 17:52 dev lrwxr-xr-x 1 root wheel 12 Jan 20 2015 develop -> /usr/develop drwxr-xr-x 20 root wheel 105 Nov 12 19:37 etc lrwxr-xr-x 1 root wheel 8 Jan 13 2015 home -> usr/home lrwxr-xr-x 1 root wheel 13 Jan 13 2015 lib -> /basejail/lib lrwxr-xr-x 1 root wheel 17 Jan 13 2015 libexec -> /basejail/libexec dr-xr-xr-x 2 root wheel 2 Jan 13 2015 proc lrwxr-xr-x 1 root wheel 16 Jan 13 2015 rescue -> /basejail/rescue drwxr-xr-x 10 root wheel 29 May 12 2015 root lrwxr-xr-x 1 root wheel 14 Jan 13 2015 sbin -> /basejail/sbin lrwxr-xr-x 1 root wheel 11 Jan 13 2015 sys -> usr/src/sys drwxrwxrwt 9 root wheel 10 Feb 22 03:43 tmp drwxr-xr-x 7 root wheel 17 Jan 20 2015 usr drwxr-xr-x 22 root wheel 22 Oct 18 17:52 var Nullfs mounts can be specified in fstab files # cat /etc/fstab.alpha /vol0/jail/_basejail /vol0/jail/alpha/basejail nullfs ro 0 0 # cat /etc/fstab.beta /vol0/jail/_basejail /vol0/jail/beta/basejail nullfs ro 0 0 So if jails are running, you wil see this tank/vol0/jail/alpha on /vol0/jail/alpha (zfs, local, noatime, nfsv4acls) tank/vol0/jail/beta on /vol0/jail/beta (zfs, local, noatime, nfsv4acls) /vol0/jail/_basejail on /vol0/jail/alpha/basejail (nullfs, local, read-only) /vol0/jail/_basejail on /vol0/jail/beta/basejail (nullfs, local, read-only) And you can have gamma with another basejail called _basejail93 mounted as tank/vol0/jail/gamma on /vol0/jail/gamma (zfs, local, noatime, nfsv4acls) /vol0/jail/_basejail93 on /vol0/jail/gamma/basejail (nullfs, local, read-only) Migrate this jail to _basejail is just a matter of change one line if fstab.gamma All commong settings are in /etc/jail.conf It can be something like this ## Typical static defaults: ## Use the rc scripts to start and stop jails. Mount jail's /dev. exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.clean; exec.system_user = "root"; exec.jail_user = "root"; mount.devfs; devfs_ruleset = 4; enforce_statfs = 1; #allow.set_hostname = false; #allow.mount; allow.set_hostname = 0; allow.sysvipc = 0; allow.raw_sockets = 0; ## Dynamic wildcard parameter: ## Base the path off the jail name. path = "/vol0/jail/$name"; exec.consolelog = "/var/log/jail/$name.console"; mount.fstab = "/etc/fstab.$name"; ## Alpha alpha { host.hostname = "alpha.example.com"; ip4.addr = 10.10.10.20; allow.sysvipc = 1; } ## Beta beta { host.hostname = "beta.example.com"; ip4.addr = 10.10.10.30; } ## Gamma gamma { host.hostname = "gamma.example.com"; ip4.addr = 10.10.10.40; } > As for shell scripts: my only goal in life is to write *fewer* shell scripts. My adoption of saltstack was spurred by shell everywhere, mostly not under version control. So less shell and more python centrally managed and versioned is my dream. I understand this approach. You can look at it as your own port (package) and not as unversioned shell script. :) Miroslav Lachman