Date: Tue, 30 Jul 2013 23:38:04 +0200 From: Jeremie Le Hen <jlh@FreeBSD.org> To: Ollivier Robert <roberto@keltia.net> Cc: freebsd-jail@freebsd.org Subject: Re: jail design Message-ID: <20130730213804.GI13628@caravan.chchile.org> In-Reply-To: <20130729134335.GD13529@roberto02-aw.erc.corp.eurocontrol.int> References: <20130729134335.GD13529@roberto02-aw.erc.corp.eurocontrol.int>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Ollivier, On Mon, Jul 29, 2013 at 03:43:58PM +0200, Ollivier Robert wrote: > Hello, > > I have a new server I'm going to run all my services on (www, smtp/imap, and so on). Running 9.2-BETA1, full ZFS-on-root. > > What is the best practices about jails knowing that: > - I have only one IPv4 > - I have a full /48 IPv6 to play with > > I've looked at ezjail which is doing most of what I need but it does not support ip4/ip6=inherit parameters (and no jail.conf support either) so my networking setup is more complicated. All the other packages like qjail have only limited ZFS support. > > Do I need to setup pf to redirect all traffic in/out for specific ports to my jails? Or do I try to shoehorn "inherit" into ezjail? Is inherit easier to deal with? What are the security implications? > > I need something as easy as ezjail or a way to tweek it, with > - one jail for smtp/imap > - one for www stuff, ideally one jail per hosted domain (using nginx) > > I'm a jail newbie, in case you haven't found it already :) I've run services in jail for roughly a decade, always by hand. This is not a big deal if you do it correctly because, well, you don't create jail everyday :). Especially with recent tools. - First I started creating a jail for each service, sharing host's directories like /lib, /bin, /usr/lib, /usr/share, ... with nullfs in each jail in order to save space. - Then I found that it was painful because when I upgraded the host I had to upgrade all the jails at once. So I began to use the setup described in the handbook, section 16.6 [1]. Basically you create a base jail which is mounted r/o in each jail; all r/w directory are actually symlinks to a r/w space dedicated to the jail. This worked pretty well because when you want to upgrade your jail, you just create a new base jail and you progressively switch each jail to the new base, applying the upgrade procedure in each of them. At that time, every single service ran in its own jail. Jails were bound to 127.0.0.2, 127.0.0.3 and I used the right rdr rules on my public IP address. - Finally as time goes on and my free time decreases, I switched to the current model where I have very few jails, based on security domains (what you describe make sense: mail and web; you could also create one for shell access; honestly I don't advice to multiply jails, like one for each www domain, this is way to much adminitration overhead). These jails are full-fledged systems each on their own ZFS dataset. The very nice thing about that is that you can run a zfs send/receive every minute or so to backup your jails remotely, this is extremely efficient. Each jail is updated as if you had different servers, save the kernel of course. Besides, with pkgng and poudriere (kudos to the authors), this is extremely simple to upgrade. Regarding IP address, I bought one IP address for each jail (Kimsufi/OVH and Dedibox both provide this I think). The jail is simply brought up and down along with the jail. As you may have understood, I have a spare server which synchronizes the jails very often. If the main server fails, I can just start them on the backup one and perform an "IP failover" on my provider admin console. As for firewalling, this is not rocket science. The only difference is that you need to avoid using interfaces for filtering. You need to consider all traffic to both go out and come in on the loopback interface successively. Hope this helps. Cheers, [1] http://www.freebsd.org/doc/handbook/jails-application.html -- Jeremie Le Hen Scientists say the world is made up of Protons, Neutrons and Electrons. They forgot to mention Morons.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130730213804.GI13628>