From owner-freebsd-stable@FreeBSD.ORG Wed Feb 10 12:04:50 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAAB7106566C for ; Wed, 10 Feb 2010 12:04:50 +0000 (UTC) (envelope-from mamalos@eng.auth.gr) Received: from vergina.eng.auth.gr (vergina.eng.auth.gr [155.207.18.1]) by mx1.freebsd.org (Postfix) with ESMTP id 0F4148FC0C for ; Wed, 10 Feb 2010 12:04:49 +0000 (UTC) Received: from mamalacation.ee.auth.gr (mamalacation.ee.auth.gr [155.207.33.29]) by vergina.eng.auth.gr (8.14.3/8.14.1) with ESMTP id o1AC4mZd049479; Wed, 10 Feb 2010 14:04:48 +0200 (EET) (envelope-from mamalos@eng.auth.gr) Message-ID: <4B72A0DB.5010806@eng.auth.gr> Date: Wed, 10 Feb 2010 14:04:43 +0200 From: George Mamalakis User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.5) Gecko/20100115 Thunderbird/3.0 MIME-Version: 1.0 To: freebsd-doc@freebsd.org, freebsd-stable Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: A more secure approach of jail establishment. It could be included in jail chapter of fbsd handbook X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2010 12:04:50 -0000 Dear all, After working with jails for a long time, I ended up with a quite different approach (architecture) of setting up jails, that seems more secure than the one proposed in the freebsd handbook. The concept is very similar to that of handbook's chapter 15.6.1, only it involves a few more concepts. I'll try to be descriptive and understandable. Instead of having the mroot directory (mounted ro) containing the base system, and the s directory containing the different configs of each different jail (mounted rw), I introduced one more directory, let's call it t. Now ls -la /jails/j/mroot gives: total 32 drwxr-xr-x 14 root wheel - 512 Sep 24 18:16 ./ drwxr-xr-x 10 root wheel - 512 Feb 10 13:17 ../ drwxr-xr-x 2 root wheel - 1024 Sep 24 17:56 bin/ drwxr-xr-x 7 root wheel - 512 Sep 24 17:56 boot/ dr-xr-xr-x 2 root wheel - 512 Sep 24 17:56 dev/ lrwxr-xr-x 1 root wheel - 5 Sep 24 18:15 etc@ -> s/etc drwxr-xr-x 3 root wheel - 1536 Sep 24 17:56 lib/ drwxr-xr-x 2 root wheel - 512 Sep 24 17:57 libexec/ drwxr-xr-x 2 root wheel - 512 Sep 24 17:56 media/ dr-xr-xr-x 2 root wheel - 512 Sep 24 17:56 proc/ drwxr-xr-x 2 root wheel - 2560 Sep 24 17:56 rescue/ lrwxr-xr-x 1 root wheel - 6 Sep 24 18:15 root@ -> s/root drwxr-xr-x 2 root wheel - 512 Sep 24 18:14 s/ drwxr-xr-x 2 root wheel - 2560 Sep 24 17:57 sbin/ lrwxr-xr-x 1 root wheel - 11 Sep 24 17:56 sys@ -> usr/src/sys drwxr-xr-x 2 root wheel - 512 Sep 24 18:14 t/ lrwxr-xr-x 1 root wheel - 5 Sep 24 18:15 tmp@ -> t/tmp drwxr-xr-x 14 root wheel - 512 Sep 24 18:50 usr/ lrwxr-xr-x 1 root wheel - 5 Sep 24 18:16 var@ -> t/var And ls -la /jails/j/mroot/usr gives: drwxr-xr-x 14 root wheel - 512 Sep 24 18:50 ./ drwxr-xr-x 14 root wheel - 512 Sep 24 18:16 ../ lrwxr-xr-x 1 root wheel - 15 Sep 24 18:15 X11R6@ -> ../s/usr/-X11R6 drwxr-xr-x 2 root wheel - 7168 Sep 24 17:57 bin/ drwxr-xr-x 2 root wheel - 512 Sep 24 17:56 games/ drwxr-xr-x 46 root wheel - 5120 Sep 24 17:56 include/ drwxr-xr-x 6 root wheel - 11776 Sep 24 17:57 lib/ drwxr-xr-x 3 root wheel - 12288 Sep 24 17:57 lib32/ drwxr-xr-x 5 root wheel - 512 Sep 24 17:56 libdata/ drwxr-xr-x 5 root wheel - 1536 Sep 24 17:57 libexec/ lrwxr-xr-x 1 root wheel - 14 Sep 24 18:15 local@ -> ../s/usr-local drwxr-xr-x 2 root wheel - 512 Sep 24 17:56 obj/ drwxr-xr-x 68 root wheel - 1536 Sep 24 18:50 ports/ drwxr-xr-x 2 root wheel - 5120 Sep 24 17:57 sbin/ drwxr-xr-x 25 root wheel - 512 Sep 24 17:56 share/ drwxr-xr-x 22 root wheel - 1024 Sep 24 18:14 src/ As we can see, only tmp and var is linked to t, while all other symlinks are the same with the configuration proposed in the handbook. The difference is the following: In my steup, s is mounted ro and t is mounted rw. In this way, jail's /etc is mounted ro, /usr/ is mounted ro, which means that /usr/local is mounted ro, which means that /s/portbuild is mounted read onlty, which means that /root is mounted read only, and so on... So, all ports installed on the system, and deamons running through /usr/local are now read only and trojaned versions of binaries are much more difficult to be installed if the jail is compromised. In order to be able to manage those jails, I did the following trick: # cat /etc/fstab /dev/da0s1a / ufs rw 1 1 /dev/da0s1b none swap sw 0 0 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 # Jails /jails/j/mroot /jails/j/bind nullfs ro 0 0 /jails/j/mroot /jails/j/heimdal nullfs ro 0 0 /jails/j/mroot /jails/j/ldap nullfs ro 0 0 /jails/s/ldap /jails/j/ldap/s nullfs ro 0 0 /jails/s/bind /jails/j/bind/s nullfs ro 0 0 /jails/s/heimdal /jails/j/heimdal/s nullfs ro 0 0 /jails/t/heimdal /jails/j/heimdal/t nullfs rw 0 0 /jails/t/bind /jails/j/bind/t nullfs rw 0 0 /jails/t/ldap /jails/j/ldap/t nullfs rw 0 0 # Manage jails /jails/j/mroot /jails/manage/bind nullfs rw 0 0 /jails/j/mroot /jails/manage/heimdal nullfs rw 0 0 /jails/j/mroot /jails/manage/ldap nullfs rw 0 0 /jails/s/ldap /jails/manage/ldap/s nullfs rw 0 0 /jails/s/bind /jails/manage/bind/s nullfs rw 0 0 /jails/s/heimdal /jails/manage/heimdal/s nullfs rw 0 0 /jails/t/heimdal /jails/manage/heimdal/t nullfs rw 0 0 /jails/t/bind /jails/manage/bind/t nullfs rw 0 0 /jails/t/ldap /jails/manage/ldap/t nullfs rw 0 0 devfs /jails/manage/heimdal/dev devfs rw 0 0 devfs /jails/manage/bind/dev devfs rw 0 0 devfs /jails/manage/ldap/dev devfs rw 0 0 In my setup I have three jails: ldap, heimdal and bind. mroot is monted in each jail's root folder, and the s and t directories of each jail are null_mounted on each jail's s and t folder respectively. I introduced another folder, /jails/manage which has the analogous usage to /jails/j folder, only this folder's subfolders are not used as jails. I chroot to each folder when I want to have write access on it's corresponding jail, and I can perform all my administrative operations (if you notice, even mroot is mounted rw in this chroot, so changes in base disto can be performed - eg when installing perl where you are asked to change /usr/bin/perl). This is the reason why I mount devfs on these chroots. As a result, with this setup, only /var and /tmp are mounted rw in each jail, and all other filesystems are mounted ro. Management (meaning jobs that require write access, not starting and/or stopping services) is achieved by chrooting to each jail's corresponding chroot. I am working with this setup for more than 4 months without having any problems; I thought that I could propose it as another paragraph in chapter 15.6 of freebsd handbook, as an even more secure jail setup, instead of just publishing it to some blog,forum,etc. Thank you all for your time. -- George Mamalakis IT Officer Electrical and Computer Engineer (Aristotle Un. of Thessaloniki), MSc (Imperial College of London) Department of Electrical and Computer Engineering Faculty of Engineering Aristotle University of Thessaloniki phone number : +30 (2310) 994379