From owner-freebsd-jail@FreeBSD.ORG Sun Sep 1 04:32:57 2013 Return-Path: Delivered-To: freebsd-jail@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4A96814C; Sun, 1 Sep 2013 04:32:57 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 16E9C2524; Sun, 1 Sep 2013 04:32:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r814WujH021031; Sun, 1 Sep 2013 04:32:56 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r814Wu2B021030; Sun, 1 Sep 2013 04:32:56 GMT (envelope-from linimon) Date: Sun, 1 Sep 2013 04:32:56 GMT Message-Id: <201309010432.r814Wu2B021030@freefall.freebsd.org> To: mw@wzff.de, linimon@FreeBSD.org, freebsd-jail@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/176092: [jail] [panic] Starting a jail on my releng/9.1 kernel with pf and VIMAGE enabled crashes the kernel X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Sep 2013 04:32:57 -0000 Synopsis: [jail] [panic] Starting a jail on my releng/9.1 kernel with pf and VIMAGE enabled crashes the kernel State-Changed-From-To: open->closed State-Changed-By: linimon State-Changed-When: Sun Sep 1 04:32:37 UTC 2013 State-Changed-Why: See kern/176112. http://www.freebsd.org/cgi/query-pr.cgi?pr=176092 From owner-freebsd-jail@FreeBSD.ORG Mon Sep 2 11:06:47 2013 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2A2BA100 for ; Mon, 2 Sep 2013 11:06:47 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1818D23A8 for ; Mon, 2 Sep 2013 11:06:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r82B6klA016066 for ; Mon, 2 Sep 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r82B6krQ016064 for freebsd-jail@FreeBSD.org; Mon, 2 Sep 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 2 Sep 2013 11:06:46 GMT Message-Id: <201309021106.r82B6krQ016064@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-jail@FreeBSD.org Subject: Current problem reports assigned to freebsd-jail@FreeBSD.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 11:06:47 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o conf/181650 jail [jail] [patch] /etc/rc.d/jail fails if a kernel built o kern/180916 jail [jail] [regression] jail startup is broken for 8.4 wit o kern/180067 jail [jail] [patch] fix multicast support within jails o bin/178302 jail jail(8): unknown parameter: ip6.addr when kernel compi o kern/176112 jail [jail] [panic] kernel panic when starting jails o kern/174902 jail [jail] jail should provide validator for jail names o bin/173469 jail [jail] regression: security.jail.sysvipc_allowed=1 no o kern/169751 jail [jail] reading routing information does not work in ja o bin/167911 jail new jail(8) problem with removal, ifconfg -alias and k o kern/159918 jail [jail] inter-jail communication failure o kern/156111 jail [jail] procstat -b not supported in jail o misc/155765 jail [patch] `buildworld' does not honors WITHOUT_JAIL o conf/154246 jail [jail] [patch] Bad symlink created if devfs mount poin s conf/142972 jail [jail] [patch] Support JAILv2 and vnet in rc.d/jail o conf/141317 jail [patch] uncorrect jail stop in /etc/rc.d/jail o kern/133265 jail [jail] is there a solution how to run nfs client in ja o kern/119842 jail [smbfs] [jail] "Bad address" with smbfs inside a jail o bin/99566 jail [jail] [patch] fstat(1) according to specified jid 18 problems total. From owner-freebsd-jail@FreeBSD.ORG Mon Sep 2 23:25:37 2013 Return-Path: Delivered-To: freebsd-jail@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8A16C281; Mon, 2 Sep 2013 23:25:37 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 131DB20AB; Mon, 2 Sep 2013 23:25:36 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 84D8B2842B; Tue, 3 Sep 2013 01:25:29 +0200 (CEST) Received: from [192.168.1.2] (ip-89-177-49-222.net.upcbroadband.cz [89.177.49.222]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 56FAB28426; Tue, 3 Sep 2013 01:25:28 +0200 (CEST) Message-ID: <52251E67.1060002@quip.cz> Date: Tue, 03 Sep 2013 01:25:27 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: Jamie Gritton Subject: Re: jail.conf & cpuset.id References: <076B486D-A526-4945-BA38-DD7167365749@inbox.im> <514B09B2.70607@FreeBSD.org> <52224088.6040508@quip.cz> <5224F6E8.802@FreeBSD.org> In-Reply-To: <5224F6E8.802@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD-Jail X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 23:25:37 -0000 Jamie Gritton wrote: [...] >> >> Hi Jamie, >> I tried your suggestion with exec_poststart for setting the cpuset. >> It doesn't work. I don't know if it worked for you with any older >> version of FreeBSD. I tried it on FreeBSD 9.1-RELESE. >> >> I have this in rc.conf >> >> jail_fox_exec_poststart0="cpuset -c -l 5-6 -j `cat /var/run/jail_fox.id`" >> >> With rc_debug="YES", I get this error >> >> # service jail start fox >> cat: /var/run/jail_fox.id: No such file or directory >> cat: /var/run/jail_fox.id: No such file or directory >> >> [snip] >> >> /etc/rc.d/jail: DEBUG: fox exec post-start #1: cpuset -c -l 5-6 -j >> >> [snip] >> >> fox.example.comcpuset: option requires an argument -- j >> usage: cpuset [-l cpu-list] [-s setid] cmd ... >> cpuset [-l cpu-list] [-s setid] -p pid >> cpuset [-c] [-l cpu-list] -C -p pid >> cpuset [-cr] [-l cpu-list] [-j jailid | -p pid | -t tid | -s >> setid | -x irq] >> cpuset [-cgir] [-j jailid | -p pid | -t tid | -s setid | -x irq] >> >> >> I think the problem is, that the command is evaluated befor the jail is >> started. >> >> Or am I doing something wrong? >> >> I also tried following with no luck: >> >> jail_fox_exec_poststart0="cpuset -c -l 5-6 -j `jls -j fox jid`" > > I'm not seeing /var/run/jail_*.id either, despite it being mentioned in > /etc/rc.d/jail. The jls one works for me, but then I'm running current > so maybe that's the difference. I made a diff between rc.d/jail from HEAD, 8.4-RELEASE and 9.1-RELEASE and there is no changes in code for _exec_poststart. I don't know how it is possible, that it works for you and doesn't for for me. If I interpret it correctly, the following code in the begining of init_variables() does the eval on the command and backticks are executed at this time - before the jail is started. Am I wrong? i=0 while : ; do eval _exec_poststart${i}=\"\${jail_${_j}_exec_poststart${i}:-\${jail_exec_poststart${i}}}\" [ -z "$(eval echo \"\$_exec_poststart${i}\")" ] && break i=$((i + 1)) done Then there is code for debug printing only i=0 while : ; do eval out=\"\${_exec_poststart${i}:-''}\" if [ -z "$out" ]; then break fi debug "$_j exec post-start #${i}: ${out}" i=$((i + 1)) done And in jail_start() there is jail command executions followed by execution of poststart commands, but at this time, the command is "cpuset -c -l 5-6 -j" and not "cpuset -c -l 5-6 -j `jls -j fox jid`" eval ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \ \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1 \ If all else fails, you can hardcode the > jail number - that'll work as long as you only ever let the jails load > on startup. Unluckily I need to restart some jails manually, so I think I cannot use hardcoded JID numbers :( Thank you for your reply. Miroslav Lachman From owner-freebsd-jail@FreeBSD.ORG Wed Sep 4 09:40:42 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 70D8C627 for ; Wed, 4 Sep 2013 09:40:42 +0000 (UTC) (envelope-from olevole@olevole.ru) Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com [209.85.217.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EB90B235E for ; Wed, 4 Sep 2013 09:40:41 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id u14so160262lbd.2 for ; Wed, 04 Sep 2013 02:40:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version:content-transfer-encoding:content-type; bh=pMbtpV1H7bdh8dJaWA/TKFz9S4DCWFcMe2T2vYcBQ6Y=; b=kW5aHfw7weofz64hDVQolsh47Jg77rU/h5A5AZJfPL+dk+0uFUvbH5CZJwUTl6wPNu AhVbrMz87Nrdy5iOL34DbcHzTgc9NXiI3DL8oqrtiOaFWqMEZzCm5bxciMR6wdWt9bih 6os5SllMWzn4AIXnwS75tnaPGnaePDpGz4b8qQZdliUjsaPt3EpBjyxXeU1Fe2LUZrVP CdLELyAGezKT1tc54FE+8jxcJk8y2EeLcHQpIlSVG1bIK8EXhGl8Zc36vh/ajdOBl+8O nmOA24IziTXgaE8Q3HCI9jP/+iS3iFExv+GjGfsnWhmbD6WwkqAQtu1CDEfCv+MmDhPb QlJQ== X-Gm-Message-State: ALoCoQkDDOuvxtjZtBwhsFdhRW9IXpemotQF2b9QLTTHXZ516ARCCrmaM55lXAovzbXrs06bvfkJ X-Received: by 10.152.30.74 with SMTP id q10mr1838116lah.27.1378287634318; Wed, 04 Sep 2013 02:40:34 -0700 (PDT) Received: from gizmo.nevosoft.local (ip-195-182-142-52.clients.cmk.ru. [195.182.142.52]) by mx.google.com with ESMTPSA id f17sm10163232lbo.12.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 04 Sep 2013 02:40:33 -0700 (PDT) From: olevole To: freebsd-doc@freebsd.org, freebsd-jail@freebsd.org Subject: handbook chapter for jail best practices needs for security remark Date: Wed, 04 Sep 2013 13:40:31 +0400 Message-ID: <2169287.FiyytKgDHO@gizmo.nevosoft.local> User-Agent: KMail/4.10.5 (FreeBSD/10.0-CURRENT; KDE/4.10.5; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 09:40:42 -0000 Mounting directory via nullfs when RW part mounted above RO from one filesystem is insecure for RO location, because it allows you to edit a file by hardlink on RO place, due to the fact that the files have one inode. For example (by root user): % mkdir /usr/chroot % bsdinstall jail /usr/chroot % mount_nullfs -oro /bin /usr/chroot/bin % mkdir /rw % mount_nullfs /rw /usr/chroot/root % chroot /usr/chroot % touch /bin/date touch: /bin/date: Read-only file system % cd ~ % ln /bin/date % ls -i /bin/date /root/date 58182 /bin/date 58182 /root/date (open /root/date in vi editor and change something) % vi date dd :wq! (logout from chroot) % exit (now /bin/date is corrupted) % /bin/date /bin/date: Exec format error. Binary file not executable. Such scheme when the RW data is overlaid above RO data is popular for jail hosting and described in Handbook: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails-application.html Perhaps it is worth mentioning in the article about the need to separate base to cross-device storage or place it on a read-only system. From owner-freebsd-jail@FreeBSD.ORG Wed Sep 4 15:22:42 2013 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DAF2BE91; Wed, 4 Sep 2013 15:22:42 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.52.97]) by mx1.freebsd.org (Postfix) with ESMTP id 73D3F2B4F; Wed, 4 Sep 2013 15:22:42 +0000 (UTC) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 0342ACB8C92; Wed, 4 Sep 2013 10:22:35 -0500 (CDT) Received: from 128.135.70.2 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Wed, 4 Sep 2013 10:22:35 -0500 (CDT) Message-ID: <23025.128.135.70.2.1378308155.squirrel@cosmo.uchicago.edu> In-Reply-To: <2169287.FiyytKgDHO@gizmo.nevosoft.local> References: <2169287.FiyytKgDHO@gizmo.nevosoft.local> Date: Wed, 4 Sep 2013 10:22:35 -0500 (CDT) Subject: Re: handbook chapter for jail best practices needs for security remark From: "Valeri Galtsev" To: "olevole" User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-doc@freebsd.org, freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: galtsev@kicp.uchicago.edu List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 15:22:42 -0000 Nice observation! Yet: for that to work both rw and ro portions mounted inside the same jail have to be on the same filesystem. For hardlinks to work, both parts of hardlink ("source" and "destination") should be on the same filesystem. Even though I'm not considering myself an expert in security, I will never have ro and rw filesystem (mounted inside the same jail) to live physically on the same filesystem... That said, I'm never using ezjail or some other scripts to lay out jails for me. So, apart from making a warning in handbook (which is always instructive and educational!), one may need to audit jail creating scripts. I'm certain, they are good about that (and my great respects to authors!), but taking an extra look at specific thing never hurts. Thanks. Valeri On Wed, September 4, 2013 4:40 am, olevole wrote: > Mounting directory via nullfs when RW part mounted above RO from one > filesystem > is insecure for RO location, > because it allows you to edit a file by hardlink on RO place, due to the > fact > that the files have one inode. > > For example (by root user): > > % mkdir /usr/chroot > % bsdinstall jail /usr/chroot > % mount_nullfs -oro /bin /usr/chroot/bin > % mkdir /rw > % mount_nullfs /rw /usr/chroot/root > > % chroot /usr/chroot > % touch /bin/date > touch: /bin/date: Read-only file system > > % cd ~ > % ln /bin/date > % ls -i /bin/date /root/date > 58182 /bin/date 58182 /root/date > > (open /root/date in vi editor and change something) > % vi date > dd > :wq! > > (logout from chroot) > % exit > > (now /bin/date is corrupted) > % /bin/date > /bin/date: Exec format error. Binary file not executable. > > Such scheme when the RW data is overlaid above RO data is popular for jail > hosting and described in Handbook: > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails-application.html > > Perhaps it is worth mentioning in the article about > the need to separate base to cross-device storage or place it on a > read-only > system. > > _______________________________________________ > freebsd-jail@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" > ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++