From owner-freebsd-jail@FreeBSD.ORG Tue Oct 9 14:26:19 2007 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 142C816A41B for ; Tue, 9 Oct 2007 14:26:19 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.freebsd.org (Postfix) with ESMTP id 9EF0913C48A for ; Tue, 9 Oct 2007 14:26:18 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from vanquish.pitbpa0.priv.collaborativefusion.com (vanquish.pitbpa0.priv.collaborativefusion.com [192.168.2.61]) (SSL: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Tue, 09 Oct 2007 10:16:15 -0400 id 0005641E.470B8D2F.0000CFDB Date: Tue, 9 Oct 2007 10:16:15 -0400 From: Bill Moran To: freebsd-jail@freebsd.org Message-Id: <20071009101615.bd2601de.wmoran@collaborativefusion.com> Organization: Collaborative Fusion X-Mailer: Sylpheed 2.4.4 (GTK+ 2.10.14; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: netops@collaborativefusion.com Subject: Mysterious jail lockups X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2007 14:26:19 -0000 Has anyone else seen this? The symptoms are a jail that has no processes in it, and thus can not be stopped/killed/whatever. Only solution is to reboot the host system. Trying to jexec into the jail results in an error, so new processes can't be started therein. It doesn't happen very often, and I've been unable to reproduce it on demand. What I'm looking for at this point are whether or not anyone else has seen this, and advice on how to track it down/reproduce it, with the eventual goal of fixing the problem. It would be nice if there were a command, let's say "jkill" that killed the _jail_. There is a port called jkill that (allegedly) does this, but looking at the perl code, all it does it loop through a ps listing killing off processes. In the event of a jail with no processes, this doesn't help any. Theoretically, this would be some sort of kernel bug, whereby the reference counter to the jail is not properly decremented when processes die and thus the jail never shuts down. Given the infrequency of the occurrence and my inability to produce a reproducible case, I expect it to be challenging to track down. Any advice? -- Bill Moran Collaborative Fusion Inc. http://people.collaborativefusion.com/~wmoran/ wmoran@collaborativefusion.com Phone: 412-422-3463x4023 From owner-freebsd-jail@FreeBSD.ORG Tue Oct 9 19:49:31 2007 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F3E416A5C9 for ; Tue, 9 Oct 2007 19:49:31 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from mx00.pub.collaborativefusion.com (mx00.pub.collaborativefusion.com [206.210.89.199]) by mx1.freebsd.org (Postfix) with ESMTP id 964F313C4BA for ; Tue, 9 Oct 2007 19:49:30 +0000 (UTC) (envelope-from wmoran@collaborativefusion.com) Received: from vanquish.pitbpa0.priv.collaborativefusion.com (vanquish.pitbpa0.priv.collaborativefusion.com [192.168.2.61]) (SSL: TLSv1/SSLv3,256bits,AES256-SHA) by wingspan with esmtp; Tue, 09 Oct 2007 15:49:29 -0400 id 00056437.470BDB49.0001387F Date: Tue, 9 Oct 2007 15:49:29 -0400 From: Bill Moran To: Kurt Jaeger Message-Id: <20071009154929.8c0e8e78.wmoran@collaborativefusion.com> In-Reply-To: <20071009144903.GD12479@home.c0mplx.org> References: <20071009101615.bd2601de.wmoran@collaborativefusion.com> <20071009144903.GD12479@home.c0mplx.org> Organization: Collaborative Fusion X-Mailer: Sylpheed 2.4.4 (GTK+ 2.10.14; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-jail@freebsd.org Subject: Re: Mysterious jail lockups X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2007 19:49:31 -0000 In response to Kurt Jaeger : > Hi! > > > Has anyone else seen this? > > > > The symptoms are a jail that has no processes in it, and thus can not > > be stopped/killed/whatever. Only solution is to reboot the host system. > > Trying to jexec into the jail results in an error, so new processes can't > > be started therein. > > Have a look at this: > > http://g-rave.nl/junk/freebsd-jail-nodevdep.diff > wg. http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/89528 Thanks, Kurt. I'm following up to see if I can get someone interested in fixing this who knows that part of the code. Any suggestions? -- Bill Moran Collaborative Fusion Inc. http://people.collaborativefusion.com/~wmoran/ wmoran@collaborativefusion.com Phone: 412-422-3463x4023 From owner-freebsd-jail@FreeBSD.ORG Tue Oct 9 19:55:07 2007 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0E7916A41B for ; Tue, 9 Oct 2007 19:55:07 +0000 (UTC) (envelope-from d.hill@yournetplus.com) Received: from duane.dbq.yournetplus.com (duane.dbq.yournetplus.com [65.124.230.214]) by mx1.freebsd.org (Postfix) with ESMTP id 6EF8113C465 for ; Tue, 9 Oct 2007 19:55:07 +0000 (UTC) (envelope-from d.hill@yournetplus.com) Received: by duane.dbq.yournetplus.com (Postfix, from userid 125) id DCB066D432; Tue, 9 Oct 2007 19:51:58 +0000 (UTC) Received: by duane.dbq.yournetplus.com (Postfix, from userid 58) id C50046D44F; Tue, 9 Oct 2007 19:39:52 +0000 (UTC) X-Spam-Flag: NO X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) X-Spam-Level: X-Spam-Status: Reqd:5.0 Hits:0.0 Learn:ham Tests:none Received: from [192.168.1.10] (unknown [192.168.1.1]) by duane.dbq.yournetplus.com (Postfix) with ESMTP id 7DD2D6D431; Tue, 9 Oct 2007 19:39:51 +0000 (UTC) Date: Tue, 9 Oct 2007 19:39:50 +0000 (UTC) From: D Hill X-X-Sender: d.hill@duane.dbq.yournetplus.com To: Bill Moran In-Reply-To: <20071009101615.bd2601de.wmoran@collaborativefusion.com> Message-ID: <20071009192346.S71315@duane.dbq.yournetplus.com> References: <20071009101615.bd2601de.wmoran@collaborativefusion.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-jail@freebsd.org, netops@collaborativefusion.com Subject: Re: Mysterious jail lockups X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2007 19:55:07 -0000 On Tue, 9 Oct 2007 at 10:16 -0400, wmoran@collaborativefusion.com confabulated: > Has anyone else seen this? > > The symptoms are a jail that has no processes in it, and thus can not > be stopped/killed/whatever. Only solution is to reboot the host system. > Trying to jexec into the jail results in an error, so new processes can't > be started therein. > > It doesn't happen very often, and I've been unable to reproduce it on > demand. What I'm looking for at this point are whether or not anyone > else has seen this, and advice on how to track it down/reproduce it, with > the eventual goal of fixing the problem. > > It would be nice if there were a command, let's say "jkill" that killed > the _jail_. There is a port called jkill that (allegedly) does this, but > looking at the perl code, all it does it loop through a ps listing > killing off processes. In the event of a jail with no processes, this > doesn't help any. > > Theoretically, this would be some sort of kernel bug, whereby the > reference counter to the jail is not properly decremented when processes > die and thus the jail never shuts down. Given the infrequency of the > occurrence and my inability to produce a reproducible case, I expect > it to be challenging to track down. > > Any advice? Same thing seen here running: FreeBSD ns1jail.example.com 6.2-RELEASE-p7 FreeBSD 6.2-RELEASE-p7 #1: Thu Aug 9 18:59:52 UTC 2007 d.hill@ns1jail.example.com:/usr/obj/usr/src/sys/NS1JAIL i386 After I '/etc/rc.d/jail stop ns1_ynp' it will linger and finally stop. Other times it will still show a day or two later. Displaying the process list on the host does not show any process running with the 'J' jail indicator. Like you, the host has to be rebooted. ----- _|_ (_| | From owner-freebsd-jail@FreeBSD.ORG Sat Oct 13 10:33:34 2007 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E918616A41A for ; Sat, 13 Oct 2007 10:33:33 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [82.208.36.70]) by mx1.freebsd.org (Postfix) with ESMTP id AA15D13C478 for ; Sat, 13 Oct 2007 10:33:33 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id E4ABC19E02A; Sat, 13 Oct 2007 12:33:31 +0200 (CEST) Received: from [192.168.1.2] (r3a200.net.upc.cz [213.220.192.200]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTP id 334E119E027; Sat, 13 Oct 2007 12:33:29 +0200 (CEST) Message-ID: <47109F59.30602@quip.cz> Date: Sat, 13 Oct 2007 12:35:05 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: cz, cs, en, en-us MIME-Version: 1.0 To: arch@freebsd.org References: <470E5BFB.4050903@elischer.org> In-Reply-To: <470E5BFB.4050903@elischer.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-jail@freebsd.org, Julian Elischer Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2007 10:33:34 -0000 Julian Elischer wrote: [...] > I'd like to be able to say.. > I want to share the filesystem, and unix domain sockets but have a > separate routing domain for my processes, or maybe just > for some sockets. But someone else may want to have > complete separation with everything up to and including > separate userID spaces. > > My question to you, the reader, is: > what aspects of virtualisation (the appearance of multiple instances > of some resource) would you like to see in the system? > > Even a discussion as to how to frame this question is up for discussion. > > We don't even have a taxonomy to discus the issue. > > Julian It would be nice to have something from vserver, something from zones, from xen, from jails etc. From my point of view: CPU limits - specified as relative part of shares (container can get more CPU power if CPU is not 100% loaded) or set to absolute (container can't get more than specified CPU power, so one can use it to test applications on slow CPUs etc.) Memory limits - same as CPU Disk - it would be nice if I can set how many disk space each container can use. (with similar interface as disk quotas - soft+hard limits and space+inodes). Maybe setting of disk I/O in similar style as CPU and memory limits above. UIDs - independent UIDs in containers. In relation to UIDs, one can use disk quotas inside containers. Network bandwidth - same as CPU and memory Each container can have multiple IPs, can have own routing, firewalling (vimage is nice project) Hierarchical structure - container can contain another containers. Nested containers inherit/share resources from parent container, or can be limited to some part of them. For example container1 could have 5 IPs, 40% of CPU, 200MB of memory, 50GB of disk space, container1A could have 2 IPs, 50% of CPU of its parent (container1), 50MB memory, 10GB disk space, container1B could have no IP, 10% CPU of parent, 100MB memory, no disk space limits. Other not specified resources and resources for container1C are shared within parent container. Nested containers could be used to set some limits (CPU, memory, disk, bandwidth) to more than one container at a time, I could set some limits to container2 and doesn't matter of setting any limits/portioning to container2A and container2B. host OS --- container1 --- container1A | |-- container1B | \-- container1C | +-- container2 --- container2A | \-- container2B | \-- container3 Others as said by James Gritton. I know my view is too complex, but it is only subject for discussion. I am CCing freebsd-jail@freebsd.org, as it is related to Jails. (discussion continue on arch@freebsd.org) Miroslav Lachman