From owner-freebsd-jail@freebsd.org Thu Oct 6 12:01:35 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0043BAF54D8 for ; Thu, 6 Oct 2016 12:01:35 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43E03B53 for ; Thu, 6 Oct 2016 12:01:33 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 51E2A28473 for ; Thu, 6 Oct 2016 14:01:29 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 91AE12840C for ; Thu, 6 Oct 2016 14:01:28 +0200 (CEST) To: freebsd-jail@freebsd.org From: Miroslav Lachman <000.fbsd@quip.cz> Subject: some (random) jails are not started after reboot Message-ID: <57F63D18.1060701@quip.cz> Date: Thu, 6 Oct 2016 14:01:28 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2016 12:01:35 -0000 Hi, we have a few machines running jails as "vps". All jails are maintained by me and have normal setup with one IPv4. We are running usual web things in jails like Apache, PHP, Python, MySQL. I noticed that some jails are not started after reboot on one machine. (after security update to 10.3-RELEASE-p6 in the summer) I updated our machines last night to 10.3-RELEASE-p9 and one jail on the same machine did not start. Console log of this jail does not contain any startup messages. The last record are from previous start. I can start the jail manually without any problem. No errors. Only one machine has this problem. Problem started a few months back. (The machine was started as some old 8.x install and is continually upgraded to the latest 10.3) Base system is on UFS, jails are on ZFS if it matter. This is production machine so I can't play with it too much. But does anybody else seen it too? How can I debug it on next reboot? Miroslav Lachman From owner-freebsd-jail@freebsd.org Thu Oct 6 16:40:39 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BC0DAF6690 for ; Thu, 6 Oct 2016 16:40:39 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.gritton.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BD10FED for ; Thu, 6 Oct 2016 16:40:38 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) by gritton.org (8.15.2/8.15.2) with ESMTPS id u96GBKs6056471 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 6 Oct 2016 10:11:20 -0600 (MDT) (envelope-from jamie@freebsd.org) Received: (from www@localhost) by gritton.org (8.15.2/8.15.2/Submit) id u96GBKn6056454; Thu, 6 Oct 2016 10:11:20 -0600 (MDT) (envelope-from jamie@freebsd.org) X-Authentication-Warning: gritton.org: www set sender to jamie@freebsd.org using -f To: freebsd-jail@freebsd.org Subject: Re: some (random) jails are not started after reboot X-PHP-Originating-Script: 0:rcube.php MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 06 Oct 2016 10:11:20 -0600 From: James Gritton In-Reply-To: <57F63D18.1060701@quip.cz> References: <57F63D18.1060701@quip.cz> Message-ID: <7c2f0434326257c5d2cbbb97a5c5e787@gritton.org> X-Sender: jamie@freebsd.org User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2016 16:40:39 -0000 On 2016-10-06 06:01, Miroslav Lachman wrote: > Hi, > > we have a few machines running jails as "vps". All jails are > maintained by me and have normal setup with one IPv4. We are running > usual web things in jails like Apache, PHP, Python, MySQL. > > I noticed that some jails are not started after reboot on one machine. > (after security update to 10.3-RELEASE-p6 in the summer) > I updated our machines last night to 10.3-RELEASE-p9 and one jail on > the same machine did not start. > Console log of this jail does not contain any startup messages. The > last record are from previous start. > I can start the jail manually without any problem. No errors. > > Only one machine has this problem. Problem started a few months back. > (The machine was started as some old 8.x install and is continually > upgraded to the latest 10.3) > Base system is on UFS, jails are on ZFS if it matter. > > This is production machine so I can't play with it too much. But does > anybody else seen it too? > How can I debug it on next reboot? Two handy things. One is to make sure every jail has an exec.consolelog file set, so if it gets partway through startup and then goes away, you can see what it did up until then. Second is the "-v" flag to jail(8). Add it to jail_flags in your rc.conf, and then comment out the bit in /etc/rc.d/jail that removes the temp file for jail(8)'s output ($_tmp). Watch out for parallel starts - they just went to /dev/null until recently. - Jamie