From owner-freebsd-stable@FreeBSD.ORG Sat Aug 20 16:48:30 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E35821065677; Sat, 20 Aug 2011 16:48:30 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EE0C48FC1C; Sat, 20 Aug 2011 16:48:29 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA12940; Sat, 20 Aug 2011 19:48:27 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Quoig-000O3V-LU; Sat, 20 Aug 2011 19:48:26 +0300 Message-ID: <4E4FE55A.9000101@FreeBSD.org> Date: Sat, 20 Aug 2011 19:48:26 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0) Gecko/20110819 Thunderbird/6.0 MIME-Version: 1.0 To: Steven Hartland References: <47F0D04ADF034695BC8B0AC166553371@multiplay.co.uk><4E43E272.1060204@FreeBSD.org><62BF25D0ED914876BEE75E2ADF28DDF7@multiplay.co.uk><4E440865.1040500@FreeBSD.org><6F08A8DE780545ADB9FA93B0A8AA4DA1@multiplay.co.uk><4E441314.6060606@FreeBSD.org><2C4B0D05C8924F24A73B56EA652FA4B0@multiplay.co.uk><4E48D967.9060804@FreeBSD.org><9D034F992B064E8092E5D1D249B3E959@multiplay.co.uk><4E490DAF.1080009@FreeBSD.org><796FD5A096DE4558B57338A8FA1E125B@multiplay.co.uk><4E491D01.1090902@FreeBSD.org><570C5495A5E242F7946E806CA7AC5D68@multiplay.co.uk><4E4AD35C.7020504@FreeBSD.org><6A7238AED44542A880B082A40304D940@multiplay.co.uk><4E4BA21F.6010805@FreeBSD.org><581C95046B0948FC82D6F2E86948F87B@multiplay.co.uk><4E4BBA7F.30907@FreeBSD.org><88A6CE3E8B174E0694A3A9A5283479B4@multiplay.co.uk><4E4C22D6.6070407@FreeBSD.org><4019027648B5493AAC4B654BD821DE88@multiplay.co.uk><4E4F8631.1070300@FreeBSD.org> <4E4F8821.80108@Fre eBSD.org> <82E865FBA30747078AF6EE3C1701F973@multiplay.co.uk> In-Reply-To: <82E865FBA30747078AF6EE3C1701F973@multiplay.co.uk> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-jail@FreeBSD.org, freebsd-stable@FreeBSD.org Subject: Re: debugging frequent kernel panics on 8.2-RELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Aug 2011 16:48:31 -0000 on 20/08/2011 18:51 Steven Hartland said the following: > ----- Original Message ----- From: "Andriy Gapon" > >> BTW, I suspect the following scenario, but I am not able to verify it either via >> testing or in the code: >> - last process in a dying jail exits >> - pr_uref of the jail reaches zero >> - pr_uref of prison0 gets decremented >> - you attach to the jail and resurrect it >> - but pr_uref of prison0 stays decremented >> >> Repeat this enough times and prison0.pr_uref reaches zero. >> To reach zero even sooner just kill enough of non-jailed processes. > > I've just checked across a number of the panic dumps from the > past few days and they all have prison0.pr_uref = 0 which confirms > the cause of the panic. > > I've tried scripting continuous jail start stops, but even after 1000's > of iterations have been unable to trigger this on my test machine, so > I'm going to dig into the jail code to see if I can find out how its > incorrectly decrementing prison0 via inspection. Steve, thanks for doing this! I'll reiterate my suspicion just in case - I think that you should look for the cases where you stop a jail, but then re-attach and resurrect the jail before it's completely dead. -- Andriy Gapon