From owner-freebsd-stable@FreeBSD.ORG Wed Nov 12 18:52:54 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 801D81065676 for ; Wed, 12 Nov 2008 18:52:54 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by mx1.freebsd.org (Postfix) with ESMTP id 218508FC13 for ; Wed, 12 Nov 2008 18:52:53 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by QMTA01.emeryville.ca.mail.comcast.net with comcast id eESt1a00r0cQ2SLA1Jst9y; Wed, 12 Nov 2008 18:52:53 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA10.emeryville.ca.mail.comcast.net with comcast id eJsr1a00B2P6wsM8WJsrm4; Wed, 12 Nov 2008 18:52:52 +0000 X-Authority-Analysis: v=1.0 c=1 a=becna3ql8DEA:10 a=0kYo9GeZAlkA:10 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=e7F8RB_cu7ubLTsXovMA:9 a=XfXpIeQpwVlGu-eAZygA:7 a=yizOmbJetH5hUlKGhPG31p_kcR4A:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 5458F5C19; Wed, 12 Nov 2008 10:52:51 -0800 (PST) Date: Wed, 12 Nov 2008 10:52:51 -0800 From: Jeremy Chadwick To: Tim Bishop Message-ID: <20081112185251.GA1838@icarus.home.lan> References: <20081112175826.GD26195@carrick.bishnet.net> <20081112182234.GE26195@carrick.bishnet.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081112182234.GE26195@carrick.bishnet.net> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: Tim Bishop , freebsd-stable@freebsd.org, David Peall Subject: Re: System deadlock when using mksnap_ffs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2008 18:52:54 -0000 On Wed, Nov 12, 2008 at 06:22:35PM +0000, Tim Bishop wrote: > On Wed, Nov 12, 2008 at 08:10:50PM +0200, David Peall wrote: > > > FreeBSD paladin 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #8: Mon Nov 10 > > > 20:49:51 GMT 2008 tdb@paladin:/usr/obj/usr/src/sys/PALADIN amd64 > > > > > > I run the mksnap_ffs command to take the snapshot and some time later > > > the system completely freezes up: > > > > If the file system is UFS2 it's a known problem but should have been > > fixed. > > http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues > > > > ident /boot/kernel/kernel | grep subr_sleepqueue > > > > version should be greater than 1.39.2.3? > > Yes it's UFS2, and yes it's greater than 1.39.2.3: > > $FreeBSD: src/sys/kern/subr_sleepqueue.c,v 1.39.2.5 2008/09/16 20:01:57 jhb Exp $ > > Are you sure the problem referenced on that page is the same? It talks > about "dog slow" snapshotting, which I see on other filesystems and > machines. But in this particular case the system is dead, and does not > recover. This problem gets brought up every few weeks on average, I think. The problem still exists regardless of subr_sleepqueue.c 1.39.2.3. I can still reproduce it on every FreeBSD box I have access to. The last time I tried it was on 2008/10/24, on a RELENG_7 system built from source csup'd on the same day. The result of "dump -L -0 -a -f /someplace/fs.dump /usr" was the same: the system became more or less unusable (meaning to the point where you might as well not try to do anything with it because it's so incredibly slow, especially with anything I/O-bound), and mksnap_ffs remained in the following state for many, many minutes: load: 0.00 cmd: mksnap_ffs 10480 [wdrain] 0.00u 0.06s 0% 1076k Hitting ^C at this point took 4-5 *full minutes* to execute. While ^C was (hopefully) executing, the process remained in wdrain state as well. After the process was terminated fully, the system was again responsive. That filesystem (/usr) I was dumping: Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/ad4s1f 163815904 3834316 146876316 3% 254752 20942046 1% /usr -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |