From owner-freebsd-hackers@FreeBSD.ORG Tue Aug 22 20:38:18 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0084616A4E7; Tue, 22 Aug 2006 20:38:18 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D1A8B43D46; Tue, 22 Aug 2006 20:38:16 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k7MKcG8n087674; Tue, 22 Aug 2006 15:38:16 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44EB6B37.9090302@centtech.com> Date: Tue, 22 Aug 2006 15:38:15 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: pjd@freebsd.org References: <44E45C3F.6060901@centtech.com> <20060817122502.GF1483@deviant.kiev.zoral.com.ua> <44E4614B.9000902@centtech.com> <44E482E8.50509@centtech.com> <20060817150423.GA20768@deviant.kiev.zoral.com.ua> <44E48610.3000407@centtech.com> In-Reply-To: <44E48610.3000407@centtech.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1709/Tue Aug 22 14:34:50 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: Kostik Belousov , FreeBSD Hackers Subject: Re: 6-STABLE snapshot (background fsck) lock-up X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2006 20:38:18 -0000 On 08/17/06 10:06, Eric Anderson wrote: > On 08/17/06 10:04, Kostik Belousov wrote: >> On Thu, Aug 17, 2006 at 09:53:28AM -0500, Eric Anderson wrote: >>> On 08/17/06 07:30, Eric Anderson wrote: >>>> On 08/17/06 07:25, Kostik Belousov wrote: >>>>> On Thu, Aug 17, 2006 at 07:08:31AM -0500, Eric Anderson wrote: >>>>>> I've seen this several times now, but this time I got a dump. >>>>>> Basically, the system comes up after unclean shutdown, throws a bunch of >>>>>> filesystems into the background fsck list, and begins 60 seconds later. >>>>>> After some amount of time, the system will lock up, presumably hung >>>>>> waiting on locks. >>>>>> >>>>>> Here's some info: >>>>>> lock type bufwait: EXCL (count 1) by thread 0xffffff0128812980 (pid 804) >>>>>> lock type getblk: EXCL (count 1) by thread 0xffffff0128812980 (pid >>>>>> 804)Locked vnodes >>>>>> >>>>>> 0xffffff01c5c9f000: tag ufs, type VREG >>>>>> usecount 1, writecount 0, refcount 13656 mountedhere 0 >>>>>> flags (VV_SYSTEM) >>>>>> v_object 0xffffff01b09542a0 ref 0 pages 1691637 >>>>>> lock type snaplk: EXCL (count 1) by thread 0xffffff0128812980 (pid >>>>>> 804) >>>>>> ino 4, on dev label/vol4 >>>>>> >>>>>> >>>>>> >>>>>> How can I use kgdb to get more debugging information from this? >>>>>> >>>>> Look at the >>>>> http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug-deadlocks.html >>>>> for instructions for ddb. >>>>> >>>>> What is the date of the system checkout ? >>>> 6-STABLE from ~8/12/06. >>>> >>>> So there's not much that can be done with the vmcore? >>>> >>>> I'll add the options to my kernel config (I was missing a few), and I >>>> suppose wait for another deadlock. >>> Ok, had another one. Here are screenshots: >>> >>> http://www.googlebit.com/freebsd/snapshots/ >>> >>> Is it the sysctl that is triggering it? >> Hmm, unbelievable. It would be much simpler for you to set up serial >> console. > > Not really, at least right now. I'm doing this remotely, so it'll be > some time before I can do that. > >> Anyway, you use g_journal, and the problem seems to be related to some >> geom locking. pjd@ is the right person to contact. I added him to the Cc:. > > Ahah. Ok. Thank you! > > Eric Pawel, Did you get a chance to look at those screenshots? I'm curious to know if you also think it is gjournal related. I've stopped loading gjournal, and I've had no other related deadlocks. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------