From owner-freebsd-fs@FreeBSD.ORG Mon Apr 23 18:18:50 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4512116A400 for ; Mon, 23 Apr 2007 18:18:50 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from smtp2.sccoast.net (smtp2.sccoast.net [66.153.203.174]) by mx1.freebsd.org (Postfix) with ESMTP id 230A613C45B for ; Mon, 23 Apr 2007 18:18:49 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from 31.64-150-133-net.sccoast.net ([64.150.133.31]:62561 helo=volatile.chemikals.org) by smtp2.sccoast.net with esmtp (Exim 4.60) (envelope-from ) id 1Hg37h-0006nn-F2; Mon, 23 Apr 2007 14:18:49 -0400 Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.13.8/8.13.8) with ESMTP id l3NIIk6e001334; Mon, 23 Apr 2007 14:18:47 -0400 (EDT) (envelope-from morganw@chemikals.org) Date: Mon, 23 Apr 2007 14:18:46 -0400 (EDT) From: Wes Morgan To: Kris Kennaway In-Reply-To: <20070422034253.GA88705@xor.obsecurity.org> Message-ID: <20070423141440.I997@volatile.chemikals.org> References: <20070421223552.F969@volatile.chemikals.org> <20070422034253.GA88705@xor.obsecurity.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: apparent filesystem-related hangs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2007 18:18:50 -0000 On Sat, 21 Apr 2007, Kris Kennaway wrote: > On Sat, Apr 21, 2007 at 10:51:10PM -0400, Wes Morgan wrote: >> I have several filesystems built on top of a gconcat volume consisting of >> 2 300gb and 1 500gb drive. The /usr partition constitutes the bulk of it. >> The volume is sitting around 95% full, and twice I've had the system >> become "hung" in what I believe are filesystem operations. It seems to >> crop up when rtorrent tries to create files that would fill up the system, >> but of course it doesn't actually reserve it until it is used. Any running >> process remains responsive until it needs to access the disk. >> >> The system was running a fairly recent 6.2-stable, March 29, but I've >> since updated to the most recent -stable. >> >> If anyone can give me some pointers as to how to tell exactly where these >> processes are hanging, I can try to reproduce it... But it takes a while >> to fsck the volume so I don't want to have to try too many things. If it's >> not a known problem I can then submit a PR. > > > DEBUG_LOCKS, DEBUG_VFS_LOCKS, INVARIANTS, INVARIANT_SUPPORT, then > 'show lockedvnods' and 'alltrace' from ddb via the serial console when > it hangs. Ok, I was easily able to duplicate this twice. The second time was by accident, and rtorrent held the only locked vnode. Leaving top running in a terminal, I was able to lock up rtorrent and then purposely lock another process that was visible. Both locked in "nbufkv": 1011 morganw 1 -4 0 73440K 20272K nbufkv 0:04 0.20% rtorrent 996 morganw 1 -4 0 7592K 5084K nbufkv 0:00 0.00% pine The full dumps from ddb can be found at http://volatile.chemikals.org/ddb.txt.gz. If there is no additional information that would be useful for a PR, I can go ahead and create one. -- This .signature sanitized for your protection