From owner-freebsd-fs@FreeBSD.ORG  Wed May 16 17:19:37 2007
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
X-Original-To: freebsd-fs@freebsd.org
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id C7A7716A400;
	Wed, 16 May 2007 17:19:37 +0000 (UTC)
	(envelope-from kris@obsecurity.org)
Received: from elvis.mu.org (elvis.mu.org [192.203.228.196])
	by mx1.freebsd.org (Postfix) with ESMTP id B6E9513C45D;
	Wed, 16 May 2007 17:19:37 +0000 (UTC)
	(envelope-from kris@obsecurity.org)
Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196])
	by elvis.mu.org (Postfix) with ESMTP id 84D401A3C19;
	Wed, 16 May 2007 10:20:29 -0700 (PDT)
Received: by obsecurity.dyndns.org (Postfix, from userid 1000)
	id CDE4351406; Wed, 16 May 2007 13:19:36 -0400 (EDT)
Date: Wed, 16 May 2007 13:19:36 -0400
From: Kris Kennaway <kris@obsecurity.org>
To: Andrew Edwards <aedwards@sandvine.com>
Message-ID: <20070516171936.GA74455@xor.obsecurity.org>
References: <20070516163305.GA73495@xor.obsecurity.org>
	<5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5230D3C40B842D4F9FB3CD368021BEF72F08F3@exchange-2.sandvine.com>
User-Agent: Mutt/1.4.2.2i
Cc: freebsd-fs@freebsd.org, freebsd-performance@freebsd.org
Subject: Re: Ufs dead-locks on freebsd 6.2
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 16 May 2007 17:19:37 -0000

On Wed, May 16, 2007 at 01:16:27PM -0400, Andrew Edwards wrote:
> Here's the backtrace from the last crash along with the output from show
> alllocks when the system was deadlocked.  I have been running
> 6.2-release and compliled with makeoptions debug=-g, invariants,
> invariant_support and witness.  I will update to 6-STABLE add
> diagnositc, debug_locks and debug_vfs_locks as per the handbook
> recommendation and retry.
> 
> Yes, when the system was un-usable I was still able to ping it.  I have
> the serial console setup as the default console so I can remotely access
> the box and break into the debugger etc.
> 
> (kgdb) bt
> #0  doadump () at pcpu.h:165
> #1  0xc059b480 in boot (howto=260) at
> /usr/src/sys/kern/kern_shutdown.c:409
> #2  0xc059b795 in panic (fmt=0xc0787b04 "Most recently used by %s\n")
>     at /usr/src/sys/kern/kern_shutdown.c:565

That's a memory use-after-free.  Check the DEBUG_MEMGUARD
infrastructure for debugging of this.

Kris