From owner-freebsd-current@FreeBSD.ORG Thu May 26 16:08:47 2005 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8968316A41C for ; Thu, 26 May 2005 16:08:47 +0000 (GMT) (envelope-from faber@pun.isi.edu) Received: from pun.isi.edu (pun.isi.edu [128.9.160.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D98443D1D for ; Thu, 26 May 2005 16:08:47 +0000 (GMT) (envelope-from faber@pun.isi.edu) Received: from pun.isi.edu (localhost [127.0.0.1]) by pun.isi.edu (8.13.3/8.13.1) with ESMTP id j4QG8kot006914; Thu, 26 May 2005 09:08:46 -0700 (PDT) (envelope-from faber@pun.isi.edu) Received: (from faber@localhost) by pun.isi.edu (8.13.3/8.13.1/Submit) id j4QG8kQT006913; Thu, 26 May 2005 09:08:46 -0700 (PDT) (envelope-from faber) Date: Thu, 26 May 2005 09:08:46 -0700 From: Ted Faber To: Peter Jeremy Message-ID: <20050526160846.GA6851@pun.isi.edu> References: <20050526001806.GA1008@pun.isi.edu> <20050526080928.GE12640@cirb503493.alcatel.com.au> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VS++wcV0S1rZb1Fb" Content-Disposition: inline In-Reply-To: <20050526080928.GE12640@cirb503493.alcatel.com.au> User-Agent: Mutt/1.4.2.1i X-url: http://www.isi.edu/~faber Cc: freebsd-current@freebsd.org Subject: Re: hard deadlock(?) on -current; some debugging info, need help X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 May 2005 16:08:47 -0000 --VS++wcV0S1rZb1Fb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, May 26, 2005 at 06:09:28PM +1000, Peter Jeremy wrote: > On Wed, 2005-May-25 17:18:06 -0700, Ted Faber wrote: > >The system slowly grinds to a halt, and the lockup seems to invlove the > >disk system. > > Nothing is waiting on physical I/O, but there are lots of locked vnodes. > I notice there's a sh(? - pid 10715) blocked on nfsreq. Can you reproduce > the problem without the NFS mounted filesystems? I have a laptop on the same network that uses NFS much less aggressively and it has never locked up. I understand that's anecdotal. It's pretty hard to reconfigure the desktop into a position where I get work done and don't use NFS here. > > > I have not found a sequence that triggers them (other than > >trying to write mail to the list to report them), and I know how > >difficult that makes things. It is common to have 2-5 a day. Even when > > >I can get to the debugger during a lockup, I cannot generate a crash > >dump - the kernel reports starting the dump and moves no bytes. > > Not nice. That suggests something below the filesystem is sick > because a filesystem deadlock won't affect the crashdump. I've let it sit a few minutes. I'll try it again next lockup, just in case. I've just typed "panic" from the debugger. If there's a better way, please let me know. > > >I've attached a dmesg from a -v boot and the kernel config (the dmesg is > >not from the lockup run). Last friday when the system locked I had a > >digital camera with me and took pictures of the ps output in the hopes > >that someone could look at them. These images are at > > > >http://www.isi.edu/~faber/tmp/deadlock/DSCN04{75,76,77,78,79,80,81,82}.JPG > > The other information we need is "show lockedvnods". This will hopefully > point to the process that started the problem. Next lockup I'll get it. -- Ted Faber http://www.isi.edu/~faber PGP: http://www.isi.edu/~faber/pubkeys.asc Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG --VS++wcV0S1rZb1Fb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFClfSOaUz3f+Zf+XsRAgnaAJ9K9L7nSPPs8N4tt0DwjPzxA2ilHACg99rn MbKS77iwXP5EWJ39haXbsjg= =IBOv -----END PGP SIGNATURE----- --VS++wcV0S1rZb1Fb--