From owner-freebsd-hackers  Mon Dec 17 13:41:47 2001
Delivered-To: freebsd-hackers@freebsd.org
Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2])
	by hub.freebsd.org (Postfix) with ESMTP id 51A9A37B41E
	for <freebsd-hackers@FreeBSD.ORG>; Mon, 17 Dec 2001 13:41:30 -0800 (PST)
Received: (from dillon@localhost)
	by apollo.backplane.com (8.11.6/8.9.1) id fBHLfSc33112;
	Mon, 17 Dec 2001 13:41:28 -0800 (PST)
	(envelope-from dillon)
Date: Mon, 17 Dec 2001 13:41:28 -0800 (PST)
From: Matthew Dillon <dillon@apollo.backplane.com>
Message-Id: <200112172141.fBHLfSc33112@apollo.backplane.com>
To: Martin Blapp <mb@imp.ch>
Cc: <freebsd-hackers@FreeBSD.ORG>
Subject: Re: deadlock with softupdates ?
References:  <20011217223123.P59342-100000@levais.imp.ch>
Sender: owner-freebsd-hackers@FreeBSD.ORG
Precedence: bulk
List-ID: <freebsd-hackers.FreeBSD.ORG>
List-Archive: <http://docs.freebsd.org/mail/> (Web Archive)
List-Help: <mailto:majordomo@FreeBSD.ORG?subject=help> (List Instructions)
List-Subscribe: <mailto:majordomo@FreeBSD.ORG?subject=subscribe%20freebsd-hackers>
List-Unsubscribe: <mailto:majordomo@FreeBSD.ORG?subject=unsubscribe%20freebsd-hackers>
X-Loop: FreeBSD.ORG

    If it is a vnode deadlock you have to find which process or
    processes it is deadlocking against.

    Do a ps -axl -M vmcore.X -N kernel.X  on the kernel core,
    pick out all the processes blocked on inode or whatever,
    then from gdb go to each process (proc N) and do a backtrace
    (back).

					-Matt
					Matthew Dillon 
					<dillon@backplane.com>


:
:
:Hi Matt,
:
:>     Yahoo noticed a deadlock in the vnode recycling code.  If your machine
:>     below is hitting the kern.maxvnodes limit it could be the same thing.
:>     The way to tell is to generate a kernel core along with the debug
:>     version of the kernel binary (/usr/src/sys/compile/<KERNEL>/kernel.debug),
:>     and backtrace each locked-up process to see if any of them are going
:>     through the recycle code.
:
:It cannot be kern.maxvnodes, cause the newsserver is using the cyclic
:news filesystem. And there is also enough ram.
:
:It seems to be a vnode deadlock:
:
:(kgdb) defproc 2
:    2 d78cd0c0 d6445000    0     0     0  100204 3  pagedaemon   inode c2c0c200
: frame 0 at 0xd6447e24: ebp d6447e50, eip 0xc017a529 <tsleep+417>:      mov    0x141(%ebx),%al
: frame 1 at 0xd6447e50: ebp d6447e74, eip 0xc0171c24 <acquire+140>:     mov    %eax,%esi
: frame 2 at 0xd6447e74: ebp d6447e98, eip 0xc0171ea5 <lockmgr+581>:     mov    %eax,%edi
: frame 3 at 0xd6447e98: ebp d6447eb0, eip 0xc01a2008 <vop_stdunlock+12>:       jmp    0xc01a200e <vop_stdunlo>
: frame 4 at 0xd6447eb0: ebp d6447ebc, eip 0xc022e44d <default_pager_alloc+1>:   leave
: frame 5 at 0xd6447ebc: ebp d6447ee4, eip 0xc01ac04b <vn_lock+155>:     add    $0x4,%esp
: frame 6 at 0xd6447ee4: ebp d6447f00, eip 0xc01a4e18 <vget+148>:        mov    %eax,%esi
: frame 7 at 0xd6447f00: ebp d6447f84, eip 0xc023a52d <vm_pageout_scan+873>:     add    $0xc,%esp
: frame 8 at 0xd6447f84: ebp d6447f9c, eip 0xc023af53 <pagedaemon_wakeup+31>:    movl   $0x0,0xc02eff1c
:
:(kgdb) x/x 0xd6447f00
:0xd6447f00:     0xd6447f84
:(kgdb) x/10 0xd6447f00
:0xd6447f00:     0xd6447f84      0xc023a52d      0xd7e5a180      0x00040002
:0xd6447f10:     0xd47c2c60      0x00000000      0x00000000      0x00000000
:0xd6447f20:     0xd7e5a180      0x00000020
:(kgdb) showcache 0xd7e5a180
:history.hash
:
:Can you help me to debug this bug ?
:
:Martin

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message