From owner-freebsd-current@FreeBSD.ORG Mon Oct 25 05:24:40 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C49516A4CE for ; Mon, 25 Oct 2004 05:24:40 +0000 (GMT) Received: from digger1.defence.gov.au (digger1.defence.gov.au [203.5.217.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 29DE443D39 for ; Mon, 25 Oct 2004 05:24:37 +0000 (GMT) (envelope-from wilkinsa@squash.dsto.defence.gov.au) Received: from ednmsw503.dsto.defence.gov.au (ednmsw503.dsto.defence.gov.au [131.185.2.150]) by digger1.defence.gov.au with ESMTP id i9P5NcZg023001 for ; Mon, 25 Oct 2004 14:53:38 +0930 (CST) Received: from muttley.dsto.defence.gov.au (unverified) by ednmsw503.dsto.defence.gov.au (Content Technologies SMTPRS 4.3.10) with ESMTP id for ; Mon, 25 Oct 2004 14:54:26 +0930 Received: from ednex501.dsto.defence.gov.au (ednex501.dsto.defence.gov.au [131.185.2.81]) by muttley.dsto.defence.gov.au (8.11.3/8.11.3) with ESMTP id i9P5HVh13620 for ; Mon, 25 Oct 2004 14:47:31 +0930 (CST) Received: from squash.dsto.defence.gov.au ([131.185.40.212]) by ednex501.dsto.defence.gov.au with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id RZJCG1HG; Mon, 25 Oct 2004 14:47:23 +0930 Received: from squash.dsto.defence.gov.au (localhost [127.0.0.1]) by squash.dsto.defence.gov.au (8.12.11/8.12.11) with ESMTP id i9P5HvDx031262 for ; Mon, 25 Oct 2004 14:47:57 +0930 (CST) (envelope-from wilkinsa@squash.dsto.defence.gov.au) Received: (from wilkinsa@localhost) by squash.dsto.defence.gov.au (8.12.11/8.12.11/Submit) id i9P5Hvta031261 for freebsd-current@freebsd.org; Mon, 25 Oct 2004 14:47:57 +0930 (CST) (envelope-from wilkinsa) Date: Mon, 25 Oct 2004 14:47:57 +0930 From: "Wilkinson, Alex" To: freebsd-current@freebsd.org Message-ID: <20041025051757.GA31094@squash.dsto.defence.gov.au> Mail-Followup-To: freebsd-current@freebsd.org References: <20041018025733.GA2026@squash.dsto.defence.gov.au> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.6i Subject: Re: RELENG_5 panic [nic: _mtx_lock_sleep: recursed on non-recursivemutex nfsd_mtx] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Oct 2004 05:24:40 -0000 Robert, The panic doesn't occur any longer since I upgraded world/kernel. Therefore what you have written below would be pointless. Are you suspecting that the bug is still there but not being triggered ? I can downgrade world/kernel for testing if possible and needed ? - aW 0n Mon, Oct 18, 2004 at 05:22:01AM -0400, Robert Watson wrote: Hmm. There was a bug in nfs_serv.c corrected on 2004/08/25 that corrected an incorrect lock pair, but I don't know of any related bugs corrrected since then. If it's possible to get back to a point where this can be reproduced, configuring your kernel to run with KTR and to use KTR_LOCK would be quite helpful You can find a bit of information on using KTR here: http://www.watson.org/~robert/freebsd/netperf/ktr/ Basically, once it panics, you can use the KTR-related commands in DDB to inspect the KTR buffer, and I believe also extract the KTR log from a core dump. Using the set of KTR_COMPILE flags I have on that web page would be helpeful. Basically, this generates a trace of locking, context switch, interrupt, callout, memory allocation, and system call events. The problem likely occurred pretty immediately prior to the panic. If this is timing-related, it could be KTR disrupts the timing sufficiently to prevent it. More likely, it went away because the work load on the NFS server changed, or the directory layout changed, or some other factor that caused the passage through the NFS code to change. Thanks! Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research > > - aW > > > > 0n Sun, Oct 17, 2004 at 10:19:34AM -0400, Robert Watson wrote: > > On Fri, 15 Oct 2004, Wilkinson, Alex wrote: > > > Hi all, > > > > I currently get a panic with "nfs_server_enable=YES" in /etc/rc.conf. > > > > OS: FreeBSD 5.3-BETA4 #2: Tue Sep 14 13:55:30 UTC 2004 > > > > Backtrace > > --------- > > > > panic: _mtx_lock_sleep: recursed on non-recursive mutex nfsd_mtx @ /usr /src/sys/nfsserver/nfs_serv.c:1947 > > Is the NFS server code compiled into your kernel, or is it getting loaded > as a module? Do you have any other NFS-related entries in /etc/rc.conf? > Could you show the output of "show locks" and "show witness" with witness > compiled into the kernel? > > Thanks! > > Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > robert@fledge.watson.org Principal Research Scientist, McAfee Research > > > > cpuid = 0 > > KDB: enter: panic > > [thread 100074] > > Stopped at kdb_enter+0x32: leave > > db> tr > > kdb_enter(c068ba66,0,c068aed8,dd2ed90c,c1b6b340) at kdb_enter+0x32 > > panic(c068aed8,c069885f,c0698617,79b,c0698617) at panic+0x1b0 > > _mtx_lock_sleep(c071c940,c1b6b340,0,c0698617,79b) at _mtx_lock_sleep+0x16e > > _mtx_lock_flags(c071c940,0,c0698617,79b,0) at _mtx_lock_flags+0xb0 > > nfsrv_create(c1eb6800,c1b98800,c1b6b340,dd2edc8c,0) at nfsrv_create+0x8c4 > > nfssvc(c1b6b340,dd2edd14,8,0,2) at nfssvc+0x6ea > > syscall(2f,2f,2f,0,0) at syscall+0x13b > > Xint0x80_syscall() at Xint0x80_syscall+0x1f > > --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280c60bf, esp = 0xbfbfeb1c, eb p = 0xbfbfeb38 --- > > > > > > - aW > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >