From owner-freebsd-stable@FreeBSD.ORG Fri Dec 15 19:14:41 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 03A9316A575 for ; Fri, 15 Dec 2006 19:14:41 +0000 (UTC) (envelope-from sven@dmv.com) Received: from smtp-gw-cl-c.dmv.com (smtp-gw-cl-c.dmv.com [216.240.97.41]) by mx1.FreeBSD.org (Postfix) with ESMTP id 704F243D9A for ; Fri, 15 Dec 2006 19:07:29 +0000 (GMT) (envelope-from sven@dmv.com) Received: from mail-gw-cl-a.dmv.com (mail-gw-cl-a.dmv.com [216.240.97.38]) by smtp-gw-cl-c.dmv.com (8.12.10/8.12.10) with ESMTP id kBFJ9A0F050732; Fri, 15 Dec 2006 14:09:10 -0500 (EST) (envelope-from sven@dmv.com) Received: from lanshark.dmv.com (lanshark.dmv.com [216.240.97.46]) by mail-gw-cl-a.dmv.com (8.12.9/8.12.9) with ESMTP id kBFJ95Xi099778; Fri, 15 Dec 2006 14:09:05 -0500 (EST) (envelope-from sven@dmv.com) From: Sven Willenberger To: Kris Kennaway In-Reply-To: <20061215181548.GA58555@xor.obsecurity.org> References: <20061205.004323.78708386.hrs@allbsd.org> <20061204160949.GM35681@deviant.kiev.zoral.com.ua> <20061205.123805.59655403.hrs@allbsd.org> <1166194879.6317.11.camel@lanshark.dmv.com> <20061215181548.GA58555@xor.obsecurity.org> Content-Type: text/plain Date: Fri, 15 Dec 2006 14:12:16 -0500 Message-Id: <1166209936.6317.21.camel@lanshark.dmv.com> Mime-Version: 1.0 X-Mailer: Evolution 2.6.3 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.39 X-Scanned-By: MIMEDefang 2.48 on 216.240.97.38 Cc: stable@freebsd.org Subject: Re: panic in nfsd on 6.2-RC1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2006 19:14:41 -0000 On Fri, 2006-12-15 at 13:15 -0500, Kris Kennaway wrote: > On Fri, Dec 15, 2006 at 10:01:19AM -0500, Sven Willenberger wrote: > > On Tue, 2006-12-05 at 12:38 +0900, Hiroki Sato wrote: > > > Kostik Belousov wrote > > > in <20061204160949.GM35681@deviant.kiev.zoral.com.ua>: > > > > > > ko> What version of sys/nfsserver/nfs_serv.c do you use ? If it is older than > > > ko> 1.156.2.7, please, update the system. > > > > > > Thanks, I updated it just now and see how it works. > > > > > > -- > > > | Hiroki SATO > > > > I was/am having the same issue. Updating world (6.2-stable) to include > > the above update sadly did not fix the problem for me. This is an amd64 > > box with only one client connecting to it via nfs. Reading further it > > may seem to be an issue with rpc.statd and/or rpc.lockd. As I only have > > one client connecting and it is being used as mail storage (i.e. the > > client pops/imaps the storage) would be safe to not using fcntl forwards > > over the wire? Is this same issue present in 6.1-RELENG? I am really at > > my wits end at this point and for the first time am actually considering > > moving to another OS (solaris more than likely) as I cannot have these > > types of issues interrupting services every couple days. > > > > What other information (spefically) can I provide to help the devs > > figure out what is going on? What can I do in the meantime to have some > > semblence of stability? I assume downgrading to 5.5-RELENG is out of the > > question but perhaps disabling SMP? > > Just to confirm, can you please post the panic backtrace you are > seeing? And can you explain what you mean by "may seem to be an issue > with rpc.statd and/or rpc.lockd"? > > Sometimes people think they're seeing the same problem as someone else > when really it's a completely different problem in the same subsystem, > so I'd like to rule that out here. > > Kris Well I have now added kdb and invariants/witness support to the kernel so I should be able to get some backtrace the next time it happens. Currently, the system just locks and no error is displayed on the console or /var/log/messages; sorry I cannot be of immediate help there. Regarding the rpc issue, I just ran across mention of those in sshfs/nfs threads appearing here and in particular to a link referenced within one of them (http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1362611+0 +archive/2006/freebsd-stable/20060702.freebsd-stable ) - it is more than likely not at all related but I am grasping at straws here trying to solve this. FWIW, I do see the following appearing in the /var/log/messages: ufs_rename: fvp == tvp (can't happen) about once or twice a day, but cannot correlate those to lockup. Now that I have enabled the options mentioned above in the kernel, I am seeing some LOR issues: kernel: lock order reversal: kernel: 1st 0xffffff00c3bab200 kqueue (kqueue) @ /usr/src/sys/kern/kern_event.c:1547 kernel: 2nd 0xffffff0005bb6078 struct mount mtx (struct mount mtx) @ /usr/src/sys/ufs/ufs/ufs_vnops.c:138