From owner-freebsd-stable@FreeBSD.ORG Fri Dec 15 21:22:52 2006 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AEBE016A407 for ; Fri, 15 Dec 2006 21:22:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay02.kiev.sovam.com (relay02.kiev.sovam.com [62.64.120.197]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5DAA043C9F for ; Fri, 15 Dec 2006 21:21:08 +0000 (GMT) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.227] (helo=fw.zoral.com.ua) by relay02.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.60) (envelope-from ) id 1GvKVu-000Mr7-Gw for stable@freebsd.org; Fri, 15 Dec 2006 23:22:51 +0200 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id kBFLKfYA015857 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 15 Dec 2006 23:20:41 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8) with ESMTP id kBFLKfrA091833; Fri, 15 Dec 2006 23:20:41 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8/Submit) id kBFLKeNw091832; Fri, 15 Dec 2006 23:20:40 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Dec 2006 23:20:40 +0200 From: Kostik Belousov To: Kris Kennaway Message-ID: <20061215212040.GG23698@deviant.kiev.zoral.com.ua> References: <20061205.004323.78708386.hrs@allbsd.org> <20061204160949.GM35681@deviant.kiev.zoral.com.ua> <20061205.123805.59655403.hrs@allbsd.org> <1166194879.6317.11.camel@lanshark.dmv.com> <20061215181548.GA58555@xor.obsecurity.org> <1166209936.6317.21.camel@lanshark.dmv.com> <20061215192958.GA86926@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9Iq5ULCa7nGtWwZS" Content-Disposition: inline In-Reply-To: <20061215192958.GA86926@xor.obsecurity.org> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.4, clamav-milter version 0.88.4 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=1.4 required=5.0 tests=SPF_NEUTRAL, UNPARSEABLE_RELAY autolearn=no version=3.1.4 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.1.4 (2006-07-25) on fw.zoral.com.ua X-Scanner-Signature: 5465a971ece288bb6a3bd6c55c651316 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 614 [Dec 14 2006] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 0 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: Sven Willenberger , stable@freebsd.org Subject: Re: Not panic in nfsd (Re: panic in nfsd on 6.2-RC1) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2006 21:22:52 -0000 --9Iq5ULCa7nGtWwZS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 15, 2006 at 02:29:58PM -0500, Kris Kennaway wrote: > On Fri, Dec 15, 2006 at 02:12:16PM -0500, Sven Willenberger wrote: > > On Fri, 2006-12-15 at 13:15 -0500, Kris Kennaway wrote: > > > On Fri, Dec 15, 2006 at 10:01:19AM -0500, Sven Willenberger wrote: > > > > On Tue, 2006-12-05 at 12:38 +0900, Hiroki Sato wrote: > > > > > Kostik Belousov wrote > > > > > in <20061204160949.GM35681@deviant.kiev.zoral.com.ua>: > > > > >=20 > > > > > ko> What version of sys/nfsserver/nfs_serv.c do you use ? If it i= s older than > > > > > ko> 1.156.2.7, please, update the system. > > > > >=20 > > > > > Thanks, I updated it just now and see how it works. > > > > >=20 > > > > > -- > > > > > | Hiroki SATO > > > >=20 > > > > I was/am having the same issue. Updating world (6.2-stable) to incl= ude > > > > the above update sadly did not fix the problem for me. This is an a= md64 > > > > box with only one client connecting to it via nfs. Reading further = it > > > > may seem to be an issue with rpc.statd and/or rpc.lockd. As I only = have > > > > one client connecting and it is being used as mail storage (i.e. the > > > > client pops/imaps the storage) would be safe to not using fcntl for= wards > > > > over the wire? Is this same issue present in 6.1-RELENG? I am reall= y at > > > > my wits end at this point and for the first time am actually consid= ering > > > > moving to another OS (solaris more than likely) as I cannot have th= ese > > > > types of issues interrupting services every couple days. > > > >=20 > > > > What other information (spefically) can I provide to help the devs > > > > figure out what is going on? What can I do in the meantime to have = some > > > > semblence of stability? I assume downgrading to 5.5-RELENG is out o= f the > > > > question but perhaps disabling SMP? > > >=20 > > > Just to confirm, can you please post the panic backtrace you are > > > seeing? And can you explain what you mean by "may seem to be an issue > > > with rpc.statd and/or rpc.lockd"? > > >=20 > > > Sometimes people think they're seeing the same problem as someone else > > > when really it's a completely different problem in the same subsystem, > > > so I'd like to rule that out here. > > >=20 > > > Kris > >=20 > > Well I have now added kdb and invariants/witness support to the kernel > > so I should be able to get some backtrace the next time it happens. > > Currently, the system just locks and no error is displayed on the > > console or /var/log/messages; sorry I cannot be of immediate help there. >=20 > OK, so your issue is not, in fact, a "panic in nfsd" as you were > claiming ;-) >=20 > > Regarding the rpc issue, I just ran across mention of those in sshfs/nfs > > threads appearing here and in particular to a link referenced within one > > of them (http://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D1362611+0 > > +archive/2006/freebsd-stable/20060702.freebsd-stable ) - it is more than > > likely not at all related but I am grasping at straws here trying to > > solve this. >=20 > Yes, I think you are grasping at straws. At this point, you need to > do some debugging to find out the source of your problem, and treat it > as a new bug until you find conclusive evidence that it's the same as > a previously reported bug. >=20 > Guessing without evidence that your problem is the same as someone > else's problem, because e.g. both involve your system becoming > unresponsive, is a very good way to confuse the issue and delay > resolution. > =20 > > FWIW, I do see the following appearing in the /var/log/messages: > > ufs_rename: fvp =3D=3D tvp (can't happen)=20 > > about once or twice a day, but cannot correlate those to lockup. Now > > that I have enabled the options mentioned above in the kernel, I am > > seeing some LOR issues: > >=20 > > kernel: lock order reversal: > > kernel: 1st 0xffffff00c3bab200 kqueue (kqueue) @ /usr/src/sys/kern/kern= _event.c:1547 > > kernel: 2nd 0xffffff0005bb6078 struct mount mtx (struct mount mtx) @ /u= sr/src/sys/ufs/ufs/ufs_vnops.c:138 >=20 > OK, this is interesting, so let's proceed from here. >=20 > Kris Try this. Index: ufs/ufs/ufs_vnops.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v retrieving revision 1.283 diff -u -r1.283 ufs_vnops.c --- ufs/ufs/ufs_vnops.c 6 Nov 2006 13:42:09 -0000 1.283 +++ ufs/ufs/ufs_vnops.c 15 Dec 2006 21:19:51 -0000 @@ -133,19 +133,15 @@ { struct inode *ip; struct timespec ts; - int mnt_locked; =20 ip =3D VTOI(vp); - mnt_locked =3D 0; - if ((vp->v_mount->mnt_flag & MNT_RDONLY) !=3D 0) { - VI_LOCK(vp); + VI_LOCK(vp); + if ((vp->v_mount->mnt_flag & MNT_RDONLY) !=3D 0) goto out; + if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) =3D=3D 0) { + VI_UNLOCK(vp); + return; } - MNT_ILOCK(vp->v_mount); /* For reading of mnt_kern_flags. */ - mnt_locked =3D 1; - VI_LOCK(vp); - if ((ip->i_flag & (IN_ACCESS | IN_CHANGE | IN_UPDATE)) =3D=3D 0) - goto out_unl; =20 if ((vp->v_type =3D=3D VBLK || vp->v_type =3D=3D VCHR) && !DOINGSOFTDEP(v= p)) ip->i_flag |=3D IN_LAZYMOD; @@ -172,10 +168,7 @@ =20 out: ip->i_flag &=3D ~(IN_ACCESS | IN_CHANGE | IN_UPDATE); - out_unl: VI_UNLOCK(vp); - if (mnt_locked) - MNT_IUNLOCK(vp->v_mount); } =20 /* --9Iq5ULCa7nGtWwZS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFgxGoC3+MBN1Mb4gRAlhrAJ4j5LNZBdJ+cw9DFNxjr92AHH8TaACeIPLl migJ5N+a/N0XEU6D6gHcwVE= =7lc7 -----END PGP SIGNATURE----- --9Iq5ULCa7nGtWwZS--