From owner-freebsd-current@FreeBSD.ORG Tue Mar 4 10:04:29 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EAD010656C2; Tue, 4 Mar 2008 10:04:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from relay02.kiev.sovam.com (relay02.kiev.sovam.com [62.64.120.197]) by mx1.freebsd.org (Postfix) with ESMTP id 4337E8FC25; Tue, 4 Mar 2008 10:04:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from [212.82.216.226] (helo=skuns.kiev.zoral.com.ua) by relay02.kiev.sovam.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1JWU0S-000Pb8-LC; Tue, 04 Mar 2008 12:04:26 +0200 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by skuns.kiev.zoral.com.ua (8.14.2/8.14.2) with ESMTP id m24A4S5x008402 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Mar 2008 12:04:28 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.2/8.14.2) with ESMTP id m24A4DvD008034; Tue, 4 Mar 2008 12:04:13 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.2/8.14.2/Submit) id m24A4Ca0008032; Tue, 4 Mar 2008 12:04:12 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 4 Mar 2008 12:04:12 +0200 From: Kostik Belousov To: pluknet Message-ID: <20080304100412.GZ57756@deviant.kiev.zoral.com.ua> References: <20080303133406.GV57756@deviant.kiev.zoral.com.ua> <20080303164353.GW57756@deviant.kiev.zoral.com.ua> <20080303192327.GY57756@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WSLj5HkG8jvwtSQS" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: ClamAV version 0.91.2, clamav-milter version 0.91.2 on skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.6 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_SBL autolearn=no version=3.2.4 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on skuns.kiev.zoral.com.ua X-Scanner-Signature: c94a7b229bbd1ed8a35d1eff8ed0aa98 X-DrWeb-checked: yes X-SpamTest-Envelope-From: kostikbel@gmail.com X-SpamTest-Group-ID: 00000000 X-SpamTest-Info: Profiles 2349 [Mar 3 2008] X-SpamTest-Info: helo_type=3 X-SpamTest-Info: {TO: seems autogenerated} X-SpamTest-Info: {TO: local part of email appears in body} X-SpamTest-Info: {received from trusted relay: not dialup} X-SpamTest-Method: none X-SpamTest-Method: Local Lists X-SpamTest-Rate: 19 X-SpamTest-Status: Not detected X-SpamTest-Status-Extended: not_detected X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0255], KAS30/Release Cc: FreeBSD Current Subject: Re: panic: mutex Giant owned at nfs_syscalls.c:556 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2008 10:04:29 -0000 --WSLj5HkG8jvwtSQS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 03, 2008 at 11:16:23PM +0300, pluknet wrote: > On 03/03/2008, Kostik Belousov wrote: > > On Mon, Mar 03, 2008 at 09:27:15PM +0300, pluknet wrote: > > > On 03/03/2008, Kostik Belousov wrote: > > > [snip] > > > > To summarize, I need both the tcpdump and kernel/witness messages= from > > > > the panic. > > > > > > > > > > I'm sorry. Here it is. > > > http://pluknet.nm.ru/dev/tcpdump-nfsserver-full.raw > > > > > > The messages (same as unread msgbuf in initial posting, hand-scribed= ): > > > panic: mutex Giant owned at > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_syscalls.c:556 > > > KDB: enter: panic > > > [thread pid 601 tid 100055 ] > > > Stopped at kdb_enter+0x3a: movl $0,kdb_why > > > db> show locks > > > exclusive sleep mutex nfsd_mtx r =3D 0 (0xc2e0af40) locked @ > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_syscalls.c:501 > > > exclusive sleep mutex Giant r =3D 0 (0xc07e6410) locked @ > > > /usr/src/sys/kern/vfs_lookup.c:663 > > > > > > > Nevertheless, the patch below might help with the panic during > > > > the unlinking (not tested). > > > > > > > > diff --git a/sys/nfsserver/nfs_serv.c b/sys/nfsserver/nfs_serv.c > > > > index 446651d..87e1aaa 100644 > > > > --- a/sys/nfsserver/nfs_serv.c > > > > +++ b/sys/nfsserver/nfs_serv.c > > > > @@ -2146,7 +2146,7 @@ nfsrv_remove(struct nfsrv_descript *nfsd, s= truct nfssvc_sock *slp, > > > > nfsfh_t nfh; > > > > fhandle_t *fhp; > > > > struct mount *mp =3D NULL; > > > > - int vfslocked; > > > > + int vfslocked, vfslocked1; > > > > > > > > nfsdbprintf(("%s %d\n", __FILE__, __LINE__)); > > > > ndclear(&nd); > > > > @@ -2168,7 +2168,11 @@ nfsrv_remove(struct nfsrv_descript *nfsd, = struct nfssvc_sock *slp, > > > > nd.ni_cnd.cn_flags =3D LOCKPARENT | LOCKLEAF | MPSAFE; > > > > error =3D nfs_namei(&nd, fhp, len, slp, nam, &md, &dpos, > > > > &dirp, v3, &dirfor, &dirfor_ret, td, FALSE); > > > > - vfslocked =3D NDHASGIANT(&nd); > > > > + vfslocked1 =3D NDHASGIANT(&nd); > > > > + if (vfslocked && vfslocked1) > > > > + VFS_UNLOCK_GIANT(vfslocked1); > > > > + if (vfslocked || vfslocked1) > > > > + vfslocked =3D 1; > > > > if (dirp && !v3) { > > > > vrele(dirp); > > > > dirp =3D NULL; > > > > > > > > > > > > > > Now the last lock triplex looks like: > > > vfslocked lock in > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_serv.c, 2161 > > > vfslocked lock in > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_srvsubs.c, 1106 > > > vfslocked lock in > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_srvsubs.c, 673 > > > vfslocked unlock in > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_srvsubs.c, 916 > > > vfslocked1 unlock in > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_serv.c, 2173 > > > ^^^ > > > vfslocked unlock in > > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_serv.c, 2238 > > > > > > And no panic. Thanks. > > > > > > Could you, please, clarify. As I read you mail, the patch fixed at least > > one of your panic. Are there any other situations where nfs server over > > non-MPSAFE fs panics for you ? It is possible that what you reported > > before actually contains several different reasons for Giant leak. >=20 > Of course. > That another situation is while performing /etc/rc.d/nfsd stop > > System call nfssvc returning with the following locks held: > > exclusive sleep mutex Giant r =3D 2 (0xc07e6410) locked @ > > /usr/src/sys/modules/nfsserver/../../nfsserver/nfs_srvsubs.c:1106 > > panic: witness_warn >=20 > I got no panic with this patch: I lost you again. What patch you are referencing there ? Is that the patch I sent you, or some _other_ patch ? > # /etc/rc.d/nfsd stop > Stopping nfsd. > kill: 1737: No such process > kill: 1738: No such process > kill: 1739: No such process > kill: 1740: No such process > # >=20 > wbr, > pluknet --WSLj5HkG8jvwtSQS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iEYEARECAAYFAkfNHpwACgkQC3+MBN1Mb4i1fgCgmUygC0/i2IUt3g4bp+dw4GCv /PYAoLwh15qitiEvTfWXuNgMKKMha9Tw =TzWE -----END PGP SIGNATURE----- --WSLj5HkG8jvwtSQS--