From owner-freebsd-fs@FreeBSD.ORG Fri Nov 24 13:53:53 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AB4C816A40F; Fri, 24 Nov 2006 13:53:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from fw.zoral.com.ua (fw.zoral.com.ua [213.186.206.134]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7C45743D55; Fri, 24 Nov 2006 13:53:11 +0000 (GMT) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.4) with ESMTP id kAODrkxX071688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Nov 2006 15:53:46 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8) with ESMTP id kAODrkPS078344; Fri, 24 Nov 2006 15:53:46 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.13.8/8.13.8/Submit) id kAODrhdU078343; Fri, 24 Nov 2006 15:53:43 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 24 Nov 2006 15:53:43 +0200 From: Kostik Belousov To: Steven Hartland Message-ID: <20061124135343.GC81940@deviant.kiev.zoral.com.ua> References: <20061123101246.GM1841@deviant.kiev.zoral.com.ua> <076f01c70f4f$e6b96050$b3db87d4@multiplay.co.uk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="S1BNGpv0yoYahz37" Content-Disposition: inline In-Reply-To: <076f01c70f4f$e6b96050$b3db87d4@multiplay.co.uk> User-Agent: Mutt/1.4.2.2i X-Virus-Scanned: ClamAV version 0.88.4, clamav-milter version 0.88.4 on fw.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=1.4 required=5.0 tests=SPF_NEUTRAL, UNPARSEABLE_RELAY autolearn=no version=3.1.4 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.1.4 (2006-07-25) on fw.zoral.com.ua Cc: freebsd-fs@freebsd.org, Mike Jakubik , freebsd-stable@freebsd.org Subject: Re: md deadlocks on wdrain. Was: [Re: quota and snapshotsin6.1-RELEASE] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 13:53:53 -0000 --S1BNGpv0yoYahz37 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 23, 2006 at 10:36:57PM -0000, Steven Hartland wrote: > Thanks for that, from our point of view its required > as without the machine deadlocks without even trying. >=20 > One real strange thing was that if I created a copy of > the vnode file, this particular task ( installworld ) > would succeed but given the nature of the bug ( timing > critical ) that seem to make sence. >=20 > With this still being a potential issue I think we have > no choice but to migrate away from using any vnode backed > jails. >=20 Ok, I think the following could be the right approach. Index: kern/vfs_bio.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/arch/ncvs/src/sys/kern/vfs_bio.c,v retrieving revision 1.513 diff -u -r1.513 vfs_bio.c --- kern/vfs_bio.c 29 Oct 2006 00:04:39 -0000 1.513 +++ kern/vfs_bio.c 24 Nov 2006 11:40:33 -0000 @@ -871,7 +871,8 @@ * or syncer daemon trying to clean up as that can lead * to deadlock. */ - if ((curthread->td_pflags & TDP_NORUNNINGBUF) =3D=3D 0) + if ((curthread->td_pflags & TDP_NORUNNINGBUF) =3D=3D 0 && + (bp->b_vp->v_vflag & VV_MD) =3D=3D 0) waitrunningbufspace(); } =20 Index: sys/vnode.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/arch/ncvs/src/sys/sys/vnode.h,v retrieving revision 1.318 diff -u -r1.318 vnode.h --- sys/vnode.h 13 Nov 2006 05:51:22 -0000 1.318 +++ sys/vnode.h 24 Nov 2006 11:40:33 -0000 @@ -254,6 +254,7 @@ #define VV_PROCDEP 0x0100 /* vnode is process dependent */ #define VV_NOKNOTE 0x0200 /* don't activate knotes on this vnode */ #define VV_DELETED 0x0400 /* should be removed */ +#define VV_MD 0x0800 /* vnode backs the md device */ =20 /* * Vnode attributes. A field value of VNOVAL represents a field whose val= ue Index: dev/md/md.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /usr/local/arch/ncvs/src/sys/dev/md/md.c,v retrieving revision 1.166 diff -u -r1.166 md.c --- dev/md/md.c 1 Nov 2006 18:59:06 -0000 1.166 +++ dev/md/md.c 24 Nov 2006 11:40:33 -0000 @@ -693,6 +693,8 @@ mtx_lock_spin(&sched_lock); sched_prio(curthread, PRIBIO); mtx_unlock_spin(&sched_lock); + if (sc->type =3D=3D MD_VNODE) + curthread->td_pflags |=3D TDP_NORUNNINGBUF; =20 for (;;) { mtx_lock(&sc->queue_mtx); @@ -923,6 +925,7 @@ VFS_UNLOCK_GIANT(vfslocked); return (error ? error : EINVAL); } + nd.ni_vp->v_vflag |=3D VV_MD; VOP_UNLOCK(nd.ni_vp, 0, td); =20 if (mdio->md_fwsectors !=3D 0) @@ -936,6 +939,9 @@ =20 error =3D mdsetcred(sc, td->td_ucred); if (error !=3D 0) { + vn_lock(nd.ni_vp, LK_EXCLUSIVE | LK_RETRY, td); + nd.ni_vp->v_vflag &=3D ~VV_MD; + VOP_UNLOCK(nd.ni_vp, 0, td); (void)vn_close(nd.ni_vp, flags, td->td_ucred, td); VFS_UNLOCK_GIANT(vfslocked); return (error); @@ -966,6 +972,9 @@ mtx_destroy(&sc->queue_mtx); if (sc->vnode !=3D NULL) { vfslocked =3D VFS_LOCK_GIANT(sc->vnode->v_mount); + vn_lock(sc->vnode, LK_EXCLUSIVE | LK_RETRY, td); + sc->vnode->v_vflag &=3D ~VV_MD; + VOP_UNLOCK(sc->vnode, 0, td); (void)vn_close(sc->vnode, sc->flags & MD_READONLY ? FREAD : (FREAD|FWRITE), sc->cred, td); VFS_UNLOCK_GIANT(vfslocked); --S1BNGpv0yoYahz37 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFZvlnC3+MBN1Mb4gRAtoKAKDOuEfly5wPj1YjUdCtvBY/RuSnbwCgw2zB Jn9zULhJdVk5tbPXoWlxjT4= =b5zA -----END PGP SIGNATURE----- --S1BNGpv0yoYahz37--