Date: Fri, 24 Nov 2006 15:53:43 +0200 From: Kostik Belousov <kostikbel@gmail.com> To: Steven Hartland <killing@multiplay.co.uk> Cc: freebsd-fs@freebsd.org, Mike Jakubik <mikej@rogers.com>, freebsd-stable@freebsd.org Subject: Re: md deadlocks on wdrain. Was: [Re: quota and snapshotsin6.1-RELEASE] Message-ID: <20061124135343.GC81940@deviant.kiev.zoral.com.ua> In-Reply-To: <076f01c70f4f$e6b96050$b3db87d4@multiplay.co.uk> References: <20061123101246.GM1841@deviant.kiev.zoral.com.ua> <076f01c70f4f$e6b96050$b3db87d4@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
--S1BNGpv0yoYahz37
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Thu, Nov 23, 2006 at 10:36:57PM -0000, Steven Hartland wrote:
> Thanks for that, from our point of view its required
> as without the machine deadlocks without even trying.
>=20
> One real strange thing was that if I created a copy of
> the vnode file, this particular task ( installworld )
> would succeed but given the nature of the bug ( timing
> critical ) that seem to make sence.
>=20
> With this still being a potential issue I think we have
> no choice but to migrate away from using any vnode backed
> jails.
>=20
Ok, I think the following could be the right approach.
Index: kern/vfs_bio.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /usr/local/arch/ncvs/src/sys/kern/vfs_bio.c,v
retrieving revision 1.513
diff -u -r1.513 vfs_bio.c
--- kern/vfs_bio.c 29 Oct 2006 00:04:39 -0000 1.513
+++ kern/vfs_bio.c 24 Nov 2006 11:40:33 -0000
@@ -871,7 +871,8 @@
* or syncer daemon trying to clean up as that can lead
* to deadlock.
*/
- if ((curthread->td_pflags & TDP_NORUNNINGBUF) =3D=3D 0)
+ if ((curthread->td_pflags & TDP_NORUNNINGBUF) =3D=3D 0 &&
+ (bp->b_vp->v_vflag & VV_MD) =3D=3D 0)
waitrunningbufspace();
}
=20
Index: sys/vnode.h
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /usr/local/arch/ncvs/src/sys/sys/vnode.h,v
retrieving revision 1.318
diff -u -r1.318 vnode.h
--- sys/vnode.h 13 Nov 2006 05:51:22 -0000 1.318
+++ sys/vnode.h 24 Nov 2006 11:40:33 -0000
@@ -254,6 +254,7 @@
#define VV_PROCDEP 0x0100 /* vnode is process dependent */
#define VV_NOKNOTE 0x0200 /* don't activate knotes on this vnode */
#define VV_DELETED 0x0400 /* should be removed */
+#define VV_MD 0x0800 /* vnode backs the md device */
=20
/*
* Vnode attributes. A field value of VNOVAL represents a field whose val=
ue
Index: dev/md/md.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /usr/local/arch/ncvs/src/sys/dev/md/md.c,v
retrieving revision 1.166
diff -u -r1.166 md.c
--- dev/md/md.c 1 Nov 2006 18:59:06 -0000 1.166
+++ dev/md/md.c 24 Nov 2006 11:40:33 -0000
@@ -693,6 +693,8 @@
mtx_lock_spin(&sched_lock);
sched_prio(curthread, PRIBIO);
mtx_unlock_spin(&sched_lock);
+ if (sc->type =3D=3D MD_VNODE)
+ curthread->td_pflags |=3D TDP_NORUNNINGBUF;
=20
for (;;) {
mtx_lock(&sc->queue_mtx);
@@ -923,6 +925,7 @@
VFS_UNLOCK_GIANT(vfslocked);
return (error ? error : EINVAL);
}
+ nd.ni_vp->v_vflag |=3D VV_MD;
VOP_UNLOCK(nd.ni_vp, 0, td);
=20
if (mdio->md_fwsectors !=3D 0)
@@ -936,6 +939,9 @@
=20
error =3D mdsetcred(sc, td->td_ucred);
if (error !=3D 0) {
+ vn_lock(nd.ni_vp, LK_EXCLUSIVE | LK_RETRY, td);
+ nd.ni_vp->v_vflag &=3D ~VV_MD;
+ VOP_UNLOCK(nd.ni_vp, 0, td);
(void)vn_close(nd.ni_vp, flags, td->td_ucred, td);
VFS_UNLOCK_GIANT(vfslocked);
return (error);
@@ -966,6 +972,9 @@
mtx_destroy(&sc->queue_mtx);
if (sc->vnode !=3D NULL) {
vfslocked =3D VFS_LOCK_GIANT(sc->vnode->v_mount);
+ vn_lock(sc->vnode, LK_EXCLUSIVE | LK_RETRY, td);
+ sc->vnode->v_vflag &=3D ~VV_MD;
+ VOP_UNLOCK(sc->vnode, 0, td);
(void)vn_close(sc->vnode, sc->flags & MD_READONLY ?
FREAD : (FREAD|FWRITE), sc->cred, td);
VFS_UNLOCK_GIANT(vfslocked);
--S1BNGpv0yoYahz37
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)
iD8DBQFFZvlnC3+MBN1Mb4gRAtoKAKDOuEfly5wPj1YjUdCtvBY/RuSnbwCgw2zB
Jn9zULhJdVk5tbPXoWlxjT4=
=b5zA
-----END PGP SIGNATURE-----
--S1BNGpv0yoYahz37--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061124135343.GC81940>