Date: Sun, 27 Jul 2014 21:29:04 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Harald Schmalzbauer <h.schmalzbauer@omnilan.de> Cc: FreeBSD Stable <freebsd-stable@freebsd.org> Subject: Re: panic/lock on 9.3-RELEASE with nullfs/nfs/zfs combination Message-ID: <20140727182904.GO93733@kib.kiev.ua> In-Reply-To: <53D543CD.5070607@omnilan.de> References: <20140724165917.GT93733@kib.kiev.ua> <53D1503B.2030200@omnilan.de> <20140724193048.GU93733@kib.kiev.ua> <53D2006C.7090207@omnilan.de> <20140725151448.GY93733@kib.kiev.ua> <53D380B2.2080700@omnilan.de> <20140726114839.GF93733@kib.kiev.ua> <53D5105A.6040204@omnilan.de> <20140727174757.GN93733@kib.kiev.ua> <53D543CD.5070607@omnilan.de>
next in thread | previous in thread | raw e-mail | index | archive | help
--swBRKU+0+VRhVQHB
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sun, Jul 27, 2014 at 08:24:13PM +0200, Harald Schmalzbauer wrote:
> Bez?glich Konstantin Belousov's Nachricht vom 27.07.2014 19:47
> (localtime):
> > On Sun, Jul 27, 2014 at 04:44:42PM +0200, Harald Schmalzbauer wrote:
> >> Bez??glich Konstantin Belousov's Nachricht vom 26.07.2014 13:48
> >> (localtime):
> >> I can't emphasize how much I hate my lousy C skills... Got "format '%#=
x'
> >> expects type 'unsigned int', but argument 3 has type 'u_long'
> >> [-Wformat]" from the compiler and found out that I can't read/understa=
nd
> >> any part of your patch; not even the syntax :-(
> >>
> >> After reading printf(3) I change the 3rd additional line into
> >> '("ldvp %p fl %#lx dvp %p fl %#lx flags %#x", ldvp, ldvp->v_vflag,'
> >> ??? A blind man may sometimes hit the mark.
> > This is because the patch was against HEAD, where I changed type of fla=
gs
> > to not waste space. It was not merged to stable/9 to keep KBI.
> >
> >> Here's the panic:
> >> panic: ldvp 0xfffffe001bafb000 fl 0x1 dvp 0xfffffe01b4e5a9d8 fl 0 flags
> >> 0x120e144
> >> vnode vnode 0xfffffe014bc03dc8: 0xfffffe014bc03dc8: tag null, type VDIR
> >> tag null, type VDIR
> >> usecount 1, writecount 0, refcount 1 mountedhere 0
> >> usecount 1, writecount 0, refcount 1 mountedhere 0
> >> flags (VV_ROOT|VI_ACTIVE)
> >> flags (VV_ROOT|VI_ACTIVE)
> >> v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
> >> v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0
> >> lock type zfs: SHARED (count 1)
> >> lock type zfs: SHARED (count 1)
> >> vp=3D0xfffffe014bc03dc8, lowervp=3D0xfffffe001bafb000
> >> vp=3D0xfffffe014bc03dc8, lowervp=3D0xfffffe001bafb000
> > So the root vnode for underlying ZFS filesystem somehow managed to get
> > two different upper vnodes for nullfs mount. My guess is that the real
> > problem is due to inconsistent initialization of v_hash for zfs vnodes.
> > It only get set in zfs_vget(), which causes root vnode to have zero
> > hash initially.
> >
> > Please, keep the KASSERT() patch applied, revert the VV_ROOT patch for
> > nullfs, and apply the following change. Then retest.
> >
> > diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.=
c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c
> > index 04160c6..33748fe 100644
> > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c
> > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c
> > @@ -2075,8 +2075,6 @@ zfs_vget(vfs_t *vfsp, ino_t ino, int flags, vnode=
_t **vpp)
> > err =3D vn_lock(*vpp, flags);
> > if (err !=3D 0)
> > *vpp =3D NULL;
> > - else
> > - (*vpp)->v_hash =3D ino;
> > return (err);
> > }
> > =20
> > diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c=
b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c
> > index ac55996..c8f86b8 100644
> > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c
> > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c
> > @@ -1228,9 +1228,10 @@ again:
> > vnode_t *vp =3D ZTOV(zp);
> > =20
> > err =3D insmntque(vp, zfsvfs->z_vfs);
> > - if (err =3D=3D 0)
> > + if (err =3D=3D 0) {
> > + vp->v_hash =3D obj_num;
> > VOP_UNLOCK(vp, 0);
> > - else {
> > + } else {
> > zp->z_vnode =3D NULL;
> > zfs_znode_dmu_fini(zp);
> > zfs_znode_free(zp);
>=20
> Applyed these two patches along with the KASSERT patch (reverted
> ROOT_VV) and did the same simple test: No panic, no debug message. So
> this also fixes my limited and simple test case.
Great, I believe that zfs patch fixes the root cause of the problem.
Thank you for the testing.
>=20
> Tell me if I can provide anything else, I'll keep that latest patched
> version running, tomorrow will be regular usage and I'll report again.
>=20
> Thanks,
>=20
> -Harry
>=20
--swBRKU+0+VRhVQHB
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBAgAGBQJT1UTvAAoJEJDCuSvBvK1Bw/kP/3fFSdNYRmwzN2+aYpTTM7M0
J/gXRv08LY67mzvIJfxcoeTu0JyJoOlU0369H/DVspR1bvjGiD7mXvCT/Y1x4RnG
ZUs8TbQxBt6O/WcqDRh1G/cnVj5v19PaUKRGY9/zciVcAJWk5T8LaKtz6coNYZtY
drxuJtNI34t2QTfUS+B0/cIC16gNvRv32sJMJyTkvpFUboeEnyW8moFw16XUhabl
aCSOAyFAMrTe3D5+bFCqo0HWNTd23eJ0ZwvLPBCadQkCWtVNBBrFBXVfRpSwMjyV
WbwuK5YeQOfY4E5wliVkFfwub2PNifzQQ+M7hNM1n7SVnaFmYNrEy/PCzjjcRl3w
gh2MwIZ/VjUI/5CBECTLPVVRrYn4+UTbtNHq8rSSNqC2uhSFNKXiVawezsZdkw2t
qizDTnEWOeTIEZ9SVo7XhAJf077ixBSrRx9pxIsBVLhdMYuW9k3husiU12R+8NsI
yOPZ7IYbtXRYPgzl35Ou2Q2lAvxsVtiWwOOwg2KDr8fTetT8c7n0jXY/oiKYfbtZ
Ob1i90nJNMoBWfGjyHpY0/jaKqAyDDzIVYoWa862azavLa4UpRb6JJhNSh3tG7Pp
D5qdLPG4Q5fJILmk6wB8dAEtMahX9vHbdln42f6hvkNvi1TJUz/kng6WXUsiU3Sr
AdmiUafgvISbDQyCchYB
=MQD3
-----END PGP SIGNATURE-----
--swBRKU+0+VRhVQHB--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140727182904.GO93733>