From owner-freebsd-stable@FreeBSD.ORG Sun Jul 27 17:48:06 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A042439E for ; Sun, 27 Jul 2014 17:48:06 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2721721B5 for ; Sun, 27 Jul 2014 17:48:05 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6RHlvvJ009392 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 27 Jul 2014 20:47:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua s6RHlvvJ009392 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6RHlvV1009391; Sun, 27 Jul 2014 20:47:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 27 Jul 2014 20:47:57 +0300 From: Konstantin Belousov To: Harald Schmalzbauer Subject: Re: panic/lock on 9.3-RELEASE with nullfs/nfs/zfs combination Message-ID: <20140727174757.GN93733@kib.kiev.ua> References: <53D12973.3010805@omnilan.de> <20140724165917.GT93733@kib.kiev.ua> <53D1503B.2030200@omnilan.de> <20140724193048.GU93733@kib.kiev.ua> <53D2006C.7090207@omnilan.de> <20140725151448.GY93733@kib.kiev.ua> <53D380B2.2080700@omnilan.de> <20140726114839.GF93733@kib.kiev.ua> <53D5105A.6040204@omnilan.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RSvKN2On5SRY9c4K" Content-Disposition: inline In-Reply-To: <53D5105A.6040204@omnilan.de> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: FreeBSD Stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jul 2014 17:48:06 -0000 --RSvKN2On5SRY9c4K Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 27, 2014 at 04:44:42PM +0200, Harald Schmalzbauer wrote: > Bez??glich Konstantin Belousov's Nachricht vom 26.07.2014 13:48 > (localtime): > I can't emphasize how much I hate my lousy C skills... Got "format '%#x' > expects type 'unsigned int', but argument 3 has type 'u_long' > [-Wformat]" from the compiler and found out that I can't read/understand > any part of your patch; not even the syntax :-( >=20 > After reading printf(3) I change the 3rd additional line into > '("ldvp %p fl %#lx dvp %p fl %#lx flags %#x", ldvp, ldvp->v_vflag,' > ??? A blind man may sometimes hit the mark. This is because the patch was against HEAD, where I changed type of flags to not waste space. It was not merged to stable/9 to keep KBI. > Here's the panic: > panic: ldvp 0xfffffe001bafb000 fl 0x1 dvp 0xfffffe01b4e5a9d8 fl 0 flags > 0x120e144 > vnode vnode 0xfffffe014bc03dc8: 0xfffffe014bc03dc8: tag null, type VDIR > tag null, type VDIR > usecount 1, writecount 0, refcount 1 mountedhere 0 > usecount 1, writecount 0, refcount 1 mountedhere 0 > flags (VV_ROOT|VI_ACTIVE) > flags (VV_ROOT|VI_ACTIVE) > v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0 > v_object 0xfffffe014bbf4828 ref 0 pages 0 cleanbuf 0 dirtybuf 0 > lock type zfs: SHARED (count 1) > lock type zfs: SHARED (count 1) > vp=3D0xfffffe014bc03dc8, lowervp=3D0xfffffe001bafb000 > vp=3D0xfffffe014bc03dc8, lowervp=3D0xfffffe001bafb000 So the root vnode for underlying ZFS filesystem somehow managed to get two different upper vnodes for nullfs mount. My guess is that the real problem is due to inconsistent initialization of v_hash for zfs vnodes. It only get set in zfs_vget(), which causes root vnode to have zero hash initially. Please, keep the KASSERT() patch applied, revert the VV_ROOT patch for nullfs, and apply the following change. Then retest. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c b/= sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c index 04160c6..33748fe 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c @@ -2075,8 +2075,6 @@ zfs_vget(vfs_t *vfsp, ino_t ino, int flags, vnode_t *= *vpp) err =3D vn_lock(*vpp, flags); if (err !=3D 0) *vpp =3D NULL; - else - (*vpp)->v_hash =3D ino; return (err); } =20 diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c b/s= ys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c index ac55996..c8f86b8 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c @@ -1228,9 +1228,10 @@ again: vnode_t *vp =3D ZTOV(zp); =20 err =3D insmntque(vp, zfsvfs->z_vfs); - if (err =3D=3D 0) + if (err =3D=3D 0) { + vp->v_hash =3D obj_num; VOP_UNLOCK(vp, 0); - else { + } else { zp->z_vnode =3D NULL; zfs_znode_dmu_fini(zp); zfs_znode_free(zp); --RSvKN2On5SRY9c4K Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJT1TtMAAoJEJDCuSvBvK1BnwIP+wYJdIaUQ17ygpyiWCIpdFly gbSi91GMuQKa+FXwJ3bzJmwo3TELiWnX4NKNHYtaaso1zU5dW+ADh1dzuqX1E/xc iuQ3EDlmw7QkSCQl2R5EKlanG0Bqh5n0uIvJ/OVI+kkIna4KcW38N/xXGb5A1YjK eABkhB1pTWSBpI6emQMBJXabRiRInIDCTl/8uxGAucBBgHjSYmRaWbuiWjHt9dPh aBsmV6qps/mI9M84HxbEGMSYIDp2jZy9SJtAUVqqx1G1aF1taISRAUEc/pTRE1Mz 3ldVZ6DntSsjG4ZjrL/inED8Mcxa+nnn+TnJqigqRLk9PWuIsW87EyUaBVpRjXhk JSeTDjU4zL0SDTUL+VWHlgC+N8nyiEXK5jz13TCatQi6wcuZHcIyQ7Qlt0viifJ5 Low6wOl3FfoIWKF3mSsjrzSH8BlwJKFrOZlOJV8OQzXFUET/CjeXznorD5ZdGrZI ABgZ9DoJVY8aE5nhtwOR4UqURkYBjY5AhrJsqX5ywdXmbSYWTmLUrvFpO3nEj623 GqUaKOaKZJ6MvzrK0F3ADfACblhDBsPAxJt9oyyNQiTKlYcotKgugrRik6yVHlOO 5YdOOKi9ng5UWfY7PGSSI41aXgBc8J/w5ADVl2CJsi0w/Nuyo0wJd9vnzMVkpNLw BbLI7ThP7hqKQohfl70+ =JwIH -----END PGP SIGNATURE----- --RSvKN2On5SRY9c4K--