From owner-freebsd-current@FreeBSD.ORG Sat Jun 6 11:52:01 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60AAD1065670 for ; Sat, 6 Jun 2009 11:52:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.terabit.net.ua (mail.terabit.net.ua [195.137.202.147]) by mx1.freebsd.org (Postfix) with ESMTP id 05E658FC19 for ; Sat, 6 Jun 2009 11:52:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from skuns.zoral.com.ua ([91.193.166.194] helo=mail.zoral.com.ua) by mail.terabit.net.ua with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63 (FreeBSD)) (envelope-from ) id 1MCuRK-000JbO-1i; Sat, 06 Jun 2009 14:51:58 +0300 Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id n56BptNG039578 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 6 Jun 2009 14:51:55 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id n56BptXF063251; Sat, 6 Jun 2009 14:51:55 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id n56Bpsui063250; Sat, 6 Jun 2009 14:51:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 6 Jun 2009 14:51:54 +0300 From: Kostik Belousov To: Dag-Erling Sm??rgrav Message-ID: <20090606115154.GI1927@deviant.kiev.zoral.com.ua> References: <86skief8q6.fsf@ds4.des.no> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ICd5mx19QJ1U62fT" Content-Disposition: inline In-Reply-To: <86skief8q6.fsf@ds4.des.no> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.1 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua X-Virus-Scanned: mail.terabit.net.ua 1MCuRK-000JbO-1i dc7330e68de121614c8c4e866b43936f X-Terabit: YES Cc: current@freebsd.org Subject: Re: LOR between NFS and syncer X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 11:52:01 -0000 --ICd5mx19QJ1U62fT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 06, 2009 at 03:32:01AM +0200, Dag-Erling Sm??rgrav wrote: > Anyone seen this one before? >=20 > NFS+ZFS server, diskless NFS client. Both run head. On the client: >=20 > lock order reversal: > 1st 0xc301c164 syncer (syncer) @ /usr/src/sys/kern/vfs_subr.c:1693 > 2nd 0xc322c058 nfs (nfs) @ /usr/src/sys/kern/vfs_subr.c:2083 > KDB: stack backtrace: > db_trace_self_wrapper(c07477c6,c2bbfa18,c05921c5,c058310b,c074a615,...) a= t db_trace_self_wrapper+0x26 > kdb_backtrace(c058310b,c074a615,c2cb0828,c2cb0688,c2bbfa74,...) at kdb_ba= cktrace+0x29 > _witness_debugger(c074a615,c322c058,c0759e62,c2cb0688,c0750f5e,...) at _w= itness_debugger+0x25 > witness_checkorder(c322c058,9,c0750f5e,823,0,...) at witness_checkorder+0= x839 > __lockmgr_args(c322c058,80500,c322c074,0,0,...) at __lockmgr_args+0x797 > vop_stdlock(c2bbfb88,c2bbfb70,c0591f6b,c0750f5e,c073ced7,...) at vop_stdl= ock+0x62 > VOP_LOCK1_APV(c079d720,c2bbfb88,c0750f5e,c07b5100,c322c000,...) at VOP_LO= CK1_APV+0xf3 > _vn_lock(c322c000,80500,c0750f5e,823,df,...) at _vn_lock+0x5e > vget(c322c000,80500,c2f1f480,c6e,c2f35c94,...) at vget+0xb9 > vfs_msync(c2f35c94,2,c0750f5e,d67,c2f35c94,...) at vfs_msync+0xe7 > sync_fsync(c2bbfc7c,c301c10c,80400,c0750f5e,69d,...) at sync_fsync+0x17b > VOP_FSYNC_APV(c07975c0,c2bbfc7c,c0750f5e,69d,c2f1f480,...) at VOP_FSYNC_A= PV+0xda > sync_vnode(c093d8f0,c093d8dc,3e8,6cc,4e20,...) at sync_vnode+0x168 > sched_sync(0,c2bbfd38,c073fcec,334,c2ceea90,...) at sched_sync+0x273 > fork_exit(c05d8570,0,c2bbfd38) at fork_exit+0xb8 > fork_trampoline() at fork_trampoline+0x8 > --- trap 0, eip =3D 0, esp =3D 0xc2bbfd70, ebp =3D 0 --- We lock the covered vnode when doing mount or unmount for the non-root filesystem. Since witness works by recording order by lock name, we get the order of mounted on -> mounted from vnode type. Thus, any interleave in the mount type would give us a LOR. The vnode list for the mount point also includes syncer vnode, that has a special fsync vop. Usual order is syncer vnode lock -> vnode lock for the real vnodes. But, when filesystem is unmounted, we get the order covered vnode -> any filesystem vnode during sync before unmount. This one gives the LOR you reported, assuming you unmounted nfs share mounted on the top of nfs share. I think both issues cannot result in a deadlock. --ICd5mx19QJ1U62fT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkoqWFoACgkQC3+MBN1Mb4hdjwCg2Mxfb84KWnmM2IBKoSv2hJBm ZV8An1kEadbLUXxSBAiO+pgAbCLHDzDy =K9Yo -----END PGP SIGNATURE----- --ICd5mx19QJ1U62fT--