From owner-freebsd-fs@FreeBSD.ORG Mon Jun 13 20:15:53 2011 Return-Path: <owner-freebsd-fs@FreeBSD.ORG> Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 771FD106566C; Mon, 13 Jun 2011 20:15:53 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 22D608FC08; Mon, 13 Jun 2011 20:15:52 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id CE8F245C98; Mon, 13 Jun 2011 22:15:50 +0200 (CEST) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B9E9045683; Mon, 13 Jun 2011 22:15:45 +0200 (CEST) Date: Mon, 13 Jun 2011 22:15:43 +0200 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: "Justin T. Gibbs" <gibbs@FreeBSD.org> Message-ID: <20110613201543.GA1733@garage.freebsd.pl> References: <4DF25544.3020301@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KsGdsel6WgEHnImy" Content-Disposition: inline In-Reply-To: <4DF25544.3020301@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: fs@FreeBSD.org Subject: Re: Drop of spa_namespace lock in vdev_geom.c X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems <freebsd-fs.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs> List-Post: <mailto:freebsd-fs@freebsd.org> List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 13 Jun 2011 20:15:53 -0000 --KsGdsel6WgEHnImy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 10, 2011 at 11:32:52AM -0600, Justin T. Gibbs wrote: > Dropping and reacquiring the spa_namespace lock in vdev_geom_open() > creates a lock order reversal with the spa_config locks. As the > spa_config locks are not standard mutexes, witness will not warn > about this issue. I only noticed this problem when debugging a ZFS > deadlock. The deadlock can be triggered anytime that there are > multiple insert/remove processes going on (e.g. vdev orphan processing > while a fault management daemon is onlining a replacement device for > some other vdev). >=20 > I haven't noticed any issues with just holding the namespace lock > for the duration of the open. Does anyone know why this lock drop > was added in v28? I did that as part of @182208 to fix another LOR. Full commit log: Change 182208 on 2010/08/10 by pjd@pjd_zoo OpenSolaris switched to lazy creation of /dev/ entires for ZVOLs. It creates /dev/ entries on VOP_LOOKUP() or VOP_READDIR(). This of course can't work this way on FreeBSD with GEOM, so we need to create ZVOL providers where appropriate. I found the following cases: 1. Pool first open (pool is loaded based on zpool/cache configuration and is then opened for a first time on eg. zfs mount). 2. Pool import. It's not the same as 1. 3. ZVOL creation: zfs create -V<size> <zvol>. 4. Creation of ZVOL snapshot, this includes recursive snapshot creation. To make it work I had to fix LOR between the zfsdev_state_lock, the GEOM topology lock and the spa_namespace_lock. They are now always obtained in the following order: 1. zfsdev_state_lock 2. g_topology_lock 3. spa_namespace_lock Also, we can't use taskqueue to scan for VDEVs as this introduces deadlock (because there is no way to honour the order above). This also allows to simplify vdev_geom.c quite a bit as it is no longer a problem to taste ZVOL or ZVOL-based provider. Update /etc/rc.d/zvol as there are no longer volinit and volfini subcommands to zfs(8). --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --KsGdsel6WgEHnImy Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk32b+8ACgkQForvXbEpPzTTMQCeOpNr4VS569h9QhAbnCGgVqh/ cI8AoOS/q1Y0dNsRP2hBO2KYWdtwnWUU =SQsf -----END PGP SIGNATURE----- --KsGdsel6WgEHnImy--