From owner-freebsd-fs@FreeBSD.ORG Thu Jul 18 18:52:20 2013 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B9B36FDA; Thu, 18 Jul 2013 18:52:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 13BFFFD1; Thu, 18 Jul 2013 18:52:19 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r6IIqF2c013999; Thu, 18 Jul 2013 21:52:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r6IIqF2c013999 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r6IIqFBP013998; Thu, 18 Jul 2013 21:52:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 18 Jul 2013 21:52:15 +0300 From: Konstantin Belousov To: Andriy Gapon Subject: Re: Deadlock in nullfs/zfs somewhere Message-ID: <20130718185215.GE5991@kib.kiev.ua> References: <51DCFEDA.1090901@FreeBSD.org> <51E59FD9.4020103@FreeBSD.org> <51E67F54.9080800@FreeBSD.org> <51E7B686.4090509@FreeBSD.org> <20130718112814.GA5991@kib.kiev.ua> <51E7F05A.5020609@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6PtN/jU//tuarfdA" Content-Disposition: inline In-Reply-To: <51E7F05A.5020609@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-fs@FreeBSD.org, Adrian Chadd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2013 18:52:20 -0000 --6PtN/jU//tuarfdA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 18, 2013 at 04:40:42PM +0300, Andriy Gapon wrote: > on 18/07/2013 14:28 Konstantin Belousov said the following: > > Well, I have no opinion. Making the fs suspended, in other words, prev= enting > > writers from entering the filesystem code, is probably good. I do not > > know zfs code to usefully comment on the approach. >=20 > OK, fair. >=20 > > Note that you must drain existing writers, i.e. call vfs_write_suspend(= ), > > to set MNTK_SUSPEND. >=20 > Here is my take on it, not tested at all. >=20 > diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c > b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c > index 0fc59cc..59c8cbd 100644 > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c > @@ -2263,8 +2263,12 @@ zfs_suspend_fs(zfsvfs_t *zfsvfs) > { > int error; >=20 > - if ((error =3D zfsvfs_teardown(zfsvfs, B_FALSE)) !=3D 0) > + if ((error =3D vfs_write_suspend(zfsvfs->z_vfs)) !=3D 0) > return (error); > + if ((error =3D zfsvfs_teardown(zfsvfs, B_FALSE)) !=3D 0) { > + vfs_write_resume(mp, 0); > + return (error); > + } > dmu_objset_disown(zfsvfs->z_os, zfsvfs); >=20 > return (0); > @@ -2339,5 +2343,6 @@ bail: > rrw_exit(&zfsvfs->z_teardown_lock, FTAG); >=20 > + vfs_write_resume(mp, 0); > if (err) { > /* > * Since we couldn't reopen zfsvfs::z_os, or There is VFS method VFS_SUSP_CLEAN, called when the suspension is lifted. UFS uses it to clean the back-queue of work which were not performed during the suspend, mostly inactivate the postponed inactive vnodes. ZFS probably does not need it, since it does not check for MNTK_SUSPEND, but if it starts care, there is a place to put the code. On the other hand, I believe that your patch is notoriously incomplete, because there should be a lot of threads which mutate ZFS mounts state and which do not call vn_start_write() around the mutations. I.e. all ZFS top-level code which calls into ZFS ops and which is not coming from VFS. --6PtN/jU//tuarfdA Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJR6DlfAAoJEJDCuSvBvK1BMcAP/jw+dYeUw6RjJZCmJ7I6XrHa aMYHMVjPuUH3h5jyE3avPZXd1OU67GkKmu+pQMF9qaMzPie6IkNK7FnK8Ey2Wy9D 1chQXv1ccaAvJTDk1VZbEWtctZ38V5CPm34ZbwGWk0wImjMU03C6D+fZRc9VzozM hyxQAyDc89Nmmcxn6BnL4INJAAIASVB3QcRY2lO7/FKjAR/yxmy6R74/HvvRNDhk QYvvSFJzNnB9wQByupYY69hbz18VQI3hyTMzh3xZkt9JcH2oCHanl8WkQUIxB4eJ ZCBsKFZdV+rKLnXHfP4tlXmmXLTzFVJlf5u+vS1l4PhLWsV2IZceImFDRLaxq1OC v2pF0DM2txWRODrsGY5Ie/DUwm7DoUghpu27rirkTcx84w3BWoD4/F5iEM1J+ZWY MXyC7Gj4560v0lE4mDyw0ZKDVfOsmWH5dx4ElwFCk6Wvxk4/+Wg3qtnZlTsKMZkM tJkw8lCj7xzg/FCg5ukXRnfuPixB3eiAbyEaQF1qR391YYypwfwSTfg3E2lYnSs0 BoijDzDaCj+8xlvl6CPY+/YKY6j1rTXFXcQQ5ayviOHVJjWo3kQAHukyWnZw/glx fZR8rZkv13iAnXSxulJW4AZA3O0Ahy42IuWfVdL7g3cRFEzHPvkXkbi4h/l6J/Wf uLiHucS8MyFMDh5GyDg/ =Bddh -----END PGP SIGNATURE----- --6PtN/jU//tuarfdA--