From owner-freebsd-geom@FreeBSD.ORG Tue Aug 11 13:22:59 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F4891065673 for ; Tue, 11 Aug 2009 13:22:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello087206049004.chello.pl [87.206.49.4]) by mx1.freebsd.org (Postfix) with ESMTP id B3F8A8FC4B for ; Tue, 11 Aug 2009 13:22:58 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 1D27845CDD; Tue, 11 Aug 2009 15:22:56 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D0BA545C8A; Tue, 11 Aug 2009 15:22:50 +0200 (CEST) Date: Tue, 11 Aug 2009 15:22:50 +0200 From: Pawel Jakub Dawidek To: Fabio Checconi Message-ID: <20090811132250.GM2013@garage.freebsd.pl> References: <20090811033050.GS18696@gandalf.sssup.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kERJ49nCKmnv470N" Content-Disposition: inline In-Reply-To: <20090811033050.GS18696@gandalf.sssup.it> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 8.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-geom@freebsd.org Subject: Re: On the transparent insertion of geoms, again X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Aug 2009 13:23:00 -0000 --kERJ49nCKmnv470N Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 11, 2009 at 05:30:50AM +0200, Fabio Checconi wrote: > Hi all, > some time ago there was a discussion [1, 2] on this list on the > transparent insertion of geoms in an open geom path. There was some > kind of agreement on a possible way of doing that, this is the diagram > used at the time: >=20 > > BEFORE ---> [ pp --> old_gp ...] > >=20 > > Then we can do either "geom xx create ad0" which results in > >=20 > > AFTER create ---> [ newpp --> gp --> cp ] ---> [ pp --> old_gp ..= . ] > >=20 > > or "geom xx insert ad0", which results in > >=20 > > AFTER insert ---> [ pp --> gp --> cp ] ---> [ newpp --> old_gp ..= . ] > >=20 >=20 > [ see the original threads for more details and a draft of the code ] >=20 > =20 > The solution relied on the fact that bios keep a reference to their > way back up into the geom path, so it is possible to change on-the-fly > the contents of the providers they go through without affecting the > pending bios. >=20 > Unfortunately a little problem hides behind this assumption; considering > e.g., g_disk_done(): >=20 > % if ((bp->bio_cmd & (BIO_READ|BIO_WRITE|BIO_DELETE)) && > % (dp =3D bp2->bio_to->geom->softc)) { > % devstat_end_transaction_bio(dp->d_devstat, bp); > % } >=20 > if bp2->bio_to->geom is dereferenced after the ``geom insert'' above, > pp->geom points to the new geom, thus the softc of the new class is used > as a struct disk pointer (with all the consequent breakage). >=20 > To fix that we have fallen back to waiting for the completion of all > the pending bios in the ``geom insert'' path, which involves sleeping > (with a timeout) with topology held, from the event thread. The reasons > behind this (admittedly ugly) design are: >=20 > - waiting unconditionally may lead to stall, if the transparent > insertion is done on top of a geom which serves its bios from the > event thread (like geom_slice hotspot users may do), so we need a > timeout to limit this effect; >=20 > - new requests for the old provider/geom couple may arrive, we need > to store them in a temporary queue; >=20 > - we need to sleep inside topology, releasing it to allow progress > to the event thread would mean rechecking *everything* after we > reacquire it to verify if there are no more pending bios; we tried > this and the complexity seemed to be excessive. >=20 > So the question is: does anyone see a better/simpler way of doing the > same hot-insertion without the spurious failures this method may introduc= e? >=20 > Any suggestion is welcome, thank you in advance. I also found another problem. Some classes (mine, for example) reference provider in their private metadata (sc_provider field in g_mirror_softc structure), once this provider is moved to another geom this reference will become invalid. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kERJ49nCKmnv470N Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFKgXCpForvXbEpPzQRAi9gAJ9Xbaz+d14UMsPU7C7y88ci3FwfkACgpBYd 2UeORG5iSeJ5+111csbvSQU= =UPKZ -----END PGP SIGNATURE----- --kERJ49nCKmnv470N--