Date: Tue, 11 Aug 2009 15:22:50 +0200 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: Fabio Checconi <fabio@freebsd.org> Cc: freebsd-geom@freebsd.org Subject: Re: On the transparent insertion of geoms, again Message-ID: <20090811132250.GM2013@garage.freebsd.pl> In-Reply-To: <20090811033050.GS18696@gandalf.sssup.it> References: <20090811033050.GS18696@gandalf.sssup.it>
next in thread | previous in thread | raw e-mail | index | archive | help
--kERJ49nCKmnv470N
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Aug 11, 2009 at 05:30:50AM +0200, Fabio Checconi wrote:
> Hi all,
> some time ago there was a discussion [1, 2] on this list on the
> transparent insertion of geoms in an open geom path. There was some
> kind of agreement on a possible way of doing that, this is the diagram
> used at the time:
>=20
> > BEFORE ---> [ pp --> old_gp ...]
> >=20
> > Then we can do either "geom xx create ad0" which results in
> >=20
> > AFTER create ---> [ newpp --> gp --> cp ] ---> [ pp --> old_gp ..=
. ]
> >=20
> > or "geom xx insert ad0", which results in
> >=20
> > AFTER insert ---> [ pp --> gp --> cp ] ---> [ newpp --> old_gp ..=
. ]
> >=20
>=20
> [ see the original threads for more details and a draft of the code ]
>=20
> =20
> The solution relied on the fact that bios keep a reference to their
> way back up into the geom path, so it is possible to change on-the-fly
> the contents of the providers they go through without affecting the
> pending bios.
>=20
> Unfortunately a little problem hides behind this assumption; considering
> e.g., g_disk_done():
>=20
> % if ((bp->bio_cmd & (BIO_READ|BIO_WRITE|BIO_DELETE)) &&
> % (dp =3D bp2->bio_to->geom->softc)) {
> % devstat_end_transaction_bio(dp->d_devstat, bp);
> % }
>=20
> if bp2->bio_to->geom is dereferenced after the ``geom insert'' above,
> pp->geom points to the new geom, thus the softc of the new class is used
> as a struct disk pointer (with all the consequent breakage).
>=20
> To fix that we have fallen back to waiting for the completion of all
> the pending bios in the ``geom insert'' path, which involves sleeping
> (with a timeout) with topology held, from the event thread. The reasons
> behind this (admittedly ugly) design are:
>=20
> - waiting unconditionally may lead to stall, if the transparent
> insertion is done on top of a geom which serves its bios from the
> event thread (like geom_slice hotspot users may do), so we need a
> timeout to limit this effect;
>=20
> - new requests for the old provider/geom couple may arrive, we need
> to store them in a temporary queue;
>=20
> - we need to sleep inside topology, releasing it to allow progress
> to the event thread would mean rechecking *everything* after we
> reacquire it to verify if there are no more pending bios; we tried
> this and the complexity seemed to be excessive.
>=20
> So the question is: does anyone see a better/simpler way of doing the
> same hot-insertion without the spurious failures this method may introduc=
e?
>=20
> Any suggestion is welcome, thank you in advance.
I also found another problem. Some classes (mine, for example) reference
provider in their private metadata (sc_provider field in g_mirror_softc
structure), once this provider is moved to another geom this reference
will become invalid.
--=20
Pawel Jakub Dawidek http://www.wheel.pl
pjd@FreeBSD.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
--kERJ49nCKmnv470N
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQFKgXCpForvXbEpPzQRAi9gAJ9Xbaz+d14UMsPU7C7y88ci3FwfkACgpBYd
2UeORG5iSeJ5+111csbvSQU=
=UPKZ
-----END PGP SIGNATURE-----
--kERJ49nCKmnv470N--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090811132250.GM2013>