Date: Tue, 11 Aug 2009 05:30:50 +0200 From: Fabio Checconi <fabio@freebsd.org> To: freebsd-geom@freebsd.org Subject: On the transparent insertion of geoms, again Message-ID: <20090811033050.GS18696@gandalf.sssup.it>
next in thread | raw e-mail | index | archive | help
Hi all, some time ago there was a discussion [1, 2] on this list on the transparent insertion of geoms in an open geom path. There was some kind of agreement on a possible way of doing that, this is the diagram used at the time: > BEFORE ---> [ pp --> old_gp ...] > > Then we can do either "geom xx create ad0" which results in > > AFTER create ---> [ newpp --> gp --> cp ] ---> [ pp --> old_gp ... ] > > or "geom xx insert ad0", which results in > > AFTER insert ---> [ pp --> gp --> cp ] ---> [ newpp --> old_gp ... ] > [ see the original threads for more details and a draft of the code ] The solution relied on the fact that bios keep a reference to their way back up into the geom path, so it is possible to change on-the-fly the contents of the providers they go through without affecting the pending bios. Unfortunately a little problem hides behind this assumption; considering e.g., g_disk_done(): % if ((bp->bio_cmd & (BIO_READ|BIO_WRITE|BIO_DELETE)) && % (dp = bp2->bio_to->geom->softc)) { % devstat_end_transaction_bio(dp->d_devstat, bp); % } if bp2->bio_to->geom is dereferenced after the ``geom insert'' above, pp->geom points to the new geom, thus the softc of the new class is used as a struct disk pointer (with all the consequent breakage). To fix that we have fallen back to waiting for the completion of all the pending bios in the ``geom insert'' path, which involves sleeping (with a timeout) with topology held, from the event thread. The reasons behind this (admittedly ugly) design are: - waiting unconditionally may lead to stall, if the transparent insertion is done on top of a geom which serves its bios from the event thread (like geom_slice hotspot users may do), so we need a timeout to limit this effect; - new requests for the old provider/geom couple may arrive, we need to store them in a temporary queue; - we need to sleep inside topology, releasing it to allow progress to the event thread would mean rechecking *everything* after we reacquire it to verify if there are no more pending bios; we tried this and the complexity seemed to be excessive. So the question is: does anyone see a better/simpler way of doing the same hot-insertion without the spurious failures this method may introduce? Any suggestion is welcome, thank you in advance. [1] http://lists.freebsd.org/pipermail/freebsd-geom/2009-March/003400.html [2] http://lists.freebsd.org/pipermail/freebsd-geom/2009-March/003407.html
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090811033050.GS18696>