Date: Thu, 19 Mar 2009 10:45:05 +0100 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: fabio@gandalf.sssup.it, geom@freebsd.org, arch@freebsd.org, phk@freebsd.org, ivan@freebsd.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) Message-ID: <20090319094505.GA1539@garage.freebsd.pl> In-Reply-To: <20090319081936.GA32750@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it>
next in thread | previous in thread | raw e-mail | index | archive | help
--Dxnq1zWXvFF0Q93v
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Thu, Mar 19, 2009 at 09:19:36AM +0100, Luigi Rizzo wrote:
> Hi,
>=20
> Fabio Checconi and I have been thinking on how to implement "proxy"
> geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer
> port, do not do any data transformation, and can be transparently
> inserted or removed on top of a provider port while the tree is
> actively moving data.
>=20
> Our immediate need was to add/remove a scheduler while a disk is
> mounted, but there are possibly other uses e.g. if one wants to
> "sniff" the traffic through a disk, or do other ops that are
> transparent for the data stream.
>=20
> We would like opinion on the following solution, which seems
> extremely simple in terms of implementation.
>=20
> The idea is to intercept requests coming on a provider port, pp, and
> redirect them to a geom node acting as a proxy if the port
> is configured in this way:
>=20
> +=3D=3D=3D=3D=3D...=3D=3D=3D...=3D=3D=3D=3D=3D=3D+
> H H
> H H
> H H
> +=3D=3D=3D=3D=3D...=3D=3D=3D=3D=3D=3D cp =3D=3D+
> | +---------------+
> V | V
> +=3D=3D=3D=3D=3D.....=3D=3D=3D=3D pp =3D=3D+ | +=3D=3D=3D=3D=
=3D=3D=3D proxy_pp =3D=3D+
> H 'ad0s1' H | H H
> H ------->--+ H H
> H gp -------<--+ H proxy_node H
> H H | H H
> +=3D=3D=3D=3D=3D=3D=3D....=3D=3D=3D...=3D=3D=3D+ | +=3D=3D=3D=
=3D=3D=3D=3D proxy_cp =3D=3D+
> | V
> +---------------+
>=20
> Normally the proxy does not exist, and the geom tree works as it does now.
>=20
> When we create a 'proxy' node, with something like
>=20
> geom my_proxy_class proxy ad0s1
>=20
> we do something very similar to a 'create', but:
>=20
> - the proxy node is marked specially in gp->flags, so the core will
> not generate a g_new_provider_event when the provider port is created
> (this means there is no taste() call and nobody should be able
> to attach to the port).
>=20
> - the provider port we attach to is linked, with two pointers,
> to the provider and consumer ports of the proxy_node.
>=20
> In this situation, g_io_request() finds that port pp has a proxy attached
> to it, and immediately redirects the requests to the proxy, which
> does everything a geom node does (cloning requests, etc).
> When the proxy wants to pass the request down, it sends it again to pp,
> but now there is no redirection because the source can be identified
> as the proxy. The pointers in the bio insure a correct flow of the
> requests on the reverse path.
The one advantage I see for this over using regular GEOM rules is that
new consumers go through proxy automatically. When I was working on
similar functionality I more wanted to do something like this:
consumer1 consumer2
\ /
\ /
provider
Insert the proxy in the middle of any provider-consumer pair:
consumer1 consumer2
| |
proxy_provider |
| /
proxy_consumer /
\ /
provider
This can be done (almost I think) atomically:
/* First attach to the destination provider. */
g_attach(proxy_consumer, provider);
/* Then switch original consumer to use proxy_provider (should be almost at=
omic). */
consumer1->provider =3D proxy_provider;
/* handle access counts */
In-flight I/O requests know how to go back, because they have source and
destination stored in bio_from and bio_to fields, so no races here.
> Disconnecting a proxy is almost trivial: apart from handling possible
> races on the data path, we just need to clear pp->proxy_pp and pp->proxy_=
cp.
> After that, we can just send the regular destroy events to the proxy
> node, who will have to take care of flushing any pending bio's (e.g.
> see our geom_sched node that already does this).
>=20
> Overall the change is very small (see attached patch):
> a couple of lines in g_io_request, two extra fields in the g_provider,
> and the addition of a flag to gp->flags to control the generation
> of g_new_provider_event.
> There is basically no overhead on regular operation, and only
> a couple of extra pointers in struct g_provider (we use a spare
> bit in gp->flags to mark G_GEOM_PROXY nodes).
>=20
> The only things missing in the patch should be:
>=20
> - a check to avoid races on creation&destruction of a proxy.
> I am not so sure on how to achieve this, but creation and destruction
> are rare and can normally wait, so we could just piggyback the
> small critical section (manipulating pp->proxy_cp and pp->proxy_cp)
> into some other piece of code that is guaranteed to be race-free.
>=20
> - a check to prevent attaching to a provider port of a proxy
> (not a problem, i believe);
>=20
> - a check to prevent attaching a proxy to a provider port that already
> has one. Of course you can attach a proxy to another proxy, and
> if you want to change the order it is as simple as removing the
> existing proxy and reattaching it after the new one.
Could you provide link for the patch, as it was removed from your e-mail?
--=20
Pawel Jakub Dawidek http://www.wheel.pl
pjd@FreeBSD.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
--Dxnq1zWXvFF0Q93v
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQFJwhQhForvXbEpPzQRAqtkAJ9zME7wMe4RZMsBYdAURm/9voIr/QCgwAim
NK2iPFV6J/jPGqpLNhH3Cl8=
=Fzxm
-----END PGP SIGNATURE-----
--Dxnq1zWXvFF0Q93v--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090319094505.GA1539>