Date: Thu, 19 Mar 2009 12:41:13 +0100 From: =?ISO-8859-1?Q?Marius_N=FCnnerich?= <marius@nuenneri.ch> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: fabio@gandalf.sssup.it, geom@freebsd.org, arch@freebsd.org, ivan@freebsd.org Subject: Re: RFC: adding 'proxy' nodes to provider ports (with patch) Message-ID: <b649e5e0903190441m2d511107qd95cb3cd566b11f7@mail.gmail.com> In-Reply-To: <20090319081936.GA32750@onelab2.iet.unipi.it> References: <20090319081936.GA32750@onelab2.iet.unipi.it>
next in thread | previous in thread | raw e-mail | index | archive | help
2009/3/19 Luigi Rizzo <rizzo@iet.unipi.it>: > Hi, > > Fabio Checconi and I have been thinking on how to implement "proxy" > geom nodes, i.e. nodes that have exactly 1 provider and 1 consumer > port, do not do any data transformation, and can be transparently > inserted or removed on top of a provider port while the tree is > actively moving data. > > Our immediate need was to add/remove a scheduler while a disk is > mounted, but there are possibly other uses e.g. if one wants to > "sniff" the traffic through a disk, or do other ops that are > transparent for the data stream. > > We would like opinion on the following solution, which seems > extremely simple in terms of implementation. > > The idea is to intercept requests coming on a provider port, pp, and > redirect them to a geom node acting as a proxy if the port > is configured in this way: > > =A0 =A0 +=3D=3D=3D=3D=3D...=3D=3D=3D...=3D=3D=3D=3D=3D=3D+ > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H > =A0 =A0 +=3D=3D=3D=3D=3D...=3D=3D=3D=3D=3D=3D cp =3D=3D+ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0+-----------= ----+ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 V =A0 =A0 =A0 =A0 =A0| =A0 =A0 = =A0 =A0 =A0 =A0 =A0 V > =A0 =A0 +=3D=3D=3D=3D=3D.....=3D=3D=3D=3D pp =3D=3D+ =A0 =A0 | =A0 =A0+= =3D=3D=3D=3D=3D=3D=3D proxy_pp =3D=3D+ > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 'ad0s1' =A0H =A0 =A0 | =A0 =A0H =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0------->--+ =A0 =A0H =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 H > =A0 =A0 H =A0 =A0 =A0 =A0gp =A0 =A0 =A0-------<--+ =A0 =A0H =A0 =A0proxy_= node =A0 =A0 H > =A0 =A0 H =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0H =A0 =A0 | =A0 =A0H =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 H > =A0 =A0 +=3D=3D=3D=3D=3D=3D=3D....=3D=3D=3D...=3D=3D=3D+ =A0 =A0 | =A0 = =A0+=3D=3D=3D=3D=3D=3D=3D proxy_cp =3D=3D+ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 = =A0 =A0 =A0 =A0 =A0 V > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0+---------= ------+ > > Normally the proxy does not exist, and the geom tree works as it does now= . > > When we create a 'proxy' node, with something like > > =A0 =A0 =A0 =A0geom my_proxy_class proxy ad0s1 > > we do something very similar to a 'create', but: > > - the proxy node is marked specially in gp->flags, so the core will > =A0not generate a g_new_provider_event when the provider port is created > =A0(this means there is no taste() call and nobody should be able > =A0to attach to the port). > > - the provider port we attach to is linked, with two pointers, > =A0to the provider and consumer ports of the proxy_node. > > In this situation, g_io_request() finds that port pp has a proxy attached > to it, and immediately redirects the requests to the proxy, which > does everything a geom node does (cloning requests, etc). > =A0 When the proxy wants to pass the request down, it sends it again to p= p, > but now there is no redirection because the source can be identified > as the proxy. =A0The pointers in the bio insure a correct flow of the > requests on the reverse path. > > Disconnecting a proxy is almost trivial: apart from handling possible > races on the data path, we just need to clear pp->proxy_pp and pp->proxy_= cp. > After that, we can just send the regular destroy events to the proxy > node, who will have to take care of flushing any pending bio's (e.g. > see our geom_sched node that already does this). > > Overall the change is very small (see attached patch): > a couple of lines in g_io_request, two extra fields in the g_provider, > and the addition of a flag to gp->flags to control the generation > of g_new_provider_event. > There is basically no overhead on regular operation, and only > a couple of extra pointers in struct g_provider (we use a spare > bit in gp->flags to mark G_GEOM_PROXY nodes). > > The only things missing in the patch should be: > > - a check to avoid races on creation&destruction of a proxy. > =A0I am not so sure on how to achieve this, but creation and destruction > =A0are rare and can normally wait, so we could just piggyback the > =A0small critical section (manipulating pp->proxy_cp and pp->proxy_cp) > =A0into some other piece of code that is guaranteed to be race-free. > > - a check to prevent attaching to a provider port of a proxy > =A0(not a problem, i believe); > > - a check to prevent attaching a proxy to a provider port that already > =A0has one. Of course you can attach a proxy to another proxy, and > =A0if you want to change the order it is as simple as removing the > =A0existing proxy and reattaching it after the new one. > > Feedback welcome. I wonder if it's really necessary to alter the GEOM infrastructure or if it is possible to do this with what's there already. Just an idea: Lock g_topology, put g_down and g_up to sleep, alter the consumer and provider pointers where you need it so the everything is routed through your proxy class (which isn't special in any way) and restart g_down and g_up.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b649e5e0903190441m2d511107qd95cb3cd566b11f7>