Date: Mon, 5 Jun 2017 16:06:51 +0200 From: Vincenzo Maffione <v.maffione@gmail.com> To: Harry Schmalzbauer <freebsd@omnilan.de> Cc: freebsd-net <freebsd-net@freebsd.org> Subject: Re: ovs-netmap forgotten? Message-ID: <CA%2B_eA9i4wwjXYNata6eJKBAMeAeQ2YMheeev6_juStf42EhBTQ@mail.gmail.com> In-Reply-To: <592FC60A.1030308@omnilan.de> References: <5926FFDB.7040900@omnilan.de> <592F20A0.4020702@omnilan.de> <CA%2B_eA9i5ZJuz34WmnHfovtgX5o8a=yzpe18L8Qeoai%2B7wf4k6Q@mail.gmail.com> <592FC60A.1030308@omnilan.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Harry, I've done some investigation on this issue (just for fun) , and I think I may have found the issue. When using vlan interfaces, netmap use the emulated adapter, as the "vlan" driver is not netmap-enabled (and it cannot be). To intercept RX packets, netmap replaces the "if_input" function pointer field in the kernel "struct ifnet" (the struct representing a network interface). Note that you have an instance of "struct ifnet" for em0 (physical NIC), and a different instance for each VLAN cloned interface (e.g. "vlan100") on em0. If you put vlan100 in netmap mode, netmap will replace the if_input of vlan100, and not the if_input of em0. So far, this is an expected behaviour= . Unfortunately, I see in the code here https://github.com/freebsd/freebsd/blob/master/sys/net/if_vlan.c#L1244-L124= 5 that when VLAN driver intercepts the RX packet coming from the underlying interface (e.g. em0 in our example), the em0 if_input is used rather than the vlan100 if_input. In terms of code, we have (*ifp->if_input)(ifv->ifv_ifp, m); rather than (*ifv->ifv_ifp->if_input)(ifv->ifv_ifp, m); Since em0 if_input is not replaced, netmap does not intercept it and you don't see it in your application, e.g. # pkt-gen -i vlan100 -f rx will see nothing. Now, I think that normally ifv->ifv_ifp->if_input =3D=3D ifp->if_input, so = this may explain why the code is written like that (to avoid the additional pointer dereferencing). This is not the case for netmap, where ifv->ifv_ifp->if_input !=3D ifp->if_input when em0 xor vlan100 are in netmap mode. You may try to recompile the kernel with that change and see if you can see packets coming on vlan100 with pkt-gen. I recommend you always doing tests with pkt-gen before trying to use vale-ctl -a. Cheers, Vincenzo 2017-06-01 9:45 GMT+02:00 Harry Schmalzbauer <freebsd@omnilan.de>: > Bez=C3=BCglich Vincenzo Maffione's Nachricht vom 01.06.2017 00:39 (localt= ime): > > Hi Harry, > > OVS integration with netmap is very patchy and Linux only. Most > > importantly, it is not the right way to go, for a number of reasons. > > The real solution would be to integrate netmap into OVS would be to > > follow the DPDK-OVS approach: this means implementing the switching > > logic completely in userspace, in this case using the netmap API. This > > has not been implemented nor sketched so far. > > > > `vale-ctl -n valeXXX:YYY` just creates a persistent VALE port (YYY) > > attached to the VALE switch XXX. > > There is no difference with an ephemeral VALE port, apart for the fact > > that the persistent one is visible with ifconfig. > > > > It does not really make sense to attach a VLAN interface to VALE, since > > the VLAN driver does not have netmap support, so you lose all the > > advantage of using netmap and VALE. > > In your case the best solution I see is to write a custom netmap > > application that forwards the packets between a netmap-supported NIC an= d > > one or more VMs, doing the VLAN stripping in software. > > Thanks again, Vincenzo, for your highly appreciated support! > > I can only concur to your proposed solution. Problem is, I don't speak > any programmin language well (besides sh maybe) and have abosuletely no > budget/time to do any work out on my C skills (which I'd love to do) ;-) > > So ovs-netmap wasn't the right direction, but the difference between > em0+if_brdige+vmnet|virtio-net+vtnet and vale:em0|vale:guest+vtnet is > noticable. I haven't done any measuring, but just performing typical > admin jobs via cli (ssh into the bhyve-guest, whith resorces via NFS > (v4)) behave completely different =E2=80=93 human-noticable much more sna= ppy in > the vale:guest case! > I don't think this enormous efficiency advantge is soleley caused by > em0-netmap/ring connection; More important, in the > vale:em0|vale:guest+vtnet case, I gain excellent inter-vm efficiency > (and much higher attainable performance of course, which is not crucial > at the moment; but efficiency is!). > Now replacing vale:em0 by vale:vlan0 will surely destroy one big > efficiency advantage, but I still benefit from excellent inter-vm > efficiency and most likely some small efficiency advantage left over the > if_bridge picture. > Also, ptnet is a very interisting area of optimization which is easy to > explore with the vale:vlan scenario. > > In another post I described that the vale:vlan path doesn't work, while > vale:em0 (the parent) with everything else untouched does work. > Dou you think it's possible to fix the vale:vlan coupling without netmap > experts setting up a test environment? > > Thanks, > > -harry > > --=20 Vincenzo Maffione
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B_eA9i4wwjXYNata6eJKBAMeAeQ2YMheeev6_juStf42EhBTQ>