Date: Mon, 22 Sep 2014 15:15:02 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Elof Ofel <elofu17@hotmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: How do I balance bandwidth over several virtual NICs? Message-ID: <CAJ-Vmo=m9_K96BGr7GYMbEJuqb0x8NyUirKgRMZU71dCRBAGgA@mail.gmail.com> In-Reply-To: <DUB125-W51623651A926D99844D879BCB30@phx.gbl> References: <DUB125-W13FDC584F5DF9881CF5FDEBCB30@phx.gbl> <CA%2BP_MZGA_uz_H_QsB%2BdgXEgbXNCjv7w-OToKby=ww%2BvKgnU4_Q@mail.gmail.com> <DUB125-W851F972702452D9809C8E5BCB30@phx.gbl> <CAJ-Vmo=NGGkOkPWQKZ=3gA3vYYyM2kcjd3m85ymdJY3q4ixxLw@mail.gmail.com> <DUB125-W51623651A926D99844D879BCB30@phx.gbl>
next in thread | previous in thread | raw e-mail | index | archive | help
On 22 September 2014 13:39, Elof Ofel <elofu17@hotmail.com> wrote: > Hi Adrian! > > Now this sounds promising! All my sensors use the ixgbe driver. > However, my skills in programming/compiling isn't vast. I know how to patch > and use poudriere. That's about it. > > I must admit I don't really understand what you mean with "patch it to use a > symmetric RSS key", but it sounds like the functionality I'm looking for is > not yet there in the driver. A few years ago a couple of researchers figured out you could abuse the toeplitz hash to do symmetric RSS hashing: http://www.ndsl.kaist.edu/~shinae/papers/TR-symRSS.pdf This means that the same RSS hash value is chosen no matter which direction the traffic is going on. This is what you need to ensure that the packets going both directions in a connection end up in the same NIC hardware receive ring. So, all you have to do (!) is: * grab freebsd-head, because the ixgbe driver there has some recent bug fixes that you need for this to completely work; * look at ixgbe_initialise_rss_mapping() - that's where the RSS key, mapping and RSS hash types are configured; * patch it to use the example symmetric RSS key that was provided in the paper; * patch it to only hash on IPv4 / IPv6 2-tuple, that way you don't end up with IPv4/IPv6 fragments in the wrong queue; * configure up say, 4 or 8 rings in /boot/loader.conf: hw.ix.num_queues=8 (I think it's hw.ix, it used to be hw.ixgbe..) * then, when you use netmap on ixgbe, you just bind to each TX and RX ring with a separate process or thread. That thread will get packets in both directions for a given flow. > If we assume that someone in the future write and submit the above into the > ixgbe driver, could I be so bold as to ask you for a > commandline/configuration example (a brief guide) of how one would setup > netmap and how to configure it to use the RX-queues? I don't know of any examples of using netmap in this way from the command line. I've normally written C code (And when I do, i start with the bridge example in src/tools/tools/netmap/bridge.c) . > That way I can start playing around with netmap and learning it while I wait > for the ixgbe driver to be updated... I've got two professional programmer > colleagues who've dealt extensively with e.g. the libnids and pfring source > code, so if I get a grasp of how to setup netmap, and I find it interesting, > it is likely that they can dive into and fix the ixgbe driver and improve it > as per above. So please, can you help me with a "netmap guide"? > > When I try to find documentation or examples of how to setup netmap I find > none. Not even the netmap-enabled pcaplib contain any information as how to > use it. I'm no programmer, so showing me different C structs for delivering > data is of no use. :-/ You mean: https://code.google.com/p/netmap-libpcap/ ? I've not used it before, sorry :( -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=m9_K96BGr7GYMbEJuqb0x8NyUirKgRMZU71dCRBAGgA>