Date: Sat, 10 Oct 2015 11:35:27 -0300 From: Jim Thompson <jim@netgate.com> To: Bruce Merry <bmerry@ska.ac.za> Cc: freebsd-net@freebsd.org Subject: Re: netmap: recommended NIC for 40GbE capture on Linux? Message-ID: <C17C0451-76DE-417D-8447-3196605C26D7@netgate.com> In-Reply-To: <CAOm-9aps449E9uZshtMmwNRD=azt1-Ciz0TNiuG4LHaG4UCW2Q@mail.gmail.com> References: <CAOm-9aps449E9uZshtMmwNRD=azt1-Ciz0TNiuG4LHaG4UCW2Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> Before we spend money, I'd love to hear someone report success with = capturing a single flow at >4Mpps, >20Gbps using netmap on Linux and > what NIC they use. You said linux, and this is freebsd-net, but this blog post (from = yesterday) is probably apt. https://blog.cloudflare.com/single-rx-queue-kernel-bypass-with-netmap/ > Ideally this would include transparent mode working so that ARP, ICMP = etc can be handled by the host stack. netmap-fwd (written by loos@ and announced yesterday at BSDcon Brazil) = does this (re-injecting frames designed for =E2=80=99this=E2=80=99 host = on the host ring, though there is a mode to handle ARP and ICMP in the fastpath code). netmap-fwd is an implementation of the ip_fastforward code from FreeBSD = on top of netmap. It currently does around 5Mpps, single-stream, on a = single core using Chelsio or Intel NICs. It=E2=80=99s IPv4-only right now, but once we = add IPv6(*), it will be open sourced. PPS @ 64-byte frames (pkt-gen) through DUT Device under test Network interface = Kernel forwarding Fastforward enabled Tryforward patch netmap-fwd = application C2358 (2 core, 1.7 Ghz) Intel I354 = 123 kbps 217 kbps 213 kbps = 945 kpps C2758 (8 core, 2.4 GHz) Chelsio T520 10G 270 apps = 426 apps 431 apps 1.683 = Mpps Xeon-D 1540 (8 core, 2 GHz) Intel X552 10G 439 kbps = 557 kbps 544 kbps 2.230 = Mpps Xeon E3-1275 (4 core, 3.5 GHz) Intel X520-2 1.058 = Mpps 1.331 Mpps 1.297 Mpps 5.053 = Mpps =E2=80=98Tryforward' above is the code in = https://reviews.freebsd.org/D3737 (We haven=E2=80=99t tried it on linux.) I find it likely that an application that acts as a simple filter like = https://github.com/jibi/nm-single-rx-queue/blob/master/main.c or = https://github.com/erikarn/netmap-tools/tree/master/src/bridge=20 will greatly exceed the 5Mpps figure above. See also: = http://adrianchadd.blogspot.com.br/2015/04/intel-ddio-llc-cache-buffer-ali= gnment.html Jim (*) this work will need to wait until after we can get pfSense 2.3 = released=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C17C0451-76DE-417D-8447-3196605C26D7>