Date: Sat, 10 Oct 2015 11:04:27 -0700 From: Sean Bruno <sbruno@freebsd.org> To: freebsd-net@freebsd.org Subject: Re: netmap: recommended NIC for 40GbE capture on Linux? Message-ID: <5619532B.4020304@freebsd.org> In-Reply-To: <CAOm-9aps449E9uZshtMmwNRD=azt1-Ciz0TNiuG4LHaG4UCW2Q@mail.gmail.com> References: <CAOm-9aps449E9uZshtMmwNRD=azt1-Ciz0TNiuG4LHaG4UCW2Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 10/10/15 07:03, Bruce Merry wrote: > Hi > > We're aim to do some high-speed data capture and processing on > data being sent from an FPGA over Ethernet. The data is about > 3.3Mpps with 512 bytes of application-level data per packet (plus > somewhere around 64 bytes of app-level headers, plus > UDP+IP+Ethernet headers). I'm looking for some advice on what > hardware to use. > > We currently have Mellanox ConnectX-3 NICs with QSFP+. My > experiments with pkt-gen on a single flow (generated by pkt-gen on > another machine) are topping out at receiving 2.4 Mpps, with > ksortirqd pushing one core at 100%. This is with netmap trunk, > which does not build a custom mlx4 driver. I've also tried > https://github.com/caldweba/netmap, which does, but which is > broken (receives only 1023 pps, confirmed broken by the repo > owner). > > Before we spend money, I'd love to hear someone report success > with capturing a single flow at >4Mpps, >20Gbps using netmap on > Linux and what NIC they use. Ideally this would include transparent > mode working so that ARP, ICMP etc can be handled by the host > stack. > > Alternatively, if anyone is managing this on ConnectX-3, I'd love > to know how (I have already disabled flow control and adaptive > interrupt mitigation on the receiver). > > Thanks Bruce > I'm unsure where to ask netmap+linux question to be honest. So, my responses won't be very useful to you. If you feel like doing even *more* science, FreeBSD does have the same capabilities and supports the card that you're using. It might be more work for you to do, but you might find FreeBSD-current (11.0 test) refreshing enough to warrant more investigation. ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/amd64/ISO-IMAGES/11.0/ sean -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJWGVMoXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5kyooH/1bIoaytmkUR73ysuRoBTKcY UHSc7bvngiMLK15cAKRn9UzXoQHjZRQBMXb9mr1omPwdkzsmfx3JAdOIuNhKi0li nYYth8/Cs/GUyKxjaYBs/oLydKoZahgiaeNaEqftQmxkOINOXMI6xLxr4p1kMy6k AbF/LkaSgoyBH/Yu96EK7dQBNCroJuI6BKI7EZdLL93seecsTLx78+qEfXJJI7FC w5vSD5amFMDKMiAewMzhiHzY+uO5rcYzxGbCineF+6aaKT4jbxJ/X6OoEvw8Zzto tkSmDiwxFG14YAsn0RQ4BXF3NDFQOLcH0YzTe+mSNwkrv5rSmAdUaTADuiBd+2o= =Sz+N -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5619532B.4020304>