Date: Fri, 20 Apr 2018 16:45:23 -0700 From: Kevin Bowling <kevin.bowling@kev009.com> To: Garrett Wollman <wollman@bimajority.org> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Diagnosing terrible ixl performance Message-ID: <CAK7dMtDpw-9Xi39w_DAzAfJcrZz5H4Lxxiesow1b2BPR1P6PBw@mail.gmail.com> In-Reply-To: <23257.26265.720293.659892@hergotha.csail.mit.edu> References: <23257.26265.720293.659892@hergotha.csail.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
These are all pretty much known (to intel and NF/LLNW) issues with the ixl driver. If you must run 11.1 you are best off just buying a chelsio T580. If you can run HEAD, or perhaps an eventual 11.3 MFC, https://github.com/intel-wired-ethernet/freebsd/tree/ixl-iflib may fix this but there are a few remaining issues before that hits HEAD. Regards, On Thu, Apr 19, 2018 at 9:03 PM, Garrett Wollman <wollman@bimajority.org> wrote: > I'm commissioning a new NFS server with an Intel dual-40G XL710 > interface, running 11.1. I have a few other servers with this > adapter, although not running 40G, and they work fine so long as you > disable TSO. This one ... not so much. On the receive side, it gets > about 600 Mbit/s with lots of retransmits. On the *sending* side, > though, it's not even able to sustain 10 Mbit/s -- but there's no > evidence of retransmissions, it's just sending really really slowly. > (Other machines with XL710 adapters are able to sustain full 10G.) > There is no evidence of any errors on either the adapter or the switch > it's connected to. > > So far, I've tried: > > - Using the latest Intel driver (no change) > - Using the latest Intel firmware (breaks the adapter) > - Disabling performance tweaks in loader.conf and sysctl.conf > - Changing congestion-control algorithms > > Anyone have suggestions while I still have time to test this? (My > plan B is to fall back to an X520 card that I have in my spares kit, > because I *know* those work great with no faffing about.) Any > relevant MIBs to inspect? > > The test I'm doing here is simple iperf over TCP, with MTU 9120. It > takes about 10 seconds for the sending side to complete, but buffers > are severely constipated for 20 seconds after that (delaying all > traffic, including ssh connections). > > I'm at the point of trying different switch ports just to eliminate > that as a possibility. > > -GAWollman > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAK7dMtDpw-9Xi39w_DAzAfJcrZz5H4Lxxiesow1b2BPR1P6PBw>