Date: Mon, 19 Oct 2015 08:55:53 -0700 From: Luigi Rizzo <rizzo@iet.unipi.it> To: "Eggert, Lars" <lars@netapp.com> Cc: "jfv@FreeBSD.org" <jfv@freebsd.org>, "ricera10@gmail.com" <ricera10@gmail.com>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, "net@freebsd.org" <net@freebsd.org> Subject: Re: ixl 40G bad performance? Message-ID: <CA%2BhQ2%2Biz8LQcOT75xqK3qfo%2BBbq0eAFTdUanH_fjtdK3MLDHZQ@mail.gmail.com> In-Reply-To: <D845A1C3-77AE-401F-BDD7-AEC10EE6F0F3@netapp.com> References: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> <CA%2BhQ2%2Bj5PkmRWe-ydahExfvOj%2BzestFEn%2B5DZ9_acG2Z_v=8_w@mail.gmail.com> <C81B629B-67D5-4E5B-A215-452E837BE673@netapp.com> <CA%2BhQ2%2BiGk5Bx5mZ8v7K9cPdiNcJgBz99kmP8=twFnETw-bi2vw@mail.gmail.com> <D845A1C3-77AE-401F-BDD7-AEC10EE6F0F3@netapp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Oct 19, 2015 at 8:34 AM, Eggert, Lars <lars@netapp.com> wrote: > Hi, > > in order to eliminate network or hardware weirdness, I've rerun the test = with Linux 4.3rc6, where I get 13.1 Gbits/sec throughput and 52 usec flood = ping latency. Not great either, but in line with earlier experiments with M= ellanox NICs and an untuned Linux system. > ... >> There must be some other control for the actual (fixed, not dynamic) >> moderation. > > The only other sysctls in ixl(4) that look relevant are: > > hw.ixl.rx_itr > The RX interrupt rate value, set to 8K by default. > > hw.ixl.tx_itr > The TX interrupt rate value, set to 4K by default. > yes those. raise to 20-50k and see what you get in terms of ping latency. Note that 4k on tx means you get to reclaim buffers in the tx queue (unless it is done opportunistically) every 250us which is dangerously close to the 300us capacity of the queue itself. cheers luigi > I'll play with those. > >>> Also no change with "hw.ixl.ringsz=3D256" in loader.conf. >> >> Any better success with 2048 slots? >> 3.5 gbit is what I used to see on the ixgbe with tso disabled, probably >> hitting a CPU bound. > > Will try. > > Thanks! > > Lars --=20 -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BhQ2%2Biz8LQcOT75xqK3qfo%2BBbq0eAFTdUanH_fjtdK3MLDHZQ>