Date: Mon, 19 Oct 2015 15:34:47 +0000 From: "Eggert, Lars" <lars@netapp.com> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: "jfv@FreeBSD.org" <jfv@freebsd.org>, "ricera10@gmail.com" <ricera10@gmail.com>, Giuseppe Lettieri <g.lettieri@iet.unipi.it>, "net@freebsd.org" <net@freebsd.org> Subject: Re: ixl 40G bad performance? Message-ID: <D845A1C3-77AE-401F-BDD7-AEC10EE6F0F3@netapp.com> In-Reply-To: <CA%2BhQ2%2BiGk5Bx5mZ8v7K9cPdiNcJgBz99kmP8=twFnETw-bi2vw@mail.gmail.com> References: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> <CA%2BhQ2%2Bj5PkmRWe-ydahExfvOj%2BzestFEn%2B5DZ9_acG2Z_v=8_w@mail.gmail.com> <C81B629B-67D5-4E5B-A215-452E837BE673@netapp.com> <CA%2BhQ2%2BiGk5Bx5mZ8v7K9cPdiNcJgBz99kmP8=twFnETw-bi2vw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_3EA9F340-C819-48AB-BA2A-F8B6B26FAA02
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
Hi,
in order to eliminate network or hardware weirdness, I've rerun the test =
with Linux 4.3rc6, where I get 13.1 Gbits/sec throughput and 52 usec =
flood ping latency. Not great either, but in line with earlier =
experiments with Mellanox NICs and an untuned Linux system.
On 2015-10-19, at 17:11, Luigi Rizzo <rizzo@iet.unipi.it> wrote:
> I suspect it might not touch the c states, but better check. The =
safest is
> disable them in the bios.
I'll try that.
>> hw.ixl.dynamic_tx_itr: 0
>> hw.ixl.dynamic_rx_itr: 0
>>=20
>>=20
> There must be some other control for the actual (fixed, not dynamic)
> moderation.
The only other sysctls in ixl(4) that look relevant are:
hw.ixl.rx_itr
The RX interrupt rate value, set to 8K by default.
hw.ixl.tx_itr
The TX interrupt rate value, set to 4K by default.
I'll play with those.
>> Also no change with "hw.ixl.ringsz=3D256" in loader.conf.
>=20
> Any better success with 2048 slots?
> 3.5 gbit is what I used to see on the ixgbe with tso disabled, =
probably
> hitting a CPU bound.
Will try.
Thanks!
Lars
--Apple-Mail=_3EA9F340-C819-48AB-BA2A-F8B6B26FAA02
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="signature.asc"
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Message signed with OpenPGP using GPGMail
-----BEGIN PGP SIGNATURE-----
iQCVAwUBViUNm9ZcnpRveo1xAQihLQP/TL9IlhVgdX5D4soii576IC/ZSqq4xYw3
h2iT2QjQBNks6US3p2pv8ZT+5SkmtbN0c8GPFd3mmJ/PM2aov/JSb2wp+Xmz+zTD
MOqjhGXKs84g5dc1Cb83dlkm2EmrrGxotpIvYp+ioui13BNqxDWFYVvIXyPN/wQP
Dx+sdHklhHo=
=74hg
-----END PGP SIGNATURE-----
--Apple-Mail=_3EA9F340-C819-48AB-BA2A-F8B6B26FAA02--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D845A1C3-77AE-401F-BDD7-AEC10EE6F0F3>