Date: Tue, 4 Feb 2020 01:23:21 +0300 From: Slawa Olhovchenkov <slw@zxy.spb.ru> To: Navdeep Parhar <np@FreeBSD.org> Cc: freebsd-net@freebsd.org Subject: Re: Chelsio NETMAP performance Message-ID: <20200203222321.GB8012@zxy.spb.ru> In-Reply-To: <863de9e1-42cc-6f3a-5c1f-1bf737714c9f@FreeBSD.org> References: <20200203201728.GC8028@zxy.spb.ru> <863de9e1-42cc-6f3a-5c1f-1bf737714c9f@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Feb 03, 2020 at 01:39:52PM -0800, Navdeep Parhar wrote: > On 2/3/20 12:17 PM, Slawa Olhovchenkov wrote: > > I am try to use Chelsio T540-CR in netmap mode and see poor (compared > > to Intel 82599ES) performance. > > What approximate FreeBSD version is this? 12.1-STABLE > > > > Same application ac receive only about 8.9Mpss, compared to 12.5Mpps > > at Intel. > > > > pmc profile show mostly time spend in: > > > > 49.76% [17802] service_nm_rxq @ /boot/kernel/if_cxgbe.ko > > 100.0% [17802] t4_vi_intr > > 100.0% [17802] ithread_loop @ /boot/kernel/kernel > > 100.0% [17802] fork_exit > > > > > > to be exact at line > > > > while ((d->rsp.u.type_gen & F_RSPD_GEN) == nm_rxq->iq_gen) { > > > > Is this maximum limit for this vendor? > > No, a T540 should be able to sink full 10Gbps (14.88Mpps) on a single rx > queue. Try adding this to your loader.conf: > > hw.cxgbe.toecaps_allowed="0" > > Then try simple netmap "pkt-gen -f rx" instead of any custom app and see > how many pps it's able to sink. Thanks! `hw.cxgbe.toecaps_allowed="0"` allow recive 14Mpps for may application too! Now I am got only 10% less performance compared to Intel, as I see by higher Chelsio interrupt cpu time (top show about 30% for every interrupt handler). Is this normal? Is this posible to optimize? This is interrupt rate I see: 54651 t5nex0:3b0 56475 t5nex0:3b1 55181 t5nex0:3b2 56577 t5nex0:3b3
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200203222321.GB8012>