From owner-freebsd-current@freebsd.org Wed Jan 11 20:13:25 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B07BCAB9C5; Wed, 11 Jan 2017 20:13:25 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3A9A21863; Wed, 11 Jan 2017 20:13:24 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1484165600128352.09294656141446; Wed, 11 Jan 2017 12:13:20 -0800 (PST) Date: Wed, 11 Jan 2017 12:13:20 -0800 From: Matthew Macy To: "Sean Bruno" Cc: =?UTF-8?Q?=22Olivier_Cochard-Labb=C3=A9=22?= , "freebsd-net@freebsd.org" , "freebsd-current@freebsd.org" Message-ID: <1598f29d379.ea6360351471.8752933472741761813@nextbsd.org> In-Reply-To: <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org> <1598d97bf2a.c6bcb76838987.6501340920645175463@nextbsd.org> <574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org> <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Priority: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jan 2017 20:13:25 -0000 ---- On Wed, 11 Jan 2017 12:02:06 -0800 Sean Bruno wr= ote ----=20 > =20 > =20 > On 01/11/17 12:47, Olivier Cochard-Labb=C3=A9 wrote:=20 > > On Wed, Jan 11, 2017 at 4:17 PM, Sean Bruno > > wrote:=20 > > =20 > > =20 > > =20 > > Olivier:=20 > > =20 > > Give this a quick try. This isn't the correct way to do this, but= I=20 > > want to see if I'm on the right path:=20 > > =20 > > =20 > > =E2=80=8Bthanks, it fix the problem, I've got back the 4 queues:=E2=80= =8B=20 > > =20 > > =E2=80=8Bigb2: port 0x3= 000-0x301f=20 > > mem 0xdfea0000-0xdfebffff,0xdff24000-0xdff27fff irq 18 at device 20.0 = on=20 > > pci0=20 > > igb2: attach_pre capping queues at 8=20 > > igb2: using 1024 tx descriptors and 1024 rx descriptors=20 > > igb2: msix_init qsets capped at 8=20 > > igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1=20 > > igb2: using 4 rx queues 4 tx queues=20 > > igb2: Using MSIX interrupts with 5 vectors=20 > > igb2: allocated for 4 tx_queues=20 > > igb2: allocated for 4 rx_queues=20 > > igb2: Ethernet address: 00:08:a2:09:33:da=20 > > igb2: netmap queues/slots: TX 4/1024, RX 4/1024=20 > > =E2=80=8B=20 > > In forwarding mode, I measure about 10% performance drop with this new= =20 > > drivers on this hardware:=20 > > =20 > > x head r311848: packets per second=20 > > + head r311849 and BAR patch: packets per second=20 > > +---------------------------------------------------------------------= -----+=20 > > |++ ++ + xxx = x x|=20 > > | |__M__A__= __| |=20 > > | |___AM__| = |=20 > > +---------------------------------------------------------------------= -----+=20 > > N Min Max Median Avg S= tddev=20 > > x 5 924170 943071 927509 931612.1 8096= .8269=20 > > + 5 831452 845929.5 840940 838730.5 6413= .5602=20 > > Difference at 95.0% confidence=20 > > -92881.6 +/- 10652.2=20 > > -9.96999% +/- 1.07481%=20 > > (Student's t, pooled s =3D 7303.85)=20 > > =20 > > Regards,=20 > > =20 > > Olivier=20 > > =20 > =20 > =20 > Hmmm ... did your old tests do 4 or 8 queues on this hardware?=20 > =20 > Did the old tests run 1024 tx/rx slots or the max 4096?=20 That's a great point, only having one thread per core could easily account = for this. I'm hoping Sean can make txq !=3D rxq work so that you can have 8= txqs and 4 rxqs. -M