From owner-freebsd-stable@FreeBSD.ORG Wed Aug 21 15:36:53 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BD959429; Wed, 21 Aug 2013 15:36:53 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from s1.omnilan.de (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 56B8924BC; Wed, 21 Aug 2013 15:36:53 +0000 (UTC) Received: from titan.inop.wdn.omnilan.net (titan.inop.wdn.omnilan.net [172.21.3.1]) (authenticated bits=0) by s1.omnilan.de (8.13.8/8.13.8) with ESMTP id r7LEnfUV097103 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 21 Aug 2013 16:49:41 +0200 (CEST) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <5214D37F.5000307@omnilan.de> Date: Wed, 21 Aug 2013 16:49:35 +0200 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Bryan Venteicher Subject: Re: [CFT] VMware vmxnet3 ethernet driver References: <601099152.721.1375661537866.JavaMail.root@daemoninthecloset.org> In-Reply-To: <601099152.721.1375661537866.JavaMail.root@daemoninthecloset.org> X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig68C5E2ACB8ED11F3249CE469" Cc: FreeBSD Stable , current@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 15:36:53 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig68C5E2ACB8ED11F3249CE469 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Bez=FCglich Bryan Venteicher's Nachricht vom 05.08.2013 02:12 (localtime= ): > Hi, > > I've ported the OpenBSD vmxnet3 ethernet driver to FreeBSD. I did a > lot of cleanup, bug fixes, new features, etc (+2000 new lines) along > the way so there is not much of a resemblance left. > > The driver is in good enough shape I'd like additional testers. A patch= > against -CURRENT is at [1]. Alternatively, the driver and a Makefile is= > at [2]; this should compile at least as far back as 9.1. I can look at > 8-STABLE if there is interest. > > Obviously, besides reports of 'it works', I'm interested performance vs= > the emulated e1000, and (for those using it) the VMware tools vmxnet3 > driver. Hopefully it is no worse :) Hello Bryan, thanks a lot for your hard work! It seems if_vmx doesn't support jumbo frames. If I set mtu 9000, I get =BBvmx0: cannot populate Rx queue 0=AB, I have no problems using jumbo frames with vmxnet3. I took a oldish host (4x2,8GHz Core2[LGA775]) with recent software: ESXi 5.1U1 and FreeBSD-9.2-RC2 Two guests are connected to one MTU9000 "VMware Software Switch". Simple iperf (standard TCP) results: vmxnet3jumbo <-> vmxnet3jumbo 5.3Gbits/sec, load: 40-60%Sys 0.5-2%Intr vmxnet3 <-> vmxnet3 1.85 GBits/sec, load: 60-80%Sys 0-0.8%Intr if_vmx <-> if_vmx 1.51 GBits/sec, load: 10-45%Sys 40-48%Intr !!! if_vmxjumbo <-> if_vmxjumbo not possible if_em(e1000) <-> if_em(e1000) 1.23 GBits/sec, load: 80-60%Sys 0.5-8%Intr if_em(e1000)jumbo <-> if_em(e1000)jumbo 2.27Gbits/sec, load: 40-30%Sys 0.5-5%Intr if_igb(e1000e)junmbo <-> if_igb(e1000e)jumbo 5.03 Gbits/s, load: 70-60%Sys 0.5%Intr if_igb(e1000e) <-> if_igb(e1000e) 1.39 Gbits/s, load: 60-80%Sys 0.5%Intr f_igb(e1000e) <-> if_igb(e1000e), both hw.em.[rt]xd=3D4096 1.66 Gbits/s, load: 65-90%Sys 0.5%Intr if_igb(e1000e)junmbo <-> if_igb(e1000e)jumbo, both hw.em.[rt]xd=3D4096 4.81 Gbits/s, load: 65%Sys 0.5%Intr Conclusion: if_vmx performs well compared to the regular emulated nics and standard MTU, but it's behind tuned e1000e nic emulation and can't reach vmxnet3 performance with regular mtu. If one needs throughput, the missing jumbo frame support in if_vmx is a show stopper. e1000e is preferable over e1000, even if not officially choosable with "FreeBSD"-selection as guest (edit .vmx and alter ethernet0.virtualDev =3D= "e1000e", and dont forget to set hw.em.enable_msix=3D0 in loader.conf, although the driver e1000e attaches is if_igb!) Thanks, -Harry --------------enig68C5E2ACB8ED11F3249CE469 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAlIU04UACgkQLDqVQ9VXb8hmmgCgqjslR9vbXAE44fjkm2eSIUqH AhYAoMW6CZ3z3+5etkrA4RV9nJo2XoyO =4WUd -----END PGP SIGNATURE----- --------------enig68C5E2ACB8ED11F3249CE469--