Date: Thu, 28 Apr 2011 09:25:13 -0400 From: Adam Stylinski <kungfujesus06@gmail.com> To: Pierre Lamy <pierre@userid.org> Cc: freebsd-net@freebsd.org Subject: Re: em0 performance subpar Message-ID: <20110428132513.GB2800@ossumpossum.geop.uc.edu> In-Reply-To: <4DB961EA.4080407@userid.org> References: <20110428072946.GA11391@zephyr.adamsnet> <4DB961EA.4080407@userid.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--1LKvkjL3sHcu1TtY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 28, 2011 at 08:47:38AM -0400, Pierre Lamy wrote: > Try using netblast on FreeBSD instead of iperf, there have been a lot of= =20 > discussions about this on this list. >=20 > Is it possible you're maxing out the system's PCI-xxx bus? Did you tune= =20 > up the system buffers? Data doesn't just get queued up on the NIC=20 > driver, it also queues within the system's kernel buffers. Try these, I= =20 > have no idea if it will help: >=20 > sysctl -w net.inet.tcp.sendspace=3D373760 > sysctl -w net.inet.tcp.recvspace=3D373760 > sysctl -w net.local.stream.sendspace=3D82320 > sysctl -w net.local.stream.recvspace=3D82320 > sysctl -w net.local.stream.recvspace=3D373760 > sysctl -w net.local.stream.sendspace=3D373760 > sysctl -w net.raw.recvspace=3D373760 > sysctl -w net.raw.sendspace=3D373760 > sysctl -w net.inet.tcp.local_slowstart_flightsize=3D10 > sysctl -a net.inet.tcp.delayed_ack=3D0 > sysctl -w kern.maxvnodes=3D600000 > sysctl -w net.local.dgram.recvspace=3D8192 > sysctl -w net.local.dgram.maxdgram=3D8192 > sysctl -w net.inet.tcp.slowstart_flightsize=3D10 > sysctl -w net.inet.tcp.path_mtu_discovery=3D0 >=20 > They're all tunable while system is running. >=20 > -Pierre >=20 > On 4/28/2011 3:29 AM, Adam Stylinski wrote: > > Hello, > > > > I have an intel gigabit network adapter (the 1000 GT w/chipset 82541PI)= which performs poorly in Freebsd compared to the same card in Linux. I've= tried this card in two different freebsd boxes and for whatever reason I g= et poor transmit performance. I've done all of the tweaking specified in j= ust about every guide out there (the usual TCP window scaling, larger nmbcl= usters, delayed acks, etc) and still I get only around 600mbps. I'm using = jumbo frames, with an MTU of 9000. I'm testing this with iperf. While I r= ealize that this may not be the most realistic test, linux hosts with the s= ame card can achieve 995Mbit/s to another host running this. When the Free= bsd box is the server, Linux hosts can transmit to it at around 800 somethi= ng Mbit/s. I've increased the transmit descriptors as specified in the if_= em man page, and while that gave me 20 or 30 more mbit/s, my transmit perfo= rmance is still below normal. > > > > sysctl stats report that the card is trigger a lot of tx_desc_fail2: > > dev.em.0.tx_desc_fail2: 3431 > > > > Looking at a comment in the source code this indicates that the card wa= s not able to obtain enough transmit descriptors (but I've given the card t= he maximum of 4096 in my loader.conf tunable). Is this a bug or a performa= nce regression of some kind? Does anybody have a fix for this? I tried an= other card with the same chip in a different box on 8-STABLE to no avail (t= he box I'm trying to improve performance on is on 8.2-RELEASE-p1). > > > > Anybody manage to make this card push above 600mbps in ideal network be= nchmarks? Any help would be gladly appreciated. >=20 >=20 >=20 I doubt I'm saturating the PCI bus, the only other thing on the bus is a re= ally really crappy PCI video card. The same card on lesser powerful machin= es with Linux (Pentium 4) are able to achieve much more throughput, so it's= not likely a bus limitation. I adjusted the listed sysctl live tunables which I hadn't already compensat= ed for and it didn't seem to have an effect. --=20 Adam Stylinski PGP Key: http://pohl.ececs.uc.edu/~adam/publickey.pub Blog: http://technicallyliving.blogspot.com --1LKvkjL3sHcu1TtY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iQIcBAEBAgAGBQJNuWq5AAoJED6sRHE6Tvmnmm4P/19McIX8pZAIfBAvWU9WzIBi iEsbkyKMgTjRTxbWR1fvEw6XYqcqDkialypiQH66AdIppCFs4bli1APG7Urw+XAE LY01qgcWWWsFIxZgxADlMHmrVPtBH+rCsdNlnhFmyhS85Mq8sxfyBqbB5h6OjUJW mOBdacpfDb2BAkaMxni4B5yjJoLZu9R2iS+ttnNtp2x6ccfgtgEWW1G3jaUY6qpW SUM7MXh5hiQzpmHnl1VQoFcW31CeCwgeK6xsscl2rlQM5+FFD5+sLnBl5M0vsyGt 4gOe3DmzP7+v+Ul3TYW9VvIYPE+p6fFn1ZgM65ry4nTkABR0AvAyKfiP2ebTzCyR Y81Q6aXP5LPUWxwJPSMqFKZkGP+KwtgO3ZMkScz4EBGE8b163+DmJFT1rnKrX6io f55JYsoh2OeLFRstmSnu5HalhtcTSQZVAUFWpQIPogmgc76g3Sov9s51RCsU7RYo 5u0qTTmphACs4UvHc5NFV6znu/5MJcJdZSDa3kzmffhmBg8ViL2yVuW1e20KSKmG /8F4kJycvuJ+QesYC82umyR1wYRj6uWJOvSbA2Jkr4+RfNMiTw++bnJ+rWr49yBj Xo5lB5jLhaXU/tq3sIwaNBjm9MgglgIylCLMS4ZpuHxpG4dR3t/wwI6D3Y1NqUZ/ yYDQB0yb6hf6tOMZekg6 =/JUS -----END PGP SIGNATURE----- --1LKvkjL3sHcu1TtY--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110428132513.GB2800>