Date: Mon, 23 May 2011 10:43:10 -0700 From: Ben Hutchings <bhutchings@solarflare.com> To: Luigi Rizzo <rizzo@iet.unipi.it> Cc: net@freebsd.org Subject: Re: strange performance receiving with ixgbe/82599 Message-ID: <1306172590.3456.75.camel@localhost> In-Reply-To: <20110523172010.GA540@onelab2.iet.unipi.it> References: <20110523172010.GA540@onelab2.iet.unipi.it>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 2011-05-23 at 19:20 +0200, Luigi Rizzo wrote: > Jack, > i am doing some experiments with a modified ixgbe driver, and i am > seeing some strange performance numbers which I'd like to explain. > > My card uses the 82599 chip, monted on an x16 slot on an Asus > motherboard. CPU is i7-870 @ 2.93GHz. Both -head and RELENG_8 > exhibit the same behaviour. I am using a modified driver which > essentially accesses the rings from userspace, sending or > receiving frames as soon as there is room. > > On the transmit side, using 64 byte packets (60+4 CRC) i can saturate > the link (measured 14.87-14.88 Mpps) with no problem with just one > core at 1.33 GHz and 2 or 4 queues (one queue tops at 12Mpps, > but that's not a big issue). > > On the receive side, i can sustain 14.2Mpps (on 2-4 queues) if the > sender generates 68-byte packets (i.e. 64+4CRC), but as soon as i > go to 63+4 or less, the receive rate drops to 11.5-11.6Mpps. > CPU cycles abound, because the speed is unchanged down to 1.45GHz > > Any reason to explain that suddend drop with short (but legal) > packet sizes ? [...] With 64-bit wide RAM, a DMA write of 64 bytes with 8-byte alignment covers exactly 8 memory words. However a DMA write of 63 bytes with 8-byte alignment at the start covers 7 full and one partial memory word. So a read-modify-write operation is required for the last word. Ben. -- Ben Hutchings, Senior Software Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1306172590.3456.75.camel>