Date: Wed, 29 Jan 2014 18:05:23 -0800 From: Navdeep Parhar <nparhar@gmail.com> To: Adrian Chadd <adrian@freebsd.org>, Garrett Wollman <wollman@csail.mit.edu>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Big physically contiguous mbuf clusters Message-ID: <20140130020523.GB18434@ox> In-Reply-To: <20140130013434.GP93141@funkthat.com> References: <21225.20047.947384.390241@khavrinen.csail.mit.edu> <CAJ-VmomC5Ge3JwfUsgMrJ_rGqiYxfxR4wWzn5A-KAu7HBsueMw@mail.gmail.com> <20140129231121.GA18434@ox> <20140130013434.GP93141@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jan 29, 2014 at 05:34:34PM -0800, John-Mark Gurney wrote: > Navdeep Parhar wrote this message on Wed, Jan 29, 2014 at 15:11 -0800: > > > (scatter list has to be fed to the chip and now it's 3x what it has to > > be), 3x the number of "wrapper" mbuf allocations (one for each 4K > > cluster) which will then be stitched together to form a frame, etc. etc. > > And what is that in percentage of overall traffic? .4% (assuming 16 bytes > per 4k page)... If your PCIe bus is saturating and you need that extra > .4% traffic, then you have a serious issue w/ your bus layout... The 16B and 4KB are in different directions, the former is from host to chip and the latter from chip to host memory. So the 16B eats into the transmit bandwidth. FWIW, I do deal with cards where PCIe is the limiting factor (a 4x10G card with a pcie gen2 x8 block, a 2x40G card with pcie gen3 x8 block) and the effects of 4K vs. 9K rx on the transmit bandwidth are measurable. These days chips can even place multiple frames into a single buffer (if they'd fit) and that's another reason I tend to advocate for larger contiguous buffer sizes. Regards, Navdeep
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140130020523.GB18434>