Date: Mon, 13 Sep 2010 12:33:22 -0700 From: Pyun YongHyeon <pyunyh@gmail.com> To: Tom Judge <tom@tomjudge.com> Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) Message-ID: <20100913193322.GG1229@michelle.cdnetworks.com> In-Reply-To: <4C8E768E.7000003@tomjudge.com> References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> <4C8E768E.7000003@tomjudge.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Sep 13, 2010 at 02:07:58PM -0500, Tom Judge wrote: > On 09/13/2010 01:48 PM, Pyun YongHyeon wrote: > > On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > > > >> > <SNIP/> > >> Does this mean that these cards are going to perform badly? This is was > >> what I gathered from the previous thread. > >> > >> > > I mean there are still many rooms to be done in driver for better > > performance. bce(4) controllers are one of best controllers for > > servers and driver didn't take full advantage of it. > > > > > > So far our experiences with bce(4) on FreeBSD have been very > disappointing. Starting with when Dell switched to bce(4) based NIC's > (around the time 6.2 was released and with the introduction of the Power > Edge X9XX hardware) we have always had problems with the driver in every > release we have used: 6.2, 7.0 and 7.1. Luckily David has been helpful > and helped us fix the issues. > > <SNIP/> > > > >> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number > >> of errors, however the rate seems to be reduced compaired to the > >> previous version of the driver. > >> > >> > > It seems there are issues in header splitting and it was disabled > > by default. Header splitting reduces packet processing overhead in > > upper layer so it's normal to see better performance with header > > splitting. > > > > The reason that we have had header splitting enabled in the past is that > historically there have been issues with memory fragmentation when using > 8k jumbo frames (resulting in 9k mbuf's). > Yes, if you use jumbo frames, header splitting would help to reduce memory fragmentation as header splitting wouldn't allocate jumbo clusters. > I have a kernel with the following configuration in testing right now: > > * Flow control enabled. > * Jumbo header splitting turned off. > > > Is there any way that we can fix flow control with jumbo header > splitting turned on? > Flow control has nothing to do with header splitting(i.e. flow control is always enabled). > Thanks > > Tom > > PS. The following test was more than enough to trigger buffer shortages > with header splitting on: > > ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done > ) & > ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done > ) & > ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done > ) & > > The search in question returned about 1700 entries. > I can trigger this kind of buffer shortage with benchmark tools. Actually fixing header splitting is on my TODO list as well as other things but I don't know how long it would take.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100913193322.GG1229>