From owner-freebsd-net@FreeBSD.ORG Mon Sep 13 19:08:11 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 326BB106566B; Mon, 13 Sep 2010 19:08:11 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from eu1sys200aog114.obsmtp.com (eu1sys200aog114.obsmtp.com [207.126.144.137]) by mx1.freebsd.org (Postfix) with SMTP id D5DA18FC0A; Mon, 13 Sep 2010 19:08:09 +0000 (UTC) Received: from source ([63.174.175.251]) by eu1sys200aob114.postini.com ([207.126.147.11]) with SMTP ID DSNKTI52l2bh5Gl3RBYpFxIcIKVWVIlB7Nbw@postini.com; Mon, 13 Sep 2010 19:08:10 UTC Received: from [172.17.10.53] (unknown [172.17.10.53]) by bbbx3.usdmm.com (Postfix) with ESMTP id 9852CFD019; Mon, 13 Sep 2010 19:08:06 +0000 (UTC) Message-ID: <4C8E768E.7000003@tomjudge.com> Date: Mon, 13 Sep 2010 14:07:58 -0500 From: Tom Judge User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.12) Gecko/20100826 Lightning/1.0b1 Thunderbird/3.0.7 MIME-Version: 1.0 To: pyunyh@gmail.com References: <4C894A76.5040200@tomjudge.com> <20100910002439.GO7203@michelle.cdnetworks.com> <4C8E3D79.6090102@tomjudge.com> <20100913184833.GF1229@michelle.cdnetworks.com> In-Reply-To: <20100913184833.GF1229@michelle.cdnetworks.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, davidch@broadcom.com, yongari@freebsd.org Subject: Re: bce(4) - com_no_buffers (Again) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Sep 2010 19:08:11 -0000 On 09/13/2010 01:48 PM, Pyun YongHyeon wrote: > On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote: > >> >> Does this mean that these cards are going to perform badly? This is was >> what I gathered from the previous thread. >> >> > I mean there are still many rooms to be done in driver for better > performance. bce(4) controllers are one of best controllers for > servers and driver didn't take full advantage of it. > > So far our experiences with bce(4) on FreeBSD have been very disappointing. Starting with when Dell switched to bce(4) based NIC's (around the time 6.2 was released and with the introduction of the Power Edge X9XX hardware) we have always had problems with the driver in every release we have used: 6.2, 7.0 and 7.1. Luckily David has been helpful and helped us fix the issues. > >> Without BCE_JUMBO_HDRSPLIT then we see no errors. With it we see number >> of errors, however the rate seems to be reduced compaired to the >> previous version of the driver. >> >> > It seems there are issues in header splitting and it was disabled > by default. Header splitting reduces packet processing overhead in > upper layer so it's normal to see better performance with header > splitting. > The reason that we have had header splitting enabled in the past is that historically there have been issues with memory fragmentation when using 8k jumbo frames (resulting in 9k mbuf's). I have a kernel with the following configuration in testing right now: * Flow control enabled. * Jumbo header splitting turned off. Is there any way that we can fix flow control with jumbo header splitting turned on? Thanks Tom PS. The following test was more than enough to trigger buffer shortages with header splitting on: ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done ) & ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done ) & ( while true; do ldapsearch -h ldap-server1 -b "ou=Some,o=Base" dn; done ) & The search in question returned about 1700 entries. -- TJU13-ARIN