From owner-freebsd-net@FreeBSD.ORG Mon Apr 12 17:57:07 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 979B91065672 for ; Mon, 12 Apr 2010 17:57:07 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id 02D628FC0A for ; Mon, 12 Apr 2010 17:57:06 +0000 (UTC) Received: by bwz8 with SMTP id 8so3308482bwz.3 for ; Mon, 12 Apr 2010 10:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=S4G8FicKEI8Zi+aHYL8RQZzXk9Yt/b5Knt5uAg6IJqo=; b=E7fG6f7kiyItthPNCkqANtCYbGmAAMfztwTKpkfR4k2u8JQWyzgzMF97OqJ+kiheMZ 6FJAn+KnHw4UWG1uyKKKxhg72MB5in3V5VCZqd4qtbggYixLyOqb2kNIugt514IzqjY3 rbFqOliiauYh6M8Y61eQ+iAvwH3ziCxHFVNpA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=vmTaG67Q9G3msJxOIPAauD24iQzOSuL5XisjTV6qqGZKJ0cRbzXV281tSTiCqemjU1 2oeUxxLVK6fTfL4L5qO3MdGrmqfCuAVJHfM2+YnU861UqAtciuh1kkFMlC5gGLyvBvug GBbCpK63E3cSldJXcwqCLcSRNz1B1CCnAkrLQ= Received: by 10.204.141.27 with SMTP id k27mr5328231bku.26.1271095025833; Mon, 12 Apr 2010 10:57:05 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 15sm1903399bwz.8.2010.04.12.10.57.01 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 12 Apr 2010 10:57:03 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 12 Apr 2010 10:57:01 -0700 From: Pyun YongHyeon Date: Mon, 12 Apr 2010 10:57:01 -0700 To: "Erich Jenkins, Fuujin Group Ltd" Message-ID: <20100412175701.GC1444@michelle.cdnetworks.com> References: <4BBED9A4.3080303@fuujingroup.com> <20100409070147.GA77350@korolev-net.ru> <4BBEE18C.6040204@fuujingroup.com> <20100409173821.GD1085@michelle.cdnetworks.com> <4BC016F3.4020300@fuujingroup.com> <20100410212520.GB6481@michelle.cdnetworks.com> <4BC12097.4030508@fuujingroup.com> <4BC19324.3050800@fuujingroup.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BC19324.3050800@fuujingroup.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org, Evgenii Davidov Subject: Re: Broadcom BCM5701 / HP NC6770 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2010 17:57:07 -0000 On Sun, Apr 11, 2010 at 03:15:16AM -0600, Erich Jenkins, Fuujin Group Ltd wrote: > I've been muddling around in src/sys/dev on the old system and the new > system and there appear to be rather major changes to MII and bge, > possibly the whole stack? > It was not completely rewritten but many improvements were made. > There are a number of things that seem to have been merged with other > parts of the network stack, or perhaps written into the individual > drivers (someone working on the net stack would have to verify that). > > For instance, some files called in 5.3-REL seem to have gone away > completely, and in the new (unpatched) version of if_bge.c under > 7.3-REL, calls to these modules are gone: > > - #include /* for vtophys */ > - #include /* for vtophys */ One of the most significant changes would be bus_dma(9) conversion which is required to all drivers to make it work correctly on a variety of platforms. bus_dma(9) does not directly use vtophys anymore so these headers were nuked. > - #include /* for DELAY */ > - #include > > - #include (called but something changed in here) > - #include (ditto above) > No, these headers are still present. > It appears that the checksum features have been completely rewritten, Checksum offloading was not completely rewritten but workaround for buggy controllers was added. > and some of the ring settings have changed. It's interesting that the > driver only fills 256 of the rx rings in the hopes that the cpu is "fast > enough to keep up with the NIC". Would a subroutine here to grab the cpu That magic number 256 is adequate for most cases but it may not be enough to handle heavy loads. Internally the controller use fixed 512 RX buffers but bge(4) used only half of the buffers to save resources. I think you can increase SSLOTS to 512 to get full 512 RX buffers. > clock and count (number of procs/pipelines) be more trouble than it's > worth to "automagically" increase the number of rx rings the driver > fills based on the system in which it's installed? > Dynamically increasing number of RX buffers is doable but it would add much more code. If there is high demand for that I would just increase number of RX buffers to 512. Controller can't be configured to have more than 512 RX buffers. > Something also changed in pci/pcireg.h and pci/pcivar.h, but I haven't > had the time to hunt down and expand the source tree from the 5.3-REL > branch yet. > > I have other machines with copper nics utilizing the bge driver, and > there are no issues at all. Perhaps I'm getting ahead of things, but Yes that is expected one. :-) > since this seems to have been broken through several releases, would it > make any sense to split the support between the BCM5701KHB chipset and > the more recent BCM chipset to avoid causing issues with cards/systems > not currently experiencing troubles? > I'd like to if I can. Supporting huge number of different controllers in single driver is maintenance nightmare. However, rewriting some part that require special handling for certain controller/revision is too risky because I don't have access to most controllers. One theory for the issue I got while reading the code is link state handling. As I said in previous mail, link state handling for TBI is somewhat tricky in bge(4) and driver seemed to rely on periodic register access to keep track of link state. I guess polling(4) may give different behavior on link state handling as it does not rely on interrupts at all. So would you try to use polling(4) and see that make any difference on your box?