From owner-freebsd-doc@FreeBSD.ORG Mon Jun 8 10:53:08 2009 Return-Path: Delivered-To: doc@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04404106564A for ; Mon, 8 Jun 2009 10:53:08 +0000 (UTC) (envelope-from freebsdusb@bindone.de) Received: from mail.bindone.de (mail.bindone.de [80.190.134.51]) by mx1.freebsd.org (Postfix) with SMTP id 2BAAA8FC17 for ; Mon, 8 Jun 2009 10:53:06 +0000 (UTC) (envelope-from freebsdusb@bindone.de) Received: (qmail 98452 invoked by uid 89); 8 Jun 2009 10:26:25 -0000 Received: from unknown (HELO ufo.bindone.de) (mg@bindone.de@87.152.191.228) by mail.bindone.de with ESMTPA; 8 Jun 2009 10:26:25 -0000 Message-ID: <4A2CE74E.9020309@bindone.de> Date: Mon, 08 Jun 2009 12:26:22 +0200 From: Michael User-Agent: Thunderbird 2.0.0.17pre (X11/20090202) MIME-Version: 1.0 To: doc@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Request for errate notice on igb(4) X-BeenThere: freebsd-doc@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Documentation project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 10:53:08 -0000 Hello Doc-Team, I would like to request an errata notice on igb(4). Without disabling lro, ip forwarding results in horrible performance (at least with onboard Intel 82575EB Dual-Port Gigabit Ethernet Controllers). As a workaround lro can be disabled, but this needs to be done on reboot (sysctl.conf), tuning the sysctls it after interfaces have been activated has no effect. See also my email "Bug in igb driver" and the mail "[igb] low speed routing between two igb interfaces" a few days later (kern/135222). This problem happens on 7.1-RELEASE, 7.2-RELEASE, 7-STABLE and 8-CURRENT. I'm writing you directly, because nobody at the bugs mailing list thinks this problem needs to be adressed - I totally disagree, ecause these adapters are very common and a small errata noticed would've saved us days of frustration (and potentially will others as well). cheers Michael --- Suggested errata notice (something similar): An issue was found in the igb(4) driver that causes desastrous performance when FreeBSD acts as a gateway (ip forwarding enabled) and LRO (large receive offloading) is enabled (which is the default setting). As a workaround, disable LRO by adding the following lines to /etc/sysctl.conf: dev.igb.0.enable_lro=0 dev.igb.1.enable_lro=0 and reboot. ------- My Mail on freebsd-bugs Jun 2nd 2009 Hello, we encountered problems using igb driver on a Supermicro dual quad-core system (X7DWU). It features an Intel Gigabit dual port controller (82575). The IP forwarding performance was extremely bad (5 kb/s up to 50 kb/s) until we disabled Large Receive Offloading in /etc/sysctl.conf: dev.igb.0.enable_lro=0 dev.igb.1.enable_lro=0 (changing it using ifconfig -lro and down/up the interfaces didn't help). With the systctl values set as shown above, we get the expected performance ob 50-60 MB/s. We tested this on 7.1-RELEASE, 7.2-RELEASE and 7-STABLE. Right now we're installing 8-CURRENT on the machine to see if it makes a difference. It think it would be worthwhile to list this issue in the 7.2 release errata section (that would've saved us a significant amount of time :) . Cheers Michael ------ Mail thread on kern/135222 The following reply was made to PR kern/135222; it has been noted by GNATS. From: Michael To: Mishustin Andrew <1@hnt.ru> Cc: freebsd-gnats-submit@FreeBSD.org Subject: Re: kern/135222: [igb] low speed routing between two igb interfaces Date: Thu, 04 Jun 2009 09:41:22 +0200 Hello Andrew, look at my post "Bug in igb driver" from two days ago. We had exactly the same issues using this adapter (and presumambly the same board, your bug description matches 100% our setup :) The following should fix those issues: echo "dev.igb.0.enable_lro=0" >>/etc/sysctl.conf echo "dev.igb.1.enable_lro=0" >>/etc/sysctl.conf reboot (this disabled large receive offloading - tuning this while the machine is running has no effect). I suggested putting this into the release errata, but got no feedback on this request so far. cheers Michael Mishustin Andrew wrote: >> Number: 135222 >> Category: kern >> Synopsis: [igb] low speed routing between two igb interfaces >> Confidential: no >> Severity: serious >> Priority: medium >> Responsible: freebsd-bugs >> State: open >> Quarter: >> Keywords: >> Date-Required: >> Class: sw-bug >> Submitter-Id: current-users >> Arrival-Date: Wed Jun 03 18:30:01 UTC 2009 >> Closed-Date: >> Last-Modified: >> Originator: Mishustin Andrew >> Release: FreeBSD 7.1-RELEASE amd64, FreeBSD 7.2-RELEASE amd64 >> Organization: > HNT >> Environment: > FreeBSD test.hnt 7.2-RELEASE FreeBSD 7.2-RELEASE #12: Thu Apr 30 18:28:15 MSD 20 > 09 admin@test.hnt:/usr/src/sys/amd64/compile/GENERIC amd64 >> Description: > I made a FreeBSD multiprocesor server to act as simple gateway. > It use onboard Intel 82575EB Dual-Port Gigabit Ethernet Controller. > I observe traffic speed near 400 Kbit/s. > I test both interfaces separately - > ftp client work at speed near 1 Gbit/s in both directions. > Then I change NIC to old Intel "em" NIC - gateway work at speed near 1 Gbit/s. > > Looks like a bug in igb driver have an effect upon forwarded traffic. > > If you try > hw.igb.enable_aim=0 > The speed is near 1 Mbit/s > > hw.igb.rxd, hw.igb.txd, "ifconfig -tso" has no effect. > > Nothing in messages.log > > netstat -m > 516/1674/2190 mbufs in use (current/cache/total) > 515/927/1442/66560 mbuf clusters in use (current/cache/total/max) > 515/893 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/44/44/33280 4k (page size) jumbo clusters in use (current/cache/total/max) > 0/0/0/16640 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/8320 16k jumbo clusters in use (current/cache/total/max) > 1159K/2448K/3607K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > I use only IPv4 traffic. > >> How-To-Repeat: > On machine with two igb interfaces > use rc.conf like this: > > hostname="test.test" > gateway_enable="YES" > ifconfig_igb0="inet 10.10.10.1/24" > ifconfig_igb1="inet 10.10.11.1/24" > > And try create heavy traffic between two networks. >> Fix: > > >> Release-Note: >> Audit-Trail: >> Unformatted: > _______________________________________________ > freebsd-bugs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-bugs > To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@freebsd.org" _______________________________________________ freebsd-bugs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-bugs To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@freebsd.org" --- Feedback received by M. Andrew (outside of the mailing list) Hello Michael, Thank you. It's working. I consider it necessary to put this into the release errata.