From owner-freebsd-stable@FreeBSD.ORG Thu Dec 22 10:52:45 2005 Return-Path: X-Original-To: freebsd-stable@FreeBSD.org Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CD73116A41F for ; Thu, 22 Dec 2005 10:52:45 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (cell.sick.ru [217.72.144.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21FF643D7E for ; Thu, 22 Dec 2005 10:52:36 +0000 (GMT) (envelope-from glebius@FreeBSD.org) Received: from cell.sick.ru (glebius@localhost [127.0.0.1]) by cell.sick.ru (8.13.3/8.13.3) with ESMTP id jBMAqF5x018899 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 22 Dec 2005 13:52:15 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.sick.ru (8.13.3/8.13.1/Submit) id jBMAqFwn018898; Thu, 22 Dec 2005 13:52:15 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.sick.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 22 Dec 2005 13:52:15 +0300 From: Gleb Smirnoff To: Danny Braniss Message-ID: <20051222105215.GB41381@cell.sick.ru> References: <20051222103027.GZ41381@cell.sick.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.6i Cc: freebsd-stable@FreeBSD.org Subject: Re: em bad performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2005 10:52:45 -0000 On Thu, Dec 22, 2005 at 12:37:53PM +0200, Danny Braniss wrote: D> > On Thu, Dec 22, 2005 at 12:24:42PM +0200, Danny Braniss wrote: D> > D> ------------------------------------------------------------ D> > D> Server listening on TCP port 5001 D> > D> TCP window size: 64.0 KByte (default) D> > D> ------------------------------------------------------------ D> > D> [ 4] local 132.65.16.100 port 5001 connected with [6.0/SE7501WV2] port 58122 D> > D> (intel westvill) D> > D> [ ID] Interval Transfer Bandwidth D> > D> [ 4] 0.0-10.0 sec 1.01 GBytes 867 Mbits/sec D> > D> [ 4] local 132.65.16.100 port 5001 connected with [5.4/SE7501WV2] port 55269 D> > D> (intel westvill) D> > D> [ ID] Interval Transfer Bandwidth D> > D> [ 4] 0.0-10.0 sec 967 MBytes 811 Mbits/sec D> > D> [ 5] local 132.65.16.100 port 5001 connected with [6.0/SR1435VP2 port 58363 D> > D> (intel dual xeon/emt64) D> > D> [ ID] Interval Transfer Bandwidth D> > D> [ 5] 0.0-10.0 sec 578 MBytes 485 Mbits/sec D> > D> D> > D> i've run this several times, and the results are very similar. D> > D> i also tried i386, and the same bad results. D> > D> all hosts are connected at 1gb to the same switch. D> > D> > So we see a strong drawback between SE7501WV2 and SR1435VP2. Let's compare the NIC D> > hardware. Can you plese show pciconf -lv | grep -A3 ^em on both motherboards? D> D> on a SE7501WV2: D> em0@pci3:7:0: class=0x020000 card=0x341a8086 chip=0x10108086 rev=0x01 D> hdr=0x00 D> vendor = 'Intel Corporation' D> device = '82546EB Dual Port Gigabit Ethernet Controller (Copper)' D> class = network D> D> on a SR1435VP2: D> em0@pci4:3:0: class=0x020000 card=0x34668086 chip=0x10768086 rev=0x05 D> hdr=0x00 D> vendor = 'Intel Corporation' D> device = '82547EI Gigabit Ethernet Controller' D> class = network The first one 82546EB is attached to fast PCI-X bus, and the 82547EI is on CSA bus. The CSA bus is twice faster than old PCI bus, CSA can handle 266 Mbps. I'm not sure but may be it has same ~50% overhead as old PCI bus. Probably our em(4) driver is not optimized enough and does too many accesses to the PCI bus, thus utilizing more bandwidth than needed to handle traffic. In this case we see that NIC on slower bus (but enough to handle Gigabit) is must slower than NIC on faster bus. (This paragraph is my own theory, it can be complete bullshit.) -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE