From owner-freebsd-stable@FreeBSD.ORG Mon Aug 26 08:34:47 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9EB04F6E for ; Mon, 26 Aug 2013 08:34:47 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qe0-x22c.google.com (mail-qe0-x22c.google.com [IPv6:2607:f8b0:400d:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6145C23AC for ; Mon, 26 Aug 2013 08:34:47 +0000 (UTC) Received: by mail-qe0-f44.google.com with SMTP id 5so1604813qeb.3 for ; Mon, 26 Aug 2013 01:34:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6Ds1RzTHymgkOajSmkrn+Ecsq5Fc2WCaGlKk76p/oBw=; b=mwgy2cJjhjzGB/7GzM4deI1+3jwpeT/XpWERBRigf+AgPPAdsndMvI0Y9jDrU3K02d CaKIDv1OZeyf+wmeVk3tMLRO1WBG7Q6A1mCiKLDtCyLFaoc8ayhZVsCSTItGfaDvHT8V 8+4WuJv/wRGXgU2+e/1zEgVNmClpbTN3PrCzaGx0Fqp4VsnlbIX44eahhv2DGY1GHPqg Dpo4MHk/KEup7FblaQ9FZOhfmBSF53SdKraCWtr98a6hdp9Ywv7A7uYnH17iQ1lsK14G 4+185PbtF0W/9SBBE8dm0mFgQ3RaxfjxZ8epCRVhvSc6LczUpTVy7ApeG1AYRwD1+vOe BNEg== MIME-Version: 1.0 X-Received: by 10.224.66.74 with SMTP id m10mr14880760qai.12.1377506086401; Mon, 26 Aug 2013 01:34:46 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.128.70 with HTTP; Mon, 26 Aug 2013 01:34:46 -0700 (PDT) In-Reply-To: <521AFE7E.2040705@omnilan.de> References: <521AFE7E.2040705@omnilan.de> Date: Mon, 26 Aug 2013 01:34:46 -0700 X-Google-Sender-Auth: oGPvsLfQzfY-4Llk5D5_Wu7Ts_0 Message-ID: Subject: Re: if_em, legacy nic and GbE saturation From: Adrian Chadd To: Harald Schmalzbauer Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Stable Mailing List X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Aug 2013 08:34:47 -0000 Hi, There's bus limits on how much data you can push over a PCI bus. You can look around online to see what 32/64 bit, 33/66MHz PCI throughput estimates are. It changes massively if you use small versus large frames as well. The last time I tried it i couldn't hit gige on PCI; I only managed to get to around 350mbit doing TCP tests. -adrian On 26 August 2013 00:06, Harald Schmalzbauer wrote: > Hello, > > I recycled an older box and put an i350-2 together with a second 82541GI > (PCI-slot, one already on-board) into it. > The two i350-ports are used with VMDq for ESXi5.1. > The two 82541GI are used as lagg-nics by a 9.2-RC (amd64) guest as > passthrou PCI device. > Always had good results with such setups, but found out, that nics which > use the legacy driver part of if_em max out at ~0.6Gbits/s (1500 MTU). > > There's another NIC on board of this recycle-box, a 82566-PHY (ICH9 > integrated MAC). > This one uses also if_em, but not legacy code, it reports version 7.3.8 > (compared to 1.0.6). > And it has no problem fully saturating GbE (~925Mbits/s, no jumbo Frames > support anyways). > > I'm using iperf, with and without lagg (doesn't change anyhing, like it > doesn't influence tests on some other boxes with 82576 and i350 (igb)) > I see enough idle cycles so CPU shouldn't limit the legacy if_em nics. > Also, I see the 82541 consuming arround 8k irqs. Same does the > 82566-PHY, but with much higher throughput... > > I'd like to know if I can't generally expect to saturate older (PCI) GbE > nics the line for any reason... I can remember tigeon cards from more > than a decade ago, which indeed seemd to lack the performance to gain > GbE, but I thought that was no issue shortly later and no "modern" > Intel-GbE card had such constraints!? > Is there any special tuning for legacy if_em (no need for any TCP > tuning, 82566 doesn't have any issue)? > > Thanks, > > -Harry > >