From owner-freebsd-net@FreeBSD.ORG Sat Jan 2 19:04:42 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07802106566B for ; Sat, 2 Jan 2010 19:04:42 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63903.mail.re1.yahoo.com (web63903.mail.re1.yahoo.com [69.147.97.118]) by mx1.freebsd.org (Postfix) with SMTP id B7C158FC15 for ; Sat, 2 Jan 2010 19:04:41 +0000 (UTC) Received: (qmail 59701 invoked by uid 60001); 2 Jan 2010 19:04:33 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1262459073; bh=ON7N/p15nvTP4SJpJjTABn05Z7ZHcpfkQwvwrUENaIc=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=sCdwYuMn5Xz/VcMatDCX7Id2meFElIiG841u+PNK1w5x3sv+s4LXFUKKmST9wnWikFribqpK4cnu85b/9ILE+eAujtZqd7SDwTUwJly1UYQLPXHd/GGDME9LCc9Bbx9OmQo4E+/lT0kERGfY6pO1k2KT3ESW5YtdHB/NmueC41A= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kGeGDVZHNd0LphLGFp8w5g2lb9CIowVHjgJEOQ+v6Ah9YgzHKkp0ZjGxJY10pWo7r3gqldQ/czIMhyaNDZiFlrPUSs4TJeKD4xXDm1ikq53fhkWRGf2uan75FtiVWQzIw91+Bpp94qejlteGdRd5rEO8LjxD1V7nAtTKZ9e+3PM=; Message-ID: <422185.59153.qm@web63903.mail.re1.yahoo.com> X-YMail-OSG: tefQcxMVM1kNA_i6A9lc52dYcldYqYc4pzJrnROtsgyJBHuVNNT8hYmBt_348WDVEc87.SreK8wFRlybFWx65ivMoh7Um3p0a9pYpGTDYSS0cbsKrmc_ZA.FRGBLIqQJD1O5Nyu.Tt6YEHOq4dPCdD4mbfaoM.gHcuP5Yd1WcI0XJEfMzohlBi2Lrymp2NyoPX1VIxZjhjB35.byjb3O2ktNaIBmK7iRO1.oZo51mcHlpEhxkzpks1HZiDpVZTQ6m6wjjfeQzwWWzXO4tJBamPY5d314rl8RLZ3KaR_s7wPyZSERtg-- Received: from [98.203.21.152] by web63903.mail.re1.yahoo.com via HTTP; Sat, 02 Jan 2010 11:04:33 PST X-Mailer: YahooMailClassic/9.0.20 YahooMailWebService/0.8.100.260964 Date: Sat, 2 Jan 2010 11:04:33 -0800 (PST) From: Barney Cordoba To: George Sanders , Jack Vogel In-Reply-To: <2a41acea0912111851x598a4f5bsccf013e0c8107743@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org Subject: Re: 10gigE link with FreeBSD hosts ? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jan 2010 19:04:42 -0000 =0A=0A--- On Fri, 12/11/09, Jack Vogel wrote:=0A=0A> Fr= om: Jack Vogel =0A> Subject: Re: 10gigE link with FreeBS= D hosts ?=0A> To: "George Sanders" =0A> Cc: freebsd-n= et@freebsd.org=0A> Date: Friday, December 11, 2009, 9:51 PM=0A> Can't get 1= 0 gigabit thru a PCI-X=0A> slot, it doesnt have the bandwidth, you=0A> need= PCI Express, in=0A> fact you will find that lots of factors will come into= play=0A> and might keep=0A> you from achieving=0A> optimal thruput.=0A> = =0A> For a two port adapter you will need an 8x PCIE slot,=0A> preferably G= en 2, and=0A> if all three systems=0A> don't have that its gonna slow every= one down.=0A> =0A> Further, once you involve packet forwarding it gets even= =0A> more demanding.=0A> =0A> Trying to do it 'on the cheap' and you might = get 3 or 4 Gb,=0A> maybe even worse=0A> depending.=0A> =0A> Takes quality t= o get real performance, sorry :)=0A> =0A> Jack=0A> =0A> =0A> On Fri, Dec 11= , 2009 at 2:30 PM, George Sanders wrote:=0A> =0A> >= =0A> >=0A> > I am seeing Sun Microsystems branded 10gigE adaptors=0A> (pci-= x) on ebay=0A> > nowadays for $500.=0A> >=0A> > So ... let's say I took som= e plain-jane, modern-ish PC=0A> (intel core, or c2d,=0A> > pci-x, etc.) and= set up this topology:=0A> >=0A> >=0A> > a <-- cable --> b=A0 <-- cable -->= c=0A> >=0A> > and on each of (a) and (c) put one 10gigE adaptor, and=0A> o= n (b) put two=0A> > 10gigE adaptors, and set up bridging on (b)=0A> >=0A> >= Would it be reasonable to then expect to transfer data=0A> from (a) to (c)= at=0A> > a, roughly, 10 gigabit/s rate ?=0A> >=0A> > If not, what is the l= imiting factor ?=0A> >=0A> > The idea is that I would use a commodity PC + = multiple=0A> pci-X slots as a=0A> > "poor mans" 10gigE switch ... I can't a= fford an actual=0A> 10gigE switch, but I=0A> > don't need that many ports a= nyway.=0A> >=0A> > Any comments or field reports would be appreciated.=0A> = >=0A> >=0A=0ASupermicro makes some cheap cards that are under $500. Part# i= s=0AAOC-STG-i2 and they work with FreeBSD. You should be able to find=0Athe= m for around $450, so if you just need a 2 port switch its fairly=0Ainexpen= sive.=0A=0AThe problem is that the FreeBSD ixgbe driver can't do close to 1= 0Gb/s, =0Aso its not a very good switch. =0A=0ABarney=0A=0A=0A