From owner-freebsd-fs@FreeBSD.ORG Mon Jul 4 21:15:27 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FA931065678 for ; Mon, 4 Jul 2011 21:15:27 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta04.westchester.pa.mail.comcast.net (qmta04.westchester.pa.mail.comcast.net [76.96.62.40]) by mx1.freebsd.org (Postfix) with ESMTP id 3AB018FC13 for ; Mon, 4 Jul 2011 21:15:26 +0000 (UTC) Received: from omta22.westchester.pa.mail.comcast.net ([76.96.62.73]) by qmta04.westchester.pa.mail.comcast.net with comcast id 3xCx1h0031ap0As54xFT8C; Mon, 04 Jul 2011 21:15:27 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta22.westchester.pa.mail.comcast.net with comcast id 3xFQ1h00Q1t3BNj3ixFRps; Mon, 04 Jul 2011 21:15:26 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 0051F102C36; Mon, 4 Jul 2011 14:15:22 -0700 (PDT) Date: Mon, 4 Jul 2011 14:15:22 -0700 From: Jeremy Chadwick To: Bruce Cran Message-ID: <20110704211522.GA43675@icarus.home.lan> References: <1309793921.2618.YahooMailRC@web120016.mail.ne1.yahoo.com> <4E1212A7.70405@feral.com> <4E1224BE.1020508@cran.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E1224BE.1020508@cran.org.uk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, Matthew Jacob Subject: Re: Are thumpers still interesting in 2011 ? (raidz3 on x4500 @ 3.0gbps ...) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2011 21:15:27 -0000 On Mon, Jul 04, 2011 at 09:38:22PM +0100, Bruce Cran wrote: > On 04/07/2011 20:21, Matthew Jacob wrote: > > > >IMO, until you go with flash, even 1.5Gbps is more than adequate, > >particularly for independent busses. I mean, you're not going to > >really make use of speeds greater than rotational+density, right? > > With protocol overhead, I think 1.5 Gb is a bit of a limit - for > example my Samsung disks can read data at 155 MB/s. Agreed. Likewise, Matthew, where did you get 1.5Gbps from? SATA revision 2 is 3.0Gbps. 1.5Gbps is SATA revision 1, which isn't on the Sun x4500, nor is it what the George mentioned to begin with. The x4500 uses Marvell 88SX6081 controllers, which are SATA rev 2. Quoting him: >>> If I understand correctly, the interesting thing about a Sun x4500 (a >>> "thumper") is that every one of the 48 disks has a direct path to the >>> system board, allowing for full, independent throughput from every >>> single drive. >>> >>> The downside, in 2011, is that it is a SATA2 system @ 3.0gbps. George, I wouldn't worry about SATA rev 2 being a bottleneck. However, you should probably be made aware of the fact that the on-board Marvell SATA controllers are PCI-X, so effectively each of your controllers is limited to around 1GByte/sec worth of bandwidth: http://techreport.com/discussions.x/13849 Eight (8) mechanical drives which do 120-130MByte/sec connected to one controller could hit this bottleneck. SSDs would hit the bottleneck easily. So if you're looking for "an insane speed demon", this system is probably too old. And as I understand it, exceeding PCI/PCI-X bandwidth can have weird effects on the system. If you don't plan on using the entire capacity of the x4500 (e.g. only using 24 disks or some such), ensuring you "distribute the load" of the disks across the controllers (say, 3 disks per controller) would guarantee you don't hit this bottleneck on any of your controllers. I have no idea how to determine what physical bay is wired to what physical controller on the x4500. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |