From owner-freebsd-fs@FreeBSD.ORG Sat Sep 17 07:34:09 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88C5F106564A for ; Sat, 17 Sep 2011 07:34:09 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 126278FC17 for ; Sat, 17 Sep 2011 07:34:08 +0000 (UTC) Received: from [192.92.129.101] ([192.92.129.101]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.4/8.14.4) with ESMTP id p8H7Xwa5013234 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 17 Sep 2011 10:34:04 +0300 (EEST) (envelope-from daniel@digsys.bg) Mime-Version: 1.0 (Apple Message framework v1244.3) Content-Type: text/plain; charset=iso-8859-1 From: Daniel Kalchev In-Reply-To: Date: Sat, 17 Sep 2011 10:34:12 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <72A6ABD6-F6FD-4563-AB3F-6061E3DD9FBF@digsys.bg> References: <1316222526.31565.YahooMailNeo@web121205.mail.ne1.yahoo.com> <4E742E5C.2010900@freebsd.org> To: Rich X-Mailer: Apple Mail (2.1244.3) Cc: Jason Usher , "freebsd-fs@freebsd.org" Subject: Re: ZFS obn FreeBSD hardware model for 48 or 96 sata3 paths... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Sep 2011 07:34:09 -0000 On Sep 17, 2011, at 08:45 , Rich wrote: > The 9201-16e or similar would do fine for 6 Gbit SAS rates if you > don't want hardware RAID. >=20 > To get full bandwidth SATA 3 from 48/96 drives, that's 750 MB/s * 8/10 > (8 data bytes per 10 bytes transmitted raw - SATA 3 does an 8b10b > encoding) ~ 600 MB/s * 48/96 =3D 28800/57600 MB/s >=20 There is not single magnetic drive on the market that can saturate SATA2 = (300 Mbps), yet. Most can't match even SATA1 (150 MBps). You don't need = that much dedicated bandwidth for drives. If you intend to have 48/96 SSDs, then that is another story, but then I = am doubtful a "PC" architecture can handle that much data either. If you are looking gat IOPS rather than raw throughput, then by all = means consider (more) SSDs. You may also consider using 2.5" SAS drives = that will be much more compact and less power hungry. The LSI2008 = controllers will manage both SATA and SAS drives (at the same time, in = the same zpool). Memory is much more expensive than SSDs for L2ARC and if your workload = permits it (lots of repeated small reads), larger L2ARC will help a lot. = It will also help if you have huge spool or if you enable dedup etc. = Just populate as much RAM as the server can handle and then add L2ARC = (read-optimized). Daniel=