Date: Tue, 28 May 2002 11:45:25 -0600 From: Scott Long <scott_long@btc.adaptec.com> To: "Jan L. Peterson" <jlp@softhome.net> Cc: Phil Rosenthal <winter@villaweb.net>, freebsd-stable@freebsd.org Subject: Re: Hardware RAID vs vinum Message-ID: <20020528174525.GA88979@hollin.btc.adaptec.com> In-Reply-To: <20020528165020.8DC37AC44F@mail.flipdog.com> References: <004701c20530$68e98a20$0700a8c0@zoom> <20020528165020.8DC37AC44F@mail.flipdog.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Ok, time to trow my anecdotes in. As a warning, I work for Adaptec and have been very close to both its hardware and software RAID projects. Not all RAID controllers are created equal. Many factors play a role: cache size cache architecture onboard processor type and speed host<->processor communication interface IDE vs SCSI driver architecture (scsi vs block) parity acceleration engine battery backing for the cache failure recovery reliability RAID 0 and 1 are not particularly hard to do. Many software RAID implementations do it fairly well, in fact, and are pretty competetive with their low-end hardware-based counterparts in both speed and CPU utilization. Only on the higher end controllers can you see a difference, but that difference may not be worth the $800+ that they cost. Conversely, some RAID controllers on the market today can't even do RAID 0 or 1 well, and aren't worth the trouble. One good measure of hardware vs software RAID lies in looking at RAID 5 performance. RAID 5 will always be much faster with a hardware solution than a software solution. Calculating parity is a very repetitive task, and your CPU has better things to do with its time. Furthermore, if you happen to be writing chuncks of data that are smaller than the stripe size, the operation turns into a read-modify-write that is much better offloaded in hardware. A hardware-based solution can be even faster if it has an on-board parity calculation engine that is separate from the cpu. Another good benchmark is transaction thoroughput. This is often dictated by the cache size, queue depth of the card, and the caching algorithm used on the card. In software, you are limited by the max thoroughput of the underlying disk controller, plus the overhead of computing the data layout. You can work around this by using multiple controllers, but this really only helps if you are using SCSI controllers. IDE, being a single transaction protocol (except for the IBM tagged queueing drives, but they still aren't as fast as SCSI by a long shot), will never allow a high transaction rate for software RAID. Hardware RAID, on the other hand, can achieve very high tranaction rates if designed well. A controller that uses a block interface, large cache, and a top-down cache design can perform very well while having very low CPU utilization. SCSI-based controllers are going to perform the best, but even IDE-based controllers can do well, since the cache can insulate the host from the slowness of IDE. Sequential access benchmarks are typically poor comparisions. Unless your application is something like a streaming video server, these benchmarks do not even closely model a real-world scenario. In reality, all they are testing is the PCI and memory bandwidth of your system, neither of which stresses hardware or software RAID implementations. The exception is that if a hardware controller perfoms poorly on these tests, it's pretty much guaranteed to be a dog. Finally, there is reliability. The whole purpose of RAID is to protect your data from drive failures. No matter how well designed and coded a software RAID subsystem is, drive failures are disruptive and can cause unexpected behavior. The problem may not even manifest itself in the raid subsystem, but rather in the disk subsystem. Hardware RAID, on the other hand, is an integrated package and is usually very well tested. On most controllers, a drive failure may even go unnoticed by the host OS, with the controller quickly detecting the error and transparently doing the corrective actions. So the real answer to hardware vs. software RAID is, what are your requirements? You have to weigh cost, reliability, and the myriad of performance needs. Scott On Tue, May 28, 2002 at 10:50:20AM -0600, Jan L. Peterson wrote: > My experience may be contrary to others. > > My particular application got much much better performance with > software RAID (vinum) than with hardware RAID (HP 3-Si card a.k.a. AMI > MegaRAID). > > The application in question had about 16 Windows boxes accessing an SMB > filesystem (Samba on the FBSD box). These clients would be all over > the disk, accessing hundreds or thousands of different files at > different locations on the filesystem every second. Mostly reading, > but some writing as well. > > I got excellent performance out of the hardware raid with only about > three or four clients running, but when all sixteen were pounding on > the disk, throughput (measured with iostat/systat) was abysmal. > > Moving the disks to a Symetrics controller and vinum resulted in at > least tripple the performance. > > I accredited it to the fact that the hardware controller had a small > cache (32M) vs. the system main memory (1G), and that the system could > better schedule reads and writes as it knew more about where the data > was on the various disks. > > I guess what I'm saying is don't simply assume that hardware RAID is > good or bad... it's very dependent on your application. We are very > satisfied with the hardware RAID on a different box with different usage > patterns. > > -jan- > -- > Jan L. Peterson > <jlp@softhome.net> > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-stable" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020528174525.GA88979>