Date: Wed, 07 Jul 2010 16:12:45 +0200 From: Ivan Voras <ivoras@freebsd.org> To: freebsd-performance@freebsd.org Subject: Re: More Controllers != Higher Through Put Message-ID: <i1220f$4th$1@dough.gmane.org> In-Reply-To: <4C33BDCC.1020004@softhammer.net> References: <4C33BDCC.1020004@softhammer.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/07/10 01:35, Stephen Sanders wrote: > I'm wondering if anyone has heard of this. > > I've a system with a 3ware 9650 servicing 4 7200RPM Segate 1TB drives > and the motherboard servicing 2 7200 RPM Segate 1TB drives. So far so good. > The 4 disk array is RAID 6 while the 2 disk array is RAID 1. The drives > should deliver about 100MB/s. Ok, so you've reduced the 4-drive array's write performance nearly to equivalent of 2 drives and the 2-drive array to 1 drive. It should be even worse for random IOs. Since FreeBSD doesn't support RAID-6 I guess you are using hardware RAID? For the 2-disk RAID-1 : you are probably using software RAID, right? (on-board "SATA RAID" controllers usually are just software implementations). > 1. The most the 4 disk array is developing is 250MB/s write performance This is too much. It almost looks like something is caching what shouldn't be cached. How did you get this result? I'd expect less than 200 MB/s sequential writes on a 4-drive RAID-6 with 100 MB/s drives. > while the 2 disk array is coming in at 90MB/s write performance. This is as expected - write performance of any size RAID-1 is equivalent of 1 drive or less. > The 4 disk array seems slow. Nope - the contrary should be true. It looks like you are doing something you shouldn't if you get that much performance, or your test is overly simplistic (e.g. you're testing cache). > 2. Attempting to write to both arrays simultaneously causes the rate on > the 4 disk array to drop to 150MB/s and the 2 disk array drops to 60MB/s Are you running on an Atom CPU? What kind of system are you using? > I'd expect the 4 disk array should look more like 300+MB/s while the 2 > disk array is about right. No, you cannot get 300 MB/s from simple RAID6 of 4 drives in any direction. Think about it: all the data needs to be a) written as-is to 2 of the drives, then b) parity/ECCs calculated and c) the same amount of data written to 2 more drives. You cannot get write performance of more than 2 drives equivalent in this scheme, and will probably be worse. For reads, only if your RAID controller is very, very smart (meaning: it probably isn't), you can recover some performance by using this parity/ECC data to reconstruct more data than is read from the two "plain" drives. I think ZFS does this in a limited way. > I don't get why there should be a 'coupling' between the rates on > separate controllers. This is the only thing which is puzzling a bit. I > The system is running FreeBSD 8.0, has 16GB of RAM in the system, and > the test program is using O_DIRECT for writes in order to avoid the page > daemon. You should use some benchmark which knows how to deal with OS cache, for example bonnie++ from the ports. Use a benchmark with random IO to see just how horrible your RAID-6 performance will be for random writes.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?i1220f$4th$1>