Date: Mon, 25 Jan 2010 10:32:19 +0200 From: Dan Naumov <dan.naumov@gmail.com> To: Bob Friesenhahn <bfriesen@simple.dallas.tx.us> Cc: freebsd-fs@freebsd.org, Alexander Motin <mav@freebsd.org>, Jason Edwards <sub.mesa@gmail.com>, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org>, freebsd-questions@freebsd.org Subject: Re: 8.0-RELEASE/amd64 - full ZFS install - low read and write disk performance Message-ID: <cf9b1ee01001250032s24bf9f55r7f83d88d0ce03645@mail.gmail.com> In-Reply-To: <cf9b1ee01001242334x75cf0a2ajcf5fe83aa88c4983@mail.gmail.com> References: <883b2dc51001240905r4cfbf830i3b9b400969ac261b@mail.gmail.com> <1264368182.00211075.1264355402@10.7.7.3> <4B5CC167.5010604@FreeBSD.org> <cf9b1ee01001241614x2ccb818at7631a58cfb143153@mail.gmail.com> <alpine.GSO.2.01.1001242315000.17824@freddy.simplesystems.org> <cf9b1ee01001242334x75cf0a2ajcf5fe83aa88c4983@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 25, 2010 at 9:34 AM, Dan Naumov <dan.naumov@gmail.com> wrote: > On Mon, Jan 25, 2010 at 7:33 AM, Bob Friesenhahn > <bfriesen@simple.dallas.tx.us> wrote: >> On Mon, 25 Jan 2010, Dan Naumov wrote: >>> >>> I've checked with the manufacturer and it seems that the Sil3124 in >>> this NAS is indeed a PCI card. More info on the card in question is >>> available at >>> http://green-pcs.co.uk/2009/01/28/tranquil-bbs2-those-pci-cards/ >>> I have the card described later on the page, the one with 4 SATA ports >>> and no eSATA. Alright, so it being PCI is probably a bottleneck in >>> some ways, but that still doesn't explain the performance THAT bad, >>> considering that same hardware, same disks, same disk controller push >>> over 65mb/s in both reads and writes in Win2008. And agian, I am >>> pretty sure that I've had "close to expected" results when I was >> >> The slow PCI bus and this card look like the bottleneck to me. Remember = that >> your Win2008 tests were with just one disk, your zfs performance with ju= st >> one disk was similar to Win2008, and your zfs performance with a mirror = was >> just under 1/2 that. >> >> I don't think that your performance results are necessarily out of line = for >> the hardware you are using. >> >> On an old Sun SPARC workstation with retrofitted 15K RPM drives on Ultra= -160 >> SCSI channel, I see a zfs mirror write performance of 67,317KB/second an= d a >> read performance of 124,347KB/second. =A0The drives themselves are capab= le of >> 100MB/second range performance. Similar to yourself, I see 1/2 the write >> performance due to bandwidth limitations. >> >> Bob > > There is lots of very sweet irony in my particular situiation. > Initially I was planning to use a single X25-M 80gb SSD in the > motherboard sata port for the actual OS installation as well as to > dedicate 50gb of it to a become a designaed L2ARC vdev for my ZFS > mirrors. The SSD attached to the motherboard port would be recognized > only as a SATA150 device for some reason, but I was still seeing > 150mb/s throughput and sub 0.1 ms latencies on that disk simply > because of how crazy good the X25-M's are. However I ended up having > very bad issues with the Icydock 2,5" to 3,5" converter jacket I was > using to keep/fit the SSD in the system and it would randomly drop > write IO on heavy load due to bad connectors. Having finally figured > out the cause of my OS installations to the SSD going belly up during > applying updates, I decided to move the SSD to my desktop and use it > there instead, additionally thinking that my perhaps my idea of the > SSD was crazy overkill for what I need the system to do. Ironically > now that I am seeing how horrible the performance is when I am > operating on the mirror through this PCI card, I realize that > actually, my idea was pretty bloody brilliant, I just didn't really > know why at the time. > > An L2ARC device on the motherboard port would really help me with > random read IO, but to work around the utterly poor write performance, > I would also need a dedicaled SLOG ZIL device. The catch is that while > L2ARC devices and be removed from the pool at will (should the device > up and die all of a sudden), the dedicated ZILs cannot and currently a > "missing" ZIL device will render the pool it's included in be unable > to import and become inaccessible. There is some work happening in > Solaris to implement removing SLOGs from a pool, but that work hasn't > yet found it's way in FreeBSD yet. > > > - Sincerely, > Dan Naumov OK final question: if/when I go about adding more disks to the system and want redundancy, am I right in thinking that: ZFS pool of disk1+disk2 mirror + disk3+disk4 mirror (a la RAID10) would completely murder my write and read performance even way below the current 28mb/s / 50mb/s I am seeing with 2 disks on that PCI controller and that in order to have the least negative impact, I should simply have 2 independent mirrors in 2 independent pools (with the 5th disk slot in the NAS given to a non-redundant single disk running off the one available SATA port on the motherboard)? - Sincerely, Dan Naumov
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?cf9b1ee01001250032s24bf9f55r7f83d88d0ce03645>