Date: Wed, 25 Oct 2006 23:37:00 +0200 From: Ivan Voras <ivoras@fer.hr> To: freebsd-stable@freebsd.org Subject: Re: Gmirror performanc Message-ID: <eholdl$nkb$1@sea.gmane.org> In-Reply-To: <20061025211153.GA58556@gvr.gvr.org> References: <20061025132455.GA52157@gvr.gvr.org> <200610251350.k9PDo442086250@lurza.secnetix.de> <20061025211153.GA58556@gvr.gvr.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Guido van Rooij wrote: > But a single-threaded process accessing a large file should also be > able to see a speed increase. I really do not see why a split or roundrobin > approach to a mirror would get only half the performance of a raw access > to a non-mirror. Somehow there must a limit of 128KB of outstanding requests. > That limit is software imposed and should IMHO scale with the number of > spindles in a mirror... I've recently toyed with creating RAID arrays from 2 USB memory sticks (for fun :) ), and this is what I've found out (I may create a web page with the results if I find the time): - diskinfo -t doesn't work on small drives (needs at least 1 GB drive, maybe larger) - maximum read data rate of my USB sticks is ~~ 7.5 MB/s (with dd), but it can be achieved only with large requests (64 KB/s, as seen by iostat) - the speed declines when using smaller requests, to ~~ 4 MB/s with 4 KB requests (CPU utilisation is still ~~ 0 - the slowdown is in the controller or the drive) - gmirror split balance algorithm works as advertised - requests larger than split size are delegated to multiple devices. This has the expected effect of achieving double throughput (15 MB/s) with large enough requests. - there's a (somewhat expected but still surprising) interaction with MAXPHYS (maximum IO size) in the kernel. MAXPHYS is 128 KB, which means requests from TO gmirror (and other GEOM devices) will be <= 128 KB, which gmirror will split in two, so we arrive to maximum size of 64 KB that's requested from the hardware. In case of a four-way mirror, maximum size to hardware would be 32 KB, etc. I don't know if this is what's slowing gmirror down, but it might, since current drives are apparently best tuned for large requests. It probably makes sense to use gmirror split size of around FS block size (16 KB) to make use of split access but not to make too small requests. I've done a round of testing with gstripe and it behaves as expected (and doesn't have the side-effect of splitting requests).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?eholdl$nkb$1>