Date: Wed, 18 May 2016 11:26:06 +0300 From: Alex Tutubalin <lexa@lexa.ru> To: freebsd-fs@freebsd.org Subject: Re: ZFS performance bottlenecks: CPU or RAM or anything else? Message-ID: <411166e6-239f-0bf2-99df-e177f334270c@lexa.ru> In-Reply-To: <39be913e-32a5-2120-fee5-4521b8b95d80@multiplay.co.uk> References: <8441f4c0-f8d1-f540-b928-7ae60998ba8e@lexa.ru> <f87ec54a-104e-e712-7793-86c37285fdaa@internetx.com> <16e474da-6b20-2e51-9981-3c262eaff350@lexa.ru> <BD7DE274-04EB-4B19-988D-5A6FADC5B51A@digsys.bg> <1e012e43-a49b-6923-3f0a-ee77a5c8fa70@lexa.ru> <86shxgsdzh.fsf@WorkBox.Home> <CAHEMsqZto0wD9Ko4E9YUpYvea4jM0E4f2nC1HkAwcCG=6DfX-A@mail.gmail.com> <f1cc3ee5-c141-b366-83bf-3ee0179381bf@lexa.ru> <39be913e-32a5-2120-fee5-4521b8b95d80@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On 5/18/2016 11:02 AM, Steven Hartland wrote: > My comment was targeted under the assumption of random IOPs workload, > which is typically the case, where each RAIDZ group (vdev) will give > approximately a single drive performance. For a pretty definitive > guide / answer see: > http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/ Thank you for the link. In my workload (single write stream) IOPs count is very low, disk write locality is good (each file most likely to fit in single metaslab), so bandwidth is not limited to single drive bandwidth. My current box (6x 7200rpm HDDs in raidz1) provides about 430 Mb/s write bandwidth over SMB link and about 500Mb/s for local writes. It is ~100 Mb/s per spindle, close enough to expected. I hope, I'll see 2x in bandwidth with 2x spindle count if I do not hit another performance limiter. So, my initial question was 'is there any known raidz performance limiter, like CPU or RAM speed/latency'. > > There's also some useful practical test results here: > https://calomel.org/zfs_raid_speed_capacity.html I've already posted this link in my thread-starting message :) And, yes, there are very strange similarity in both read and write speed in 6x and 10x SSD/raidz2 cases. Unfortunately, this benchmark is not real use case because of: "Since the disk cache can artificially inflate the results we choose to disable drive caches completely using Bonnie++ in synchronous test mode only." Synchronous mode will result in double writes (ZIL, than data), without separate ZIL device ZIL to be written to the main pool. We do not know what will happen with real-life async writes on same hardware. Alex Tutubalin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?411166e6-239f-0bf2-99df-e177f334270c>