Date: Sun, 7 Sep 2014 18:16:01 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Slawa Olhovchenkov" <slw@zxy.spb.ru> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys Message-ID: <620AD65419CE4565ADBFDD9967C9FD91@multiplay.co.uk> References: <201310230954.r9N9sxAZ011750@svn.freebsd.org> <20140907160149.GC72244@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message ----- From: "Slawa Olhovchenkov" <slw@zxy.spb.ru> To: "Steven Hartland" <smh@FreeBSD.org> Cc: <src-committers@freebsd.org>; <svn-src-all@freebsd.org>; <svn-src-head@freebsd.org> Sent: Sunday, September 07, 2014 5:01 PM Subject: Re: svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys > On Wed, Oct 23, 2013 at 09:54:59AM +0000, Steven Hartland wrote: > >> Author: smh >> Date: Wed Oct 23 09:54:58 2013 >> New Revision: 256956 >> URL: http://svnweb.freebsd.org/changeset/base/256956 >> >> Log: >> Improve ZFS N-way mirror read performance by using load and locality >> information. >> >> The existing algorithm selects a preferred leaf vdev based on offset of the zio >> request modulo the number of members in the mirror. It assumes the devices are >> of equal performance and that spreading the requests randomly over both drives >> will be sufficient to saturate them. In practice this results in the leaf vdevs >> being under utilized. >> >> The new algorithm takes into the following additional factors: >> * Load of the vdevs (number outstanding I/O requests) >> * The locality of last queued I/O vs the new I/O request. >> >> Within the locality calculation additional knowledge about the underlying vdev >> is considered such as; is the device backing the vdev a rotating media device. >> >> This results in performance increases across the board as well as significant >> increases for predominantly streaming loads and for configurations which don't >> have evenly performing devices. >> >> The following are results from a setup with 3 Way Mirror with 2 x HD's and >> 1 x SSD from a basic test running multiple parrallel dd's. >> >> With pre-fetch disabled (vfs.zfs.prefetch_disable=1): >> >> == Stripe Balanced (default) == >> Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s >> == Load Balanced (zfslinux) == >> Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s >> == Load Balanced (locality freebsd) == >> Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s >> >> With pre-fetch enabled (vfs.zfs.prefetch_disable=0): >> >> == Stripe Balanced (default) == >> Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s >> == Load Balanced (zfslinux) == >> Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s >> == Load Balanced (locality freebsd) == >> Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s >> >> In addition to the performance changes the code was also restructured, with >> the help of Justin Gibbs, to provide a more logical flow which also ensures >> vdevs loads are only calculated from the set of valid candidates. >> >> The following additional sysctls where added to allow the administrator >> to tune the behaviour of the load algorithm: >> * vfs.zfs.vdev.mirror.rotating_inc >> * vfs.zfs.vdev.mirror.rotating_seek_inc >> * vfs.zfs.vdev.mirror.rotating_seek_offset >> * vfs.zfs.vdev.mirror.non_rotating_inc >> * vfs.zfs.vdev.mirror.non_rotating_seek_inc >> >> These changes where based on work started by the zfsonlinux developers: >> https://github.com/zfsonlinux/zfs/pull/1487 >> >> Reviewed by: gibbs, mav, will >> MFC after: 2 weeks >> Sponsored by: Multiplay > > No MFC yet. Grr not seen any MFC reminder :( Will send a request to re@ for permission along with r260713. Regards Steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?620AD65419CE4565ADBFDD9967C9FD91>