From owner-svn-src-all@FreeBSD.ORG Sun Sep 7 17:16:00 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AD8B1DA; Sun, 7 Sep 2014 17:16:00 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 21EFB114D; Sun, 7 Sep 2014 17:15:59 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 0884620E7088A; Sun, 7 Sep 2014 17:15:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 3FA6F20E70885; Sun, 7 Sep 2014 17:15:57 +0000 (UTC) Message-ID: <620AD65419CE4565ADBFDD9967C9FD91@multiplay.co.uk> From: "Steven Hartland" To: "Slawa Olhovchenkov" References: <201310230954.r9N9sxAZ011750@svn.freebsd.org> <20140907160149.GC72244@zxy.spb.ru> Subject: Re: svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys Date: Sun, 7 Sep 2014 18:16:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Sep 2014 17:16:00 -0000 ----- Original Message ----- From: "Slawa Olhovchenkov" To: "Steven Hartland" Cc: ; ; Sent: Sunday, September 07, 2014 5:01 PM Subject: Re: svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys > On Wed, Oct 23, 2013 at 09:54:59AM +0000, Steven Hartland wrote: > >> Author: smh >> Date: Wed Oct 23 09:54:58 2013 >> New Revision: 256956 >> URL: http://svnweb.freebsd.org/changeset/base/256956 >> >> Log: >> Improve ZFS N-way mirror read performance by using load and locality >> information. >> >> The existing algorithm selects a preferred leaf vdev based on offset of the zio >> request modulo the number of members in the mirror. It assumes the devices are >> of equal performance and that spreading the requests randomly over both drives >> will be sufficient to saturate them. In practice this results in the leaf vdevs >> being under utilized. >> >> The new algorithm takes into the following additional factors: >> * Load of the vdevs (number outstanding I/O requests) >> * The locality of last queued I/O vs the new I/O request. >> >> Within the locality calculation additional knowledge about the underlying vdev >> is considered such as; is the device backing the vdev a rotating media device. >> >> This results in performance increases across the board as well as significant >> increases for predominantly streaming loads and for configurations which don't >> have evenly performing devices. >> >> The following are results from a setup with 3 Way Mirror with 2 x HD's and >> 1 x SSD from a basic test running multiple parrallel dd's. >> >> With pre-fetch disabled (vfs.zfs.prefetch_disable=1): >> >> == Stripe Balanced (default) == >> Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s >> == Load Balanced (zfslinux) == >> Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s >> == Load Balanced (locality freebsd) == >> Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s >> >> With pre-fetch enabled (vfs.zfs.prefetch_disable=0): >> >> == Stripe Balanced (default) == >> Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s >> == Load Balanced (zfslinux) == >> Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s >> == Load Balanced (locality freebsd) == >> Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s >> >> In addition to the performance changes the code was also restructured, with >> the help of Justin Gibbs, to provide a more logical flow which also ensures >> vdevs loads are only calculated from the set of valid candidates. >> >> The following additional sysctls where added to allow the administrator >> to tune the behaviour of the load algorithm: >> * vfs.zfs.vdev.mirror.rotating_inc >> * vfs.zfs.vdev.mirror.rotating_seek_inc >> * vfs.zfs.vdev.mirror.rotating_seek_offset >> * vfs.zfs.vdev.mirror.non_rotating_inc >> * vfs.zfs.vdev.mirror.non_rotating_seek_inc >> >> These changes where based on work started by the zfsonlinux developers: >> https://github.com/zfsonlinux/zfs/pull/1487 >> >> Reviewed by: gibbs, mav, will >> MFC after: 2 weeks >> Sponsored by: Multiplay > > No MFC yet. Grr not seen any MFC reminder :( Will send a request to re@ for permission along with r260713. Regards Steve