From owner-freebsd-current@FreeBSD.ORG Thu Sep 13 19:54:46 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C204106566B; Thu, 13 Sep 2012 19:54:46 +0000 (UTC) (envelope-from sendtomatt@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id A9DF98FC08; Thu, 13 Sep 2012 19:54:45 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so4906843pbb.13 for ; Thu, 13 Sep 2012 12:54:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=8dVcIvGOVhEXL1qyZsEruHZgfOoGLVVp9gJQJ5n3Y98=; b=fJbi/Pc4pVmKAU7ETdHvwIwv+ez2+BxSwM0PlWekn//Sj2ue2I+ClAM4Vb8hOMaUX6 I4BBFh+t0jnKKMjgArSxYBBv5kk+xbzXRjeibqW30UumTUPIQBaobumpZ4qTURg8/C+3 id1ktS9vgtGvjCCndlp01voFkKKQoU+mZJVJrcL6ZVkQbGfeFDe/0eV14Gwf7sKF3zRq o9AnP5bWcpV/iSXyCdJGvksS7qpqgqN5VA9u3mSFs62ICcKVDMoLIxJDYWBrxHQWUgTT YOk+yb97OrLu2SQ0f4h5Z4Zvdy1+OGJ0aOMALCA12BppinpiCgEJaBsA/W9NqhXSFMWm te/A== Received: by 10.66.83.129 with SMTP id q1mr458778pay.4.1347566085289; Thu, 13 Sep 2012 12:54:45 -0700 (PDT) Received: from flatline.local (70-36-223-239.dsl.dynamic.sonic.net. [70.36.223.239]) by mx.google.com with ESMTPS id kp3sm13566066pbc.64.2012.09.13.12.54.26 (version=SSLv3 cipher=OTHER); Thu, 13 Sep 2012 12:54:43 -0700 (PDT) Message-ID: <505239E2.1040805@gmail.com> Date: Thu, 13 Sep 2012 12:54:10 -0700 From: matt User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: Garrett Cooper References: <6A0C3360-0A1E-4905-A33E-D6CC590D7A5A@bnc.net> <504E200A.5020604@gmail.com> <504E330A.4090806@FreeBSD.org> <504E9EB0.2040504@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Achim Patzner , Andrey Zonov , freebsd-current@freebsd.org Subject: Re: mfi driver performance X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Sep 2012 19:54:46 -0000 On 09/10/12 19:31, Garrett Cooper wrote: > On Mon, Sep 10, 2012 at 7:15 PM, matt wrote: > > ... > >> mfip was necessary, and allowed smartctl to work with '-d sat' >> >> bonnie++ comparison. Run with no options immediately after system boot. In >> both cases the same disks are used, two Seagate Barracuda 1TB 3G/S (twin >> platter) and a Barracuda 500G 3G/s (single platter) in a zfs triple mirror >> that the system was booted from. All are 7200 RPM drives with 32mb cache, >> and mediocre performance compared to my hitachi 7k3000s or the 15k sas >> cheetahs at work etc. Firmwares were the latest 2108it vs the latest imr_fw >> that work on the 9240/9220/m1015/drake skinny. I wish I had some 6g ssds to >> try! >> >> MPS: >> Version 1.96 ------Sequential Output------ --Sequential Input- --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP >> flatline.local 32G 122 99 71588 24 53293 20 284 90 222157 33 252.6 49 >> Latency 542ms 356ms 914ms 991ms 337ms 271ms >> Version 1.96 ------Sequential Create------ --------Random Create-------- >> flatline.local -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP >> 16 22197 93 9367 27 16821 99 23555 99 +++++ +++ 23717 99 >> Latency 31650us 290ms 869us 23036us 66us 131us >> 1.96,1.96,flatline.local,1,1347322810,32G,,122,99,71588,24,53293,20,284,90,222157,33,252.6,49,16,,,,,22197,93,9367,27,16821,99,23555,99,+++++,+++,23717,99,542ms,356ms,914ms,991ms,337ms,271ms,31650us,290ms,869us,23036us,66us,131us >> >> MFI: >> Version 1.96 ------Sequential Output------ --Sequential Input- --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP >> flatline.local 32G 125 99 71443 24 53177 21 317 99 220280 33 255.3 52 >> Latency 533ms 566ms 1134ms 86565us 357ms 252ms >> Version 1.96 ------Sequential Create------ --------Random Create-------- >> flatline.local -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP >> 16 22347 94 12389 30 16804 100 18729 99 27798 99 5317 99 >> Latency 33818us 233ms 558us 26581us 75us 12319us >> 1.96,1.96,flatline.local,1,1347329123,32G,,125,99,71443,24,53177,21,317,99,220280,33,255.3,52,16,,,,,22347,94,12389,30,16804,100,18729,99,27798,99,5317,99,533ms,566ms,1134ms,86565us,357ms,252ms,33818us,233ms,558us,26581us,75us,12319us >> >> A close race, with some wins for each. Latency on sequential input and >> deleted files per second appear to be interesting salients. >> A lot of the other stuff is back and forth and probably not statistically >> significant (although not much of a sample set :) ). >> >> I tried to control as many variables as possible, but obviously it's one >> controller in one configuration, Your Mileage May Vary. > Try upping the queue depth (hw.mfi.max_cmds); this is controller dependent. > Cheers, > -Garrett > It seems hw.mfi.max_cmds is read only. The performance is pretty close to expected with no nvram or bbu on this card and commodity disks from 1.5 years ago, as far as I'm concerned. I'd love better write performance, but it's probably being held back by the single platter in the mirror when it is writing far from its edge. Is there any way to check the interface speed with an mfisyspd? When I added mfip to my kernel config, the pass devices are all reporting 150MB/S which is incorrect. Matt