From owner-freebsd-stable@FreeBSD.ORG Sun Feb 14 23:25:12 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0878A10656CE for ; Sun, 14 Feb 2010 23:25:12 +0000 (UTC) (envelope-from boydjd@jbip.net) Received: from mail-fx0-f228.google.com (mail-fx0-f228.google.com [209.85.220.228]) by mx1.freebsd.org (Postfix) with ESMTP id 7ECBD8FC1B for ; Sun, 14 Feb 2010 23:25:11 +0000 (UTC) Received: by fxm28 with SMTP id 28so46325fxm.31 for ; Sun, 14 Feb 2010 15:25:10 -0800 (PST) MIME-Version: 1.0 Received: by 10.239.161.130 with SMTP id h2mr452311hbd.114.1266189909253; Sun, 14 Feb 2010 15:25:09 -0800 (PST) In-Reply-To: <33c6b0bc1002141523q72eed878qf900230026ff34b3@mail.gmail.com> References: <33c6b0bc1002141315y6671e91dife7e80b926cb9d0b@mail.gmail.com> <33c6b0bc1002141512t29c2650cp2f2dba866b9d7415@mail.gmail.com> <33c6b0bc1002141523q72eed878qf900230026ff34b3@mail.gmail.com> From: Joshua Boyd Date: Sun, 14 Feb 2010 18:24:49 -0500 Message-ID: <33c6b0bc1002141524r68aac008o4a2c34a644e8a46f@mail.gmail.com> To: freebsd-stable@freebsd.org, Jonathan Belson Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: More zfs benchmarks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2010 23:25:12 -0000 Here's my bonnie++ results: foghornleghorn# bonnie++ -s 8192 -d. -n64 -uroot Using uid:0, gid:0. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP foghornleghorn.r 8G 184 99 191627 43 150484 35 416 98 388853 49 103.3 3 Latency 47480us 1645ms 1545ms 53943us 186ms 2449ms Version 1.96 ------Sequential Create------ --------Random Create-------- foghornleghorn.res. -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 64 22256 80 37111 99 17478 88 22686 81 37208 99 15053 90 Latency 28364us 336us 274us 27001us 321us 80123us 1.96,1.96,foghornleghorn.res.openband.net ,1,1266194600,8G,,184,99,191627,43,150484,35,416,98,388853,49,103.3,3,64,,,,,22256,80,37111,99,17478,88,22686,81,37208,99,15053,90,47480us,1645ms,1545ms,53943us,186ms,2449ms,28364us,336us,274us,27001us,321us,80123us On Sun, Feb 14, 2010 at 6:23 PM, Joshua Boyd wrote: > > > On Sun, Feb 14, 2010 at 6:12 PM, Joshua Boyd wrote: > >> >> >> On Sun, Feb 14, 2010 at 5:58 PM, Jonathan Belson wrote: >> >>> On 14 Feb 2010, at 21:15, Joshua Boyd wrote: >>> >>> > Repeated the same tests on my AMD64 dual core 4GB system with 5 HD103SI >>> 1T >>> > drives in raidz1 on a Supermicro PCI-E controller, running 8-STABLE. >>> >>> [ snip results ] >>> >>> I was hoping I'd get something closer to these figures... >>> >>> > Here are my relevant settings: >>> > >>> > vfs.zfs.prefetch_disable=0 >>> > vfs.zfs.zil_disable="1" >>> >>> I already had prefetch disabled, but retrying with zil disabled made no >>> difference. >>> >> >> That setting actually enables prefetch ... I have 4GB of RAM, so I felt >> safe turning it on. >> >> >>> >>> What is your arc_min and arc_max set to? >>> >> >> System Memory: >> Physical RAM: 4015 MB >> Free Memory : 2130 MB >> >> ARC Size: >> Current Size: 783 MB (arcsize) >> Target Size (Adaptive): 783 MB (c) >> Min Size (Hard Limit): 101 MB (zfs_arc_min) >> Max Size (Hard Limit): 808 MB (zfs_arc_max) >> >> ARC Size Breakdown: >> Most Recently Used Cache Size: 97% 764 MB (p) >> Most Frequently Used Cache Size: 2% 18 MB (c-p) >> >> ARC Efficency: >> Cache Access Total: 148383 >> Cache Hit Ratio: 66% 98434 [Defined State for buffer] >> Cache Miss Ratio: 33% 49949 [Undefined State for >> Buffer] >> REAL Hit Ratio: 65% 96744 [MRU/MFU Hits Only] >> >> Data Demand Efficiency: 99% >> Data Prefetch Efficiency: 0% >> >> CACHE HITS BY CACHE LIST: >> Anon: 1% 1515 [ New >> Customer, First Cache Hit ] >> Most Recently Used: 28% 27931 (mru) [ Return >> Customer ] >> Most Frequently Used: 69% 68813 (mfu) [ Frequent >> Customer ] >> Most Recently Used Ghost: 0% 175 (mru_ghost) [ Return >> Customer Evicted, Now Back ] >> Most Frequently Used Ghost: 0% 0 (mfu_ghost) [ Frequent >> Customer Evicted, Now Back ] >> CACHE HITS BY DATA TYPE: >> Demand Data: 12% 12369 >> Prefetch Data: 0% 0 >> Demand Metadata: 85% 84375 >> Prefetch Metadata: 1% 1690 >> CACHE MISSES BY DATA TYPE: >> Demand Data: 0% 6 >> Prefetch Data: 96% 47994 >> Demand Metadata: 3% 1580 >> Prefetch Metadata: 0% 369 >> --------------------------------------------- >> >> >> >>> >>> > >>> > On Sun, Feb 14, 2010 at 3:34 PM, Michael Loftis >>> wrote: >>> >>> >>> >> You really need to test with at least 4GB of data, else you're just >>> testing >>> >> caching speeds on writing. Use a test suite like bonnie++ and you'll >>> see >>> >>> I'd expect to get more than 4MB/s if I was just measuring cache speed :-) >>> >>> Cheers, >>> >>> --Jon >>> >>> >> >> >> -- >> Joshua Boyd >> JBipNet >> >> E-mail: boydjd@jbip.net >> >> http://www.jbip.net >> > > > > -- > Joshua Boyd > JBipNet > > E-mail: boydjd@jbip.net > > http://www.jbip.net > -- Joshua Boyd JBipNet E-mail: boydjd@jbip.net Cell: (513) 375-0157 http://www.jbip.net