From owner-freebsd-stable@FreeBSD.ORG Fri Aug 20 10:30:09 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33BA21065693 for ; Fri, 20 Aug 2010 10:30:09 +0000 (UTC) (envelope-from rebehn@ant.uni-bremen.de) Received: from antsrv1.ant.uni-bremen.de (antsrv1.ant.uni-bremen.de [134.102.176.16]) by mx1.freebsd.org (Postfix) with ESMTP id ED3868FC0C for ; Fri, 20 Aug 2010 10:30:08 +0000 (UTC) Received: from bremerhaven.ant.uni-bremen.de ([134.102.176.10]) by antsrv1.ant.uni-bremen.de with esmtp (Exim 4.68 (FreeBSD)) (envelope-from ) id 1OmOkz-000Kvw-QK; Fri, 20 Aug 2010 12:23:29 +0200 From: Heinrich Rebehn Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 20 Aug 2010 12:30:07 +0200 To: freebsd-stable@freebsd.org Message-Id: Mime-Version: 1.0 (Apple Message framework v1081) X-Mailer: Apple Mail (2.1081) Subject: ZFS performance question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Aug 2010 10:30:09 -0000 Hi all, After setting up our new server (Intel Q9550 CPU, 8GB RAM, 4 x = ST31000340NS) i did a bonnie++ benchmark on the zfs raidz that i created = on 4 partitions on the 4 disks. root@antsrv4 [/data/nocompression] # bonnie++ -u root -d . Using uid:0, gid:0. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- = --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- = --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = /sec %CP antsrv4.ant.uni 16G 94 81 151823 39 100780 27 303 99 239853 30 = 148.3 4 Latency 4733ms 6051ms 8173ms 37504us 917ms = 1030ms Version 1.96 ------Sequential Create------ --------Random = Create-------- antsrv4.ant.uni-bre -Create-- --Read--- -Delete-- -Create-- --Read--- = -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = /sec %CP 16 24183 97 +++++ +++ 21715 98 17209 98 +++++ +++ = 21291 97 Latency 19779us 128us 162us 39671us 38us = 77us = 1.96,1.96,antsrv4.ant.uni-bremen.de,1,1282303232,16G,,94,81,151823,39,1007= 80,27,303,99,239853,30,148.3,4,16,,,,,24183,97,+++++,+++,21715,98,17209,98= ,+++++,+++,21291,97, = 733ms,6051ms,8173ms,37504us,917ms,1030ms,19779us,128us,162us,39671us,38us,= 77us I am somewhat concerned about the numbers for per-char-output and = per-char-input. In fact, i have never before seen that low numbers in a = bonnie test. Using a single disk with UFS yields about 6 times as much. I know that this is not crucial for a normal file server, but i want to = rule out any configuration errors. Is this normal for ZFS?. Disks are accessed using ahci. I did not = attempt any tuning yet. BTW: Running OpenSolaris on the same hardware yields 110306 for = per-char-write and 94698 for per-char-read. --Heinrich Heinrich Rebehn University of Bremen Physics / Electrical and Electronics Engineering - Department of Telecommunications - Phone : +49/421/218-62394 Fax : -3341