Date: Fri, 20 Aug 2010 14:12:57 +0200 From: Heinrich Rebehn <rebehn@ant.uni-bremen.de> To: Olivier Smedts <olivier@gid0.org> Cc: freebsd-stable@freebsd.org Subject: Re: ZFS performance question Message-ID: <EB9930BA-9CD1-4981-AEE9-F85DC534B8F2@ant.uni-bremen.de> In-Reply-To: <AANLkTi=a6mG2aJ3GezcPZws2b4V1hryOiUSnVh1sL_qt@mail.gmail.com> References: <EDFF110C-91D3-40DA-A581-9885EAAE3753@ant.uni-bremen.de> <AANLkTi=a6mG2aJ3GezcPZws2b4V1hryOiUSnVh1sL_qt@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 20.08.2010, at 12:48, Olivier Smedts wrote: > 2010/8/20 Heinrich Rebehn <rebehn@ant.uni-bremen.de>: >> Hi all, >>=20 >> After setting up our new server (Intel Q9550 CPU, 8GB RAM, 4 x = ST31000340NS) i did a bonnie++ benchmark on the zfs raidz that i created = on 4 partitions on the 4 disks. >>=20 >> root@antsrv4 [/data/nocompression] # bonnie++ -u root -d . >> Using uid:0, gid:0. >> Writing a byte at a time...done >> Writing intelligently...done >> Rewriting...done >> Reading a byte at a time...done >> Reading intelligently...done >> start 'em...done...done...done...done...done... >> Create files in sequential order...done. >> Stat files in sequential order...done. >> Delete files in sequential order...done. >> Create files in random order...done. >> Stat files in random order...done. >> Delete files in random order...done. >> Version 1.96 ------Sequential Output------ --Sequential Input- = --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- = --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = /sec %CP >> antsrv4.ant.uni 16G 94 81 151823 39 100780 27 303 99 239853 = 30 148.3 4 >> Latency 4733ms 6051ms 8173ms 37504us 917ms = 1030ms >> Version 1.96 ------Sequential Create------ --------Random = Create-------- >> antsrv4.ant.uni-bre -Create-- --Read--- -Delete-- -Create-- --Read--- = -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = /sec %CP >> 16 24183 97 +++++ +++ 21715 98 17209 98 +++++ +++ = 21291 97 >> Latency 19779us 128us 162us 39671us 38us = 77us >> = 1.96,1.96,antsrv4.ant.uni-bremen.de,1,1282303232,16G,,94,81,151823,39,1007= 80,27,303,99,239853,30,148.3,4,16,,,,,24183,97,+++++,+++,21715,98,17209,98= ,+++++,+++,21291,97, = 733ms,6051ms,8173ms,37504us,917ms,1030ms,19779us,128us,162us,39671us,38us,= 77us >>=20 >>=20 >> I am somewhat concerned about the numbers for per-char-output and = per-char-input. In fact, i have never before seen that low numbers in a = bonnie test. Using a single disk with UFS yields about 6 times as much. >>=20 >>=20 >> I know that this is not crucial for a normal file server, but i want = to rule out any configuration errors. >>=20 >> Is this normal for ZFS?. Disks are accessed using ahci. I did not = attempt any tuning yet. >=20 > Can you try without AHCI ? > = http://www.phoronix.com/scan.php?page=3Darticle&item=3Dfreebsd_zfs_cam&num= =3D5 Hmm, that looks impressive, but: root@antsrv4 [/data/nocompression] # bonnie++ -u root -d . Using uid:0, gid:0. [snip] Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- = --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- = --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = /sec %CP antsrv4.ant.uni 16G 107 99 152759 39 118765 33 304 99 227534 29 = 137.4 4 Latency 216ms 6110ms 6436ms 108ms 717ms = 784ms Version 1.96 ------Sequential Create------ --------Random = Create-------- antsrv4.ant.uni-bre -Create-- --Read--- -Delete-- -Create-- --Read--- = -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = /sec %CP 16 23264 94 +++++ +++ 20417 96 24430 97 +++++ +++ = 20866 97 Latency 20142us 138us 3506us 19454us 128us = 77us = 1.96,1.96,antsrv4.ant.uni-bremen.de,1,1282317527,16G,,107,99,152759,39,118= 765,33,304,99,227534,29,137.4,4,16,,,,,23264,94,+++++,+++,20417,96,24430,9= 7,+++++,+++,20866,97,216ms,6110ms,6436ms,108ms,717ms,784ms,20142us,138us,3= 506us,19454us,128us,77us Almost no difference. Only the create rate increased. root@antsrv4 [/data/nocompression] # kldstat Id Refs Address Size Name 1 9 0xffffffff80100000 d6aa98 kernel 2 1 0xffffffff80e6b000 19eb18 zfs.ko 3 2 0xffffffff8100a000 3868 opensolaris.ko root@antsrv4 [/data/nocompression] # ls /dev/ada* ls: /dev/ada*: No such file or directory -Heinrich >=20 >> BTW: Running OpenSolaris on the same hardware yields 110306 for = per-char-write and 94698 for per-char-read. >>=20 >=20 >=20 >=20 > --=20 > Olivier Smedts _ > ASCII ribbon campaign ( ) > e-mail: olivier@gid0.org - against HTML email & vCards X > www: http://www.gid0.org - against proprietary attachments / \ >=20 > "Il y a seulement 10 sortes de gens dans le monde : > ceux qui comprennent le binaire, > et ceux qui ne le comprennent pas." > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" Heinrich Rebehn University of Bremen Physics / Electrical and Electronics Engineering - Department of Telecommunications - Phone : +49/421/218-62394 Fax : -3341
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?EB9930BA-9CD1-4981-AEE9-F85DC534B8F2>