Date: Sun, 14 Feb 2010 11:13:55 -0800 From: Artem Belevich <fbsdlist@src.cx> To: Jonathan Belson <jon@witchspace.com> Cc: freebsd-stable@freebsd.org Subject: Re: More zfs benchmarks Message-ID: <ed91d4a81002141113m323eb9eyef40f7cbd6427bc5@mail.gmail.com> In-Reply-To: <EC10D2E9-04C7-4976-B377-62E79D9BE47A@witchspace.com> References: <EC10D2E9-04C7-4976-B377-62E79D9BE47A@witchspace.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Can you check if kstat.zfs.misc.arcstats.memory_throttle_count sysctl increments during your tests? ZFS self-throttles writes if it thinks system is running low on memory. Unfortunately on FreeBSD the 'free' list is a *very* conservative indication of available memory so ZFS often starts throttling before it's really needed. With only 2GB in the system, that's probably what slows you down. The code is in arc_memory_throttle() in sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c, if anyone's curious. --Artem On Sun, Feb 14, 2010 at 9:28 AM, Jonathan Belson <jon@witchspace.com> wrote= : > Hiya > > After reading some earlier threads about zfs performance, I decided to te= st my own server. =A0I found the results rather surprising... > > The machine is a Dell SC440, dual core 2GHz E2180, 2GB of RAM and ICH7 SA= TA300 controller. =A0There are three Hitachi 500GB drives (HDP725050GLA360)= in a raidz1 configuration (version 13). =A0I'm running amd64 7.2-STABLE fr= om 14th Jan. > > > First of all, I tried creating a 200MB file on / (the only non-zfs partit= ion): > > # dd if=3D/dev/zero of=3D/root/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 6.158355 secs (34053769 bytes/sec) > > # dd if=3D/dev/zero of=3D/root/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 5.423107 secs (38670674 bytes/sec) > > # dd if=3D/dev/zero of=3D/root/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 6.113258 secs (34304982 bytes/sec) > > > Next, I tried creating a 200MB file on a zfs partition: > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 58.540571 secs (3582391 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 46.867240 secs (4474665 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 21.145221 secs (9917853 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 19.387938 secs (10816787 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 21.378161 secs (9809787 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D200 > 200+0 records in > 200+0 records out > 209715200 bytes transferred in 23.774958 secs (8820844 bytes/sec) > > Ouch! =A0Ignoring the first result, that's still over three times slower = than the non-zfs test. > > > With a 2GB test file: > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 547.901945 secs (3827605 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 595.052017 secs (3524317 bytes/sec) > > # dd if=3D/dev/zero of=3D/tank/test/zerofile.000 bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 517.326470 secs (4053827 bytes/sec) > > Even worse :-( > > > Reading 2GB from a raw device: > > dd if=3D/dev/ad4s1a of=3D/dev/null bs=3D1M count=3D2000 > 1024+0 records in > 1024+0 records out > 1073741824 bytes transferred in 13.914145 secs (77169084 bytes/sec) > > > Reading 2GB from a zfs partition (unmounting each time): > > dd if=3D/tank/test/zerofile.000 of=3D/dev/null bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 29.905155 secs (70126772 bytes/sec) > > dd if=3D/tank/test/zerofile.000 of=3D/dev/null bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 32.557361 secs (64414066 bytes/sec) > > dd if=3D/tank/test/zerofile.000 of=3D/dev/null bs=3D1M count=3D2000 > 2000+0 records in > 2000+0 records out > 2097152000 bytes transferred in 34.137874 secs (61431828 bytes/sec) > > For reading, there seems to be much less of a disparity in performance. > > I notice that one drive is on atapci0 and the other two are on atapci1, b= ut surely it wouldn't make this much of a difference to write speeds? > > Cheers, > > --Jon > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ed91d4a81002141113m323eb9eyef40f7cbd6427bc5>