Date: Fri, 14 Oct 2011 18:04:46 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Jeremy Chadwick" <freebsd@jdc.parodius.com>, <freebsd-fs@freebsd.org> Subject: Re: ZFS/compression/performance Message-ID: <480AA0893BC748759163FE8D677EF0D8@multiplay.co.uk> References: <alpine.BSF.2.00.1110111710210.12895@Elmer.dco.penx.com><j73vl7$v02$1@dough.gmane.org><CAHHaOua7zwp4DCgcU4bSFG2d-Kz-j_ovV7YiSMqmy700wFjBHg@mail.gmail.com><20111012165126.GA26562@icarus.home.lan> <20111014152309.GA75162@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message ----- From: "Jeremy Chadwick" <freebsd@jdc.parodius.com> To: <freebsd-fs@freebsd.org> Sent: Friday, October 14, 2011 4:23 PM Subject: Re: ZFS/compression/performance > On Wed, Oct 12, 2011 at 09:51:26AM -0700, Jeremy Chadwick wrote: >> On Wed, Oct 12, 2011 at 09:37:10AM -0700, David Brodbeck wrote: >> > On Wed, Oct 12, 2011 at 5:02 AM, Johannes Totz <jtotz@imperial.ac.uk> wrote: >> > >> > > I just did a simple write test yesterday: >> > > >> > > 1) 6 MB/sec for gzip, 1.36x ratio >> > > 2) 34 MB/sec for lzjb, 1.23x ratio >> > > >> > > I'll stick with lzjb. It's good enough to get rid of most of the >> > > redundancy and speed is acceptable. >> > > >> > >> > That's what we use on our text-heavy filesystems on our OpenSolaris server. >> > (We work with large text corpora, so we have hundreds of gigabytes of pure >> > text.) My benchmarks showed the performance hit for reads is nonexistent >> > when viewed over NFS, and the performance hit for writes is relatively >> > small...plus we don't write to that filesystem much. We see about 1.5x >> > compression overall, with a little over 2x on some datasets that are >> > particularly compressible. >> >> That might be the case on OpenSolaris but the performance hit on >> FreeBSD RELENG_8 is very high -- enough that enabling compression (using >> the defaults) causes stalls when I/O occurs (easily noticeable across >> SSH; characters are delayed/stalled (not buffered)), etc.. >> >> The last time I tried it on RELENG_8 was right after ZFSv28 was MFC'd. >> If things have improved I can try again (I don't remember seeing any >> commits that could affect this), or if people really think changing the >> compression model to lzjb will help. > > Follow-up: > > Tried this out on RELENG_8 (2011/09/28) with ZFS v28 filesystems: > > zfs create -o mountpoint=/test -o compression=lzjb data/test > cd /test > dd if=/dev/urandom of=testfile bs=16k > > Then in another SSH session to the machine, held down a single key at my > bash prompt. Every 3 seconds, like clockwork, SSH I/O would stall/drop > for about ~0.4 seconds. CPU in the system is an Intel C2D E8400 (3GHz), > with ULE scheduler in use. > > Then I did this: > > rm testfile > zfs set compression=none data/test > dd if=/dev/urandom of=testfile bs=16k > > And repeated the procedure: no stalls. > > Then I tried using gzip-1: even worse. The stalls were about 3-4 full > seconds long. I imagine this is expected since gzip is much slower than > lzjb. > > The important thing to note here is that the entire machine appears to > spin hard when ZFS compression is in use. Even things like switching > virtual consoles (Alt-Fx) lock up until the compression bits do whatever > they need to do. > > I will try to find a Solaris 10 test system at work later today and > tinker with compression there to see if it behaves the same, but given > what Bob described below I'm doubting it will: > > http://lists.freebsd.org/pipermail/freebsd-fs/2011-October/012726.html > > Just an FYI for folks considering use of ZFS compression on RELENG_8 as > of this writing. We use compression heavily here on 8.2-RELEASE and cant say we've noticed such a behaviour will do some testing to see if we can reproduce here as well. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?480AA0893BC748759163FE8D677EF0D8>