Date: Fri, 14 Oct 2011 08:23:09 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: freebsd-fs@freebsd.org Subject: Re: ZFS/compression/performance Message-ID: <20111014152309.GA75162@icarus.home.lan> In-Reply-To: <20111012165126.GA26562@icarus.home.lan> References: <alpine.BSF.2.00.1110111710210.12895@Elmer.dco.penx.com> <j73vl7$v02$1@dough.gmane.org> <CAHHaOua7zwp4DCgcU4bSFG2d-Kz-j_ovV7YiSMqmy700wFjBHg@mail.gmail.com> <20111012165126.GA26562@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Oct 12, 2011 at 09:51:26AM -0700, Jeremy Chadwick wrote: > On Wed, Oct 12, 2011 at 09:37:10AM -0700, David Brodbeck wrote: > > On Wed, Oct 12, 2011 at 5:02 AM, Johannes Totz <jtotz@imperial.ac.uk> wrote: > > > > > I just did a simple write test yesterday: > > > > > > 1) 6 MB/sec for gzip, 1.36x ratio > > > 2) 34 MB/sec for lzjb, 1.23x ratio > > > > > > I'll stick with lzjb. It's good enough to get rid of most of the > > > redundancy and speed is acceptable. > > > > > > > That's what we use on our text-heavy filesystems on our OpenSolaris server. > > (We work with large text corpora, so we have hundreds of gigabytes of pure > > text.) My benchmarks showed the performance hit for reads is nonexistent > > when viewed over NFS, and the performance hit for writes is relatively > > small...plus we don't write to that filesystem much. We see about 1.5x > > compression overall, with a little over 2x on some datasets that are > > particularly compressible. > > That might be the case on OpenSolaris but the performance hit on > FreeBSD RELENG_8 is very high -- enough that enabling compression (using > the defaults) causes stalls when I/O occurs (easily noticeable across > SSH; characters are delayed/stalled (not buffered)), etc.. > > The last time I tried it on RELENG_8 was right after ZFSv28 was MFC'd. > If things have improved I can try again (I don't remember seeing any > commits that could affect this), or if people really think changing the > compression model to lzjb will help. Follow-up: Tried this out on RELENG_8 (2011/09/28) with ZFS v28 filesystems: zfs create -o mountpoint=/test -o compression=lzjb data/test cd /test dd if=/dev/urandom of=testfile bs=16k Then in another SSH session to the machine, held down a single key at my bash prompt. Every 3 seconds, like clockwork, SSH I/O would stall/drop for about ~0.4 seconds. CPU in the system is an Intel C2D E8400 (3GHz), with ULE scheduler in use. Then I did this: rm testfile zfs set compression=none data/test dd if=/dev/urandom of=testfile bs=16k And repeated the procedure: no stalls. Then I tried using gzip-1: even worse. The stalls were about 3-4 full seconds long. I imagine this is expected since gzip is much slower than lzjb. The important thing to note here is that the entire machine appears to spin hard when ZFS compression is in use. Even things like switching virtual consoles (Alt-Fx) lock up until the compression bits do whatever they need to do. I will try to find a Solaris 10 test system at work later today and tinker with compression there to see if it behaves the same, but given what Bob described below I'm doubting it will: http://lists.freebsd.org/pipermail/freebsd-fs/2011-October/012726.html Just an FYI for folks considering use of ZFS compression on RELENG_8 as of this writing. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111014152309.GA75162>