Date: Sun, 11 Aug 2013 00:11:25 -0400 From: J David <j.david.lists@gmail.com> To: freebsd-questions@freebsd.org Subject: Re: Terrible disk performance with LSI / FreeBSD 9.2-RC1 Message-ID: <CABXB=RTafdevhRY_B7eWtEkwkbHKDBobWSuiNtT9kg5u_atyJg@mail.gmail.com> In-Reply-To: <5202C558.3010305@fjl.co.uk> References: <CABXB=RSRnB41yjq5Qcbiz-JCRssNwx2AatJ2Dn%2BHhuD9GaBh%2Bw@mail.gmail.com> <CAEhBLvg7ZUMja5zpFm2UQBXESW-0fL9L7EatR2aasstXd8ALHA@mail.gmail.com> <CABXB=RRvp0BLURq7M9iBb5anqaGsrvXeA1WmAroNji6bZP8p4w@mail.gmail.com> <5202C558.3010305@fjl.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
To follow up on this issue, at one point the stats were down to this:
extended device statistics
device r/s w/s kr/s kw/s qlen svc_t %b
da0 0.0 0.0 0.0 0.0 0 0.0 0
da1 0.0 0.0 0.0 0.0 0 0.0 0
da2 127.9 0.0 202.3 0.0 1 47.5 100
da3 125.9 0.0 189.3 0.0 1 43.1 97
da4 127.9 0.0 189.8 0.0 1 45.8 100
da5 128.9 0.0 206.3 0.0 0 42.5 99
da6 127.9 0.0 202.3 0.0 1 46.2 98
da7 0.0 249.7 0.0 334.2 10 39.5 100
At some point, I figured out that 125 random iops is pretty much the
limit for 7200 RPM SATA drives. So mostly what we're looking at here
is the resilver of a raidz2 is the pathological worst case. Lesson
learned; raidz2 is just really not viable without some kind of sort on
the resilver operations. Wish I understood ZFS well enough to do
something about that, but research suggests the problem is
non-trivial. :(
There also seems to be a separate ZFS issue related to having a very
large number of snapshots (e.g. hourly for several months on a couple
of filesystems). Some combination of the OS updates we've been doing
trying to get this machine to 9.2-RC1 and deleting a ton of snapshots.
It would be nice to know which it was; I guess we'll find out in a
few months.
So it seems like the combination of these two issues is mostly what
is/was plaguing us.
Thanks!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABXB=RTafdevhRY_B7eWtEkwkbHKDBobWSuiNtT9kg5u_atyJg>