Date: Tue, 20 Jun 2017 12:06:29 -0700 From: Freddie Cash <fjwcash@gmail.com> To: "Caza, Aaron" <Aaron.Caza@ca.weatherford.com> Cc: Karl Denninger <karl@denninger.net>, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: Re: FreeBSD 11.1 Beta 2 ZFS performance degradation on SSDs Message-ID: <CAOjFWZ6q5tfoxCZBeWKm8z7-eSm=wdt7KrZfQ5Sj1QUHyb88MQ@mail.gmail.com> In-Reply-To: <b7350cca59624e91abee6697aaf9e1b6@DM2PR58MB013.032d.mgd.msft.net> References: <b7350cca59624e91abee6697aaf9e1b6@DM2PR58MB013.032d.mgd.msft.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jun 20, 2017 at 11:50 AM, Caza, Aaron <Aaron.Caza@ca.weatherford.co= m > wrote: > I've observed this performance degradation on 6 different hardware system= s > using 4 differents SSDS (2x Intel 510 120GB, 2x Intel 520 120GB, 2x Intel > 540 120GB, 2x Samsung 850 Pro SSDs) on FreeBSD10.3 RELEASE, FreeBSD 10.3 > RELEASEp6, FreeBSD 10.3RELEASEp19, FreeBSD 10-Stable, FreeBSD11.0 RELEASE= , > FreeBSD 11-Stable and now FreeBSD11.1 Beta 2. This latest testing I'm no= t > doing much in the way of writing - only logging the output of the 'dd' > command along with 'zfs-stats -a' and 'uptime' to go along with it once a= n > hour. Ran for ~20hrs before performance drop kicked in though why it > happens is inexplicable as this server isn't doing anything other than > running this test hourly. > > I have a FreeBSD9.0 system using 2x Intel 520 120GB SSDs that doesn't > exhibit this performance degradation, maintaining ~400MB/s speeds even > after many days of uptime. This is using the GEOM ELI layer to provide 4= k > sector emulation for the mirrored zpool as I previously described. > =E2=80=8BI don't remember if this has been mentioned yet in either of your = threads on this, but what is the output of this command on all your poorly performing systems: sysctl =E2=80=8Bvfs.zfs.trim.enabled If it's set to 1 (the default), set it to 0 and re-run your tests. ZFS Trim support for SSDs was added to 10.0, so any system running FreeBSD 10+ will show a performance drop after awhile when the trim function kicks in to clear out deleted/unused blocks. Especially if it's an SSD that can't run Trim commands in parallel. You can look at the various ZFS trim-related stats to see what it's doing: sysctl vfs.zfs | grep trim --=20 Freddie Cash fjwcash@gmail.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOjFWZ6q5tfoxCZBeWKm8z7-eSm=wdt7KrZfQ5Sj1QUHyb88MQ>