Date: Tue, 3 Nov 2015 10:12:02 +0100 From: Nicolas Gilles <nicolas.gilles@gmail.com> To: freebsd-stable@freebsd.org Subject: Re: ZFS, SSDs, and TRIM performance Message-ID: <CANwv7WuiDapMGfsjKHDY-ZHngKN%2BoMsxVf9HLhXyQAT%2BV_gYqQ@mail.gmail.com> In-Reply-To: <563263ED.1070402@multiplay.co.uk> References: <449F8F4D-425D-46B5-BB9C-BE5A0CD11C55@smkelly.org> <563263ED.1070402@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
Not sure about the Samsung XS1715, but lots of SSDs seem to suck at large amounts of TRIM in general leading a "let me pause everything for a while" symptom. In fact I think there is work in ZFS to make TRIMs work better, and to throttle them in case large amounts are freed to avoid this kind of starvation. -- Nicolas On Thu, Oct 29, 2015 at 7:22 PM, Steven Hartland <killing@multiplay.co.uk> wrote: > If you running NVMe, are you running a version which has this: > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D285767 > > I'm pretty sure 10.2 does have that, so you should be good, but best to > check. > > Other questions: > 1. What does "gstat -d -p" show during the stalls? > 2. Do you have any other zfs tuning in place? > > On 29/10/2015 16:54, Sean Kelly wrote: >> >> Me again. I have a new issue and I=E2=80=99m not sure if it is hardware = or >> software. I have nine servers running 10.2-RELEASE-p5 with Dell OEM=E2= =80=99d >> Samsung XS1715 NVMe SSDs. They are paired up in a single mirrored zpool = on >> each server. They perform great most of the time. However, I have a prob= lem >> when ZFS fires off TRIMs. Not during vdev creation, but like if I delete= a >> 20GB snapshot. >> >> If I destroy a 20GB snapshot or delete large files, ZFS fires off tons o= f >> TRIMs to the disks. I can see the kstat.zfs.misc.zio_trim.success and >> kstat.zfs.misc.zio_trim.bytes sysctls skyrocket. While this is happening= , >> any synchronous writes seem to block. For example, we=E2=80=99re running= PostgreSQL >> which does fsync()s all the time. While these TRIMs happen, Postgres jus= t >> hangs on writes. This causes reads to block due to lock contention as we= ll. >> >> If I change sync=3Ddisabled on my tank/pgsql dataset while this is >> happening, it unblocks for the most part. But obviously this is not an i= deal >> way to run PostgreSQL. >> >> I=E2=80=99m working with my vendor to get some Intel SSDs to test, but a= ny ideas >> if this could somehow be a software issue? Or does the Samsung XS1715 ju= st >> suck at TRIM and SYNC? >> >> We=E2=80=99re thinking of just setting the vfs.zfs.trim.enabled=3D0 tuna= ble for now >> since WAL segment turnover actually causes TRIM operations a lot, but >> unfortunately this is a reboot. But disabling TRIM does seem to fix the >> issue on other servers I=E2=80=99ve tested with the same hardware config= . >> > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANwv7WuiDapMGfsjKHDY-ZHngKN%2BoMsxVf9HLhXyQAT%2BV_gYqQ>