Date: Thu, 29 Oct 2015 18:22:37 +0000 From: Steven Hartland <killing@multiplay.co.uk> To: freebsd-stable@freebsd.org Subject: Re: ZFS, SSDs, and TRIM performance Message-ID: <563263ED.1070402@multiplay.co.uk> In-Reply-To: <449F8F4D-425D-46B5-BB9C-BE5A0CD11C55@smkelly.org> References: <449F8F4D-425D-46B5-BB9C-BE5A0CD11C55@smkelly.org>
next in thread | previous in thread | raw e-mail | index | archive | help
If you running NVMe, are you running a version which has this: https://svnweb.freebsd.org/base?view=revision&revision=285767 I'm pretty sure 10.2 does have that, so you should be good, but best to check. Other questions: 1. What does "gstat -d -p" show during the stalls? 2. Do you have any other zfs tuning in place? On 29/10/2015 16:54, Sean Kelly wrote: > Me again. I have a new issue and I’m not sure if it is hardware or software. I have nine servers running 10.2-RELEASE-p5 with Dell OEM’d Samsung XS1715 NVMe SSDs. They are paired up in a single mirrored zpool on each server. They perform great most of the time. However, I have a problem when ZFS fires off TRIMs. Not during vdev creation, but like if I delete a 20GB snapshot. > > If I destroy a 20GB snapshot or delete large files, ZFS fires off tons of TRIMs to the disks. I can see the kstat.zfs.misc.zio_trim.success and kstat.zfs.misc.zio_trim.bytes sysctls skyrocket. While this is happening, any synchronous writes seem to block. For example, we’re running PostgreSQL which does fsync()s all the time. While these TRIMs happen, Postgres just hangs on writes. This causes reads to block due to lock contention as well. > > If I change sync=disabled on my tank/pgsql dataset while this is happening, it unblocks for the most part. But obviously this is not an ideal way to run PostgreSQL. > > I’m working with my vendor to get some Intel SSDs to test, but any ideas if this could somehow be a software issue? Or does the Samsung XS1715 just suck at TRIM and SYNC? > > We’re thinking of just setting the vfs.zfs.trim.enabled=0 tunable for now since WAL segment turnover actually causes TRIM operations a lot, but unfortunately this is a reboot. But disabling TRIM does seem to fix the issue on other servers I’ve tested with the same hardware config. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?563263ED.1070402>