Date: Thu, 29 Oct 2015 11:54:38 -0500 From: Sean Kelly <smkelly@smkelly.org> To: FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Subject: ZFS, SSDs, and TRIM performance Message-ID: <449F8F4D-425D-46B5-BB9C-BE5A0CD11C55@smkelly.org>
next in thread | raw e-mail | index | archive | help
Me again. I have a new issue and I=E2=80=99m not sure if it is hardware = or software. I have nine servers running 10.2-RELEASE-p5 with Dell = OEM=E2=80=99d Samsung XS1715 NVMe SSDs. They are paired up in a single = mirrored zpool on each server. They perform great most of the time. = However, I have a problem when ZFS fires off TRIMs. Not during vdev = creation, but like if I delete a 20GB snapshot. If I destroy a 20GB snapshot or delete large files, ZFS fires off tons = of TRIMs to the disks. I can see the kstat.zfs.misc.zio_trim.success and = kstat.zfs.misc.zio_trim.bytes sysctls skyrocket. While this is = happening, any synchronous writes seem to block. For example, we=E2=80=99r= e running PostgreSQL which does fsync()s all the time. While these TRIMs = happen, Postgres just hangs on writes. This causes reads to block due to = lock contention as well. If I change sync=3Ddisabled on my tank/pgsql dataset while this is = happening, it unblocks for the most part. But obviously this is not an = ideal way to run PostgreSQL. I=E2=80=99m working with my vendor to get some Intel SSDs to test, but = any ideas if this could somehow be a software issue? Or does the Samsung = XS1715 just suck at TRIM and SYNC? We=E2=80=99re thinking of just setting the vfs.zfs.trim.enabled=3D0 = tunable for now since WAL segment turnover actually causes TRIM = operations a lot, but unfortunately this is a reboot. But disabling TRIM = does seem to fix the issue on other servers I=E2=80=99ve tested with the = same hardware config. --=20 Sean Kelly smkelly@smkelly.org http://smkelly.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?449F8F4D-425D-46B5-BB9C-BE5A0CD11C55>