Date: Tue, 11 Mar 2014 11:08:35 -0500 From: Danny Schales <dan@LaTech.edu> To: freebsd-stable@freebsd.org Subject: Re: ZFS UNMAP performance Message-ID: <531F3503.8090403@LaTech.edu> In-Reply-To: <CAFHbX1JghVaQ3xM2OZ6jpWOs6tn=Z8epd8Om3NW_CDnZHPSdng@mail.gmail.com> References: <531F2BA0.6000105@LaTech.edu> <CAFHbX1JghVaQ3xM2OZ6jpWOs6tn=Z8epd8Om3NW_CDnZHPSdng@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On 03/11/2014 10:48, Tom Evans wrote: > On Tue, Mar 11, 2014 at 3:28 PM, Danny Schales <dan@latech.edu> wrote: >> I'm seeing very slow performance with certain operations on a ZFS >> filesystem built on ISCSI LUN's on a 10.0 system (new ISCSI >> implementation). The issue appears to be with BIO_DELETE operations. >> Monitoring the system with gstat shows expected times for read and write >> operations, but deletes are in the multiple hundreds of milliseconds >> under normal operation. Destroying a snapshot sends the times to >> astronomical levels. sysctl says the system is using UNMAP for deletes: >> >> kern.cam.da.0.delete_method: UNMAP >> >> I searched and found where Oracle issued a performance alert for Solaris >> 11.1 where ZFS using UNMAP was in use. Here's a link to a blog >> discussing it: >> >> http://schalwad.blogspot.com/2013/12/solaris-111-zfs-write-performance.html >> >> >> Is FreeBSD also impacted? If so, is there a fix or a workaround? > > FreeBSD is affected the same way Solaris is. IMHO, neither are > affected, it is merely a consequence of using a device with a poor > TRIM/UNMAP algorithm. > > You can trivially tell ZFS to not use TRIM, or tweak the tunings to > suit your devices. I'm afraid I don't know what they all mean > precisely.. > > vfs.zfs.vdev.trim_on_init: 1 > vfs.zfs.vdev.trim_max_bytes: 2147483648 > vfs.zfs.vdev.trim_max_pending: 64 > vfs.zfs.trim.enabled: 1 > vfs.zfs.trim.txg_delay: 32 > vfs.zfs.trim.timeout: 30 > vfs.zfs.trim.max_interval: 1 > > Of course, if you disable TRIM you will end up with new (but > different) poor performance characteristics. Probably. The backend is an ISCSI LUN from a SAN (Dell Compellent). From research, it seems the SAN *does* support SCSI UNMAP requests for deletes, but the performance is horrible. I don't know if this is a FreeBSD issue or a Compellent issue. I haven't seen the problem with other devices, but I don't think anything else is using UNMAP (yet). Danny [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlMfNQMACgkQymYKbvs0SzE50gCeOxYTP+f4ZUWSRMxIzLAwelLH QtAAn28Fqtg98S402ew+flQ+Jq2i8Ckk =vHZk -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?531F3503.8090403>