From owner-freebsd-stable@FreeBSD.ORG Tue Mar 11 15:48:58 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C236A3FE for ; Tue, 11 Mar 2014 15:48:58 +0000 (UTC) Received: from mail-lb0-x22f.google.com (mail-lb0-x22f.google.com [IPv6:2a00:1450:4010:c04::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4F57C79E for ; Tue, 11 Mar 2014 15:48:58 +0000 (UTC) Received: by mail-lb0-f175.google.com with SMTP id w7so5679642lbi.6 for ; Tue, 11 Mar 2014 08:48:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Rn9kMdrNdk5wcVYKDBXfCPMDrdmSHTnU0sXZpW6NwO8=; b=NTKrwjwkReLIL3q+VQ+zbcVvlxTxAkaVEPm2n3WskfeikMVFY0cqIiBQkYsV0GydOu oGKiTui0BhBfcCT8JtZyxaD14yKQTKxaTsO+1OtdXparul/Ma2BFrLSkgztXiz5sw5tn fMbIwNeq89ZeINz1AlbXj9JpCvTAKZP4WZBYrrU/117MoAgb3BfSthNV6n3tWb4PU6K8 AIDH0+K1abgdJVX70/VwbMow8S8Z6+5IYqgSI/lW6dnJ89BVaA+FlpLt57WbttgcEL6b 1TKTMrI3APcDefbTPHVJOpERpbHGXGVLIMtwAjj1umUvPwl5f8W61SPoZ24mTQhhUJLT WpWA== MIME-Version: 1.0 X-Received: by 10.112.170.234 with SMTP id ap10mr26408492lbc.23.1394552936438; Tue, 11 Mar 2014 08:48:56 -0700 (PDT) Received: by 10.112.129.164 with HTTP; Tue, 11 Mar 2014 08:48:56 -0700 (PDT) In-Reply-To: <531F2BA0.6000105@LaTech.edu> References: <531F2BA0.6000105@LaTech.edu> Date: Tue, 11 Mar 2014 15:48:56 +0000 Message-ID: Subject: Re: ZFS UNMAP performance From: Tom Evans To: Danny Schales Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Mar 2014 15:48:58 -0000 On Tue, Mar 11, 2014 at 3:28 PM, Danny Schales wrote: > I'm seeing very slow performance with certain operations on a ZFS > filesystem built on ISCSI LUN's on a 10.0 system (new ISCSI > implementation). The issue appears to be with BIO_DELETE operations. > Monitoring the system with gstat shows expected times for read and write > operations, but deletes are in the multiple hundreds of milliseconds > under normal operation. Destroying a snapshot sends the times to > astronomical levels. sysctl says the system is using UNMAP for deletes: > > kern.cam.da.0.delete_method: UNMAP > > I searched and found where Oracle issued a performance alert for Solaris > 11.1 where ZFS using UNMAP was in use. Here's a link to a blog > discussing it: > > http://schalwad.blogspot.com/2013/12/solaris-111-zfs-write-performance.html > > > Is FreeBSD also impacted? If so, is there a fix or a workaround? FreeBSD is affected the same way Solaris is. IMHO, neither are affected, it is merely a consequence of using a device with a poor TRIM/UNMAP algorithm. You can trivially tell ZFS to not use TRIM, or tweak the tunings to suit your devices. I'm afraid I don't know what they all mean precisely.. vfs.zfs.vdev.trim_on_init: 1 vfs.zfs.vdev.trim_max_bytes: 2147483648 vfs.zfs.vdev.trim_max_pending: 64 vfs.zfs.trim.enabled: 1 vfs.zfs.trim.txg_delay: 32 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.max_interval: 1 Of course, if you disable TRIM you will end up with new (but different) poor performance characteristics. Probably. Cheers Tom