From owner-freebsd-fs@FreeBSD.ORG Thu Feb 19 13:33:30 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A265AEC for ; Thu, 19 Feb 2015 13:33:30 +0000 (UTC) Received: from mail-we0-f180.google.com (mail-we0-f180.google.com [74.125.82.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A40146D9 for ; Thu, 19 Feb 2015 13:33:29 +0000 (UTC) Received: by wesx3 with SMTP id x3so7327703wes.6 for ; Thu, 19 Feb 2015 05:33:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=16xfbCWYxVJ2oCQBz6a8iTrE1WMBiDprSQR3iJ1zbVE=; b=m3f5DpOqBgypadMPlkHMtE3yltQpfyqwtEFWaB4HnIu2yrM9eE22EU1tDcW4hc+LM+ 5021nZd95QM/IYc4uqGRa6h2WmHojiN0eiv0Dr2C/a7+PTING22c69tbN3/fl9C+XMwd uUd1XwlGx6+not4NzsQuZ/XxyhbD12JfIcQOUXxC1dlxSQ+J4Zhb3ikvRsEG8tNqg56p wqDm5Q3578YKU27J7CJ/G01GcWLnFudzVLI5JyNIFRCtNlGyKv/oLJ8IPOk7Cbn3dNv0 wfIWlkX+FBvium2b57abYfMOk74kUVnLyx17vGIgK0M0WF4jn4tqi3BtSbd7P78isDqS RE8A== X-Gm-Message-State: ALoCoQltu1RDF2eoJ3CFkLDknoYMIlr9/1Bpv02l92smv7H5aVzFmufEySSgDMF3Un5YKegaNlRy X-Received: by 10.180.21.133 with SMTP id v5mr6631147wie.27.1424352807779; Thu, 19 Feb 2015 05:33:27 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id kr5sm32486778wjc.1.2015.02.19.05.33.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Feb 2015 05:33:26 -0800 (PST) Message-ID: <54E5E624.2070301@multiplay.co.uk> Date: Thu, 19 Feb 2015 13:33:24 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: Creating zpool on NVMe Disks takes forever References: <54E5BB12.3060707@fuckner.net> <54E5CF6E.5030906@multiplay.co.uk> <54E5D574.8020406@fuckner.net> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Feb 2015 13:33:30 -0000 On 19/02/2015 13:02, Tom Evans wrote: > On Thu, Feb 19, 2015 at 12:22 PM, Michael Fuckner wrote: >> On 02/19/2015 12:56 PM, Steven Hartland wrote: >>> Disable trim on init: >>> sysctl vfs.zfs.vdev.trim_on_init=0 >> this fixed it, thx! >> >> but why does it take >8h to trim 2x 400GB? Or is trimming handled >> differently on NVMe than on HDD/SSD? > TRIM/UNMAP is simply an SATA/SCSI command that is sent to a device. > What the device does when it gets that command is up to the controller > and firmware on the device. > Technically is not TRIM / UNMAP its BIO_DELETE which is translated by the relevant device driver to the format the device understands. For SCSI this can be TRIM, UNMAP, WS or ZERO, for ATA this can be CFA_ERASE or TRIM and for NVMe this is a Deallocate. One reason why it might be slow is I don't see NVMe configuring geom d_delmaxsize, which if the case will force the "TRIM" to be split into single sector deallocation requests. Regards Steve