From owner-freebsd-fs@FreeBSD.ORG Thu Feb 19 19:26:36 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D2971C7 for ; Thu, 19 Feb 2015 19:26:36 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDE5760F for ; Thu, 19 Feb 2015 19:26:35 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3kp5WX1hFmzK9 for ; Thu, 19 Feb 2015 20:26:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1424373988; x=1426965989; bh=wRO xir7Vx59q2sEljGcHhqkOh0xc2LzUPyUe1FzPRWQ=; b=jdj8KZWlS/MTtkjExFH k7kp9hdXFdeJFM5VXqrhYpYU3SuiAZiKyKqnIVwIjpTO+sVgZgLqA9N55dZ/ITcY 9S3tjNxvDFQhQjYSnVoWs2qe8SbJ7OiKuRSqPgNiwh1ujzcrg8NS3iPnAFqbEgST t2u31M/I+IEZYOuOI61fEsNk= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id r2f7IgQKJXZN for ; Thu, 19 Feb 2015 20:26:28 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP for ; Thu, 19 Feb 2015 20:26:27 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by mildred.ijs.si (Postfix) with ESMTP id 3kp5WR6jMzzVZ for ; Thu, 19 Feb 2015 20:26:27 +0100 (CET) Received: from neli.ijs.si ([2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Thu, 19 Feb 2015 20:26:27 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 19 Feb 2015 20:26:27 +0100 From: Mark Martinec To: freebsd-fs@freebsd.org Subject: Re: Creating zpool on NVMe Disks takes forever Organization: J. Stefan Institute In-Reply-To: <54E61C18.8070101@fuckner.net> References: <54E5BB12.3060707@fuckner.net> <54E5CF6E.5030906@multiplay.co.uk> <54E5D574.8020406@fuckner.net> <54E5F19A.3080804@kateley.com> <54E61C18.8070101@fuckner.net> Message-ID: X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.0.5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Feb 2015 19:26:36 -0000 >>>> On 02/19/2015 12:56 PM, Steven Hartland wrote: >>>>> Disable trim on init: >>>>> sysctl vfs.zfs.vdev.trim_on_init=0 >>>> this fixed it, thx! >>>> >>>> but why does it take >8h to trim 2x 400GB? Or is trimming handled >>>> differently on NVMe than on HDD/SSD? >>> TRIM/UNMAP is simply an SATA/SCSI command that is sent to a device. >>> What the device does when it gets that command is up to the >>> controller >>> and firmware on the device. The default of vfs.zfs.vdev.trim_on_init = 1 can be counterproductive in case of enabling encryption on a device. The FreeBSD handbook for example states: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/disks-encrypting.html > 4. Create the New File System > > Next, format the device with the UFS file system and mount it > on an existing mount point: > # dd if=/dev/random of=/dev/da2.eli bs=1m > # newfs /dev/da2.eli > # mount /dev/da2.eli /private So after waiting for hours to populate the disk surface with random bytes, creating a ZFS pool out of it can ditch all the effort and possibly rewrite it with zeros before creating the pool. It should be documented to turn vfs.zfs.vdev.trim_on_init off in the above case. Mark