From owner-freebsd-fs@freebsd.org Thu Apr 28 05:48:50 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B05EB1F6F9 for ; Thu, 28 Apr 2016 05:48:50 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C74B617B3; Thu, 28 Apr 2016 05:48:49 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 8637A20028C; Thu, 28 Apr 2016 07:48:47 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 761BB406ADE; Thu, 28 Apr 2016 07:48:47 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id E2765406ADB; Thu, 28 Apr 2016 07:48:46 +0200 (CEST) Received: from arc.aei.uni-hannover.de ([130.75.117.1]) by intranet.aei.uni-hannover.de (IBM Domino Release 9.0.1FP5) with ESMTP id 2016042807484630-55750 ; Thu, 28 Apr 2016 07:48:46 +0200 Date: Thu, 28 Apr 2016 07:48:46 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Gary Palmer Cc: freebsd-fs@freebsd.org Subject: Re: zfs on nvme: gnop breaks pool, zfs gets stuck Message-Id: <20160428074846.a28da3113294f1edc6ed9ce6@aei.mpg.de> In-Reply-To: <20160427141436.GA60370@in-addr.com> References: <20160427152244.ff36ff74ae64c1f86fdc960a@aei.mpg.de> <20160427141436.GA60370@in-addr.com> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 9.0.1FP5|November 22, 2015) at 28/04/2016 07:48:46, Serialize by Router on intranet/aei-hannover(Release 9.0.1FP5|November 22, 2015) at 28/04/2016 07:48:46, Serialize complete at 28/04/2016 07:48:46 X-TNEFEvaluated: 1 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2016.4.28.53616 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_2000_2999 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, NO_URI_HTTPS 0, REFERENCES 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __C230066_P5 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __FRAUD_MONEY_BIG_COIN 0, __FRAUD_MONEY_BIG_COIN_DIG 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __LINES_OF_YELLING 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __REFERENCES 0, __SANE_MSGID 0, __STOCK_PHRASE_7 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_WWW 0, __URI_NS ' X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 05:48:50 -0000 On Wed, 27 Apr 2016 15:14:36 +0100 Gary Palmer wrote about Re: zfs on nvme: gnop breaks pool, zfs gets stuck: GP> vfs.zfs.min_auto_ashift GP> GP> which lets you manage the ashift on a new pool without having to try GP> the gnop trick I just tried this, and it appears to work fine: --- root@storage:~ # sysctl vfs.zfs.min_auto_ashift vfs.zfs.min_auto_ashift: 12 root@storage:~ # zpool status pool: data state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM data ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 gpt/disk0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 gpt/disk2 ONLINE 0 0 0 gpt/disk3 ONLINE 0 0 0 gpt/disk4 ONLINE 0 0 0 gpt/disk5 ONLINE 0 0 0 gpt/disk6 ONLINE 0 0 0 gpt/disk7 ONLINE 0 0 0 gpt/disk8 ONLINE 0 0 0 gpt/disk9 ONLINE 0 0 0 gpt/disk10 ONLINE 0 0 0 gpt/disk11 ONLINE 0 0 0 errors: No known data errors pool: flash state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM flash ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gpt/flash0 ONLINE 0 0 0 gpt/flash1 ONLINE 0 0 0 gpt/flash2 ONLINE 0 0 0 errors: No known data errors root@storage:~ # zdb | grep ashift ashift: 12 ashift: 12 root@storage:~ # zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT data 65T 1.88M 65.0T - 0% 0% 1.00x ONLINE - flash 1.39T 800K 1.39T - 0% 0% 1.00x ONLINE - --- I still wonder why the gnop workaround went so terribly wrong. Anyway, thanks again for pointing out this new sysctl to me! And for the record: this is what I get with a simple linear write test: --- root@storage:~ # dd if=/dev/zero of=/flash/Z bs=1024k count=10000 10000+0 records in 10000+0 records out 10485760000 bytes transferred in 3.912829 secs (2679840997 bytes/sec) --- I guess I won't complain... cu Gerrit