From owner-freebsd-fs@FreeBSD.ORG  Wed Jun  1 08:19:42 2011
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 34503106566C
	for <freebsd-fs@freebsd.org>; Wed,  1 Jun 2011 08:19:42 +0000 (UTC)
	(envelope-from alexander@leidinger.net)
Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de
	[217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id BAB8A8FC34
	for <freebsd-fs@freebsd.org>; Wed,  1 Jun 2011 08:19:41 +0000 (UTC)
Received: from outgoing.leidinger.net (p5B155DF9.dip.t-dialin.net
	[91.21.93.249])
	by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id E7087844010;
	Wed,  1 Jun 2011 10:19:27 +0200 (CEST)
Received: from webmail.leidinger.net (webmail.Leidinger.net
	[IPv6:fd73:10c7:2053:1::3:102])
	by outgoing.leidinger.net (Postfix) with ESMTP id 2A0CC33E0;
	Wed,  1 Jun 2011 10:19:25 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net;
	s=outgoing-alex; t=1306916365;
	bh=shvIaJugg/Lm7k465HqiPoGsF7+wTaOfyLdY0D9P8Gs=;
	h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To:
	MIME-Version:Content-Type:Content-Transfer-Encoding;
	b=fTSAW+82QqTcY8piuQTYcFppW7F7a4eR6rxyxFURnywoWTXH/JPb5W0wvIXGPim0e
	U1yH9tL0GXTMZw7ElgAwXVsNzzeS3Z7BbyxrZoGfy6ghQo5YCRmN2JTj0WNsyfFZbG
	mojwVUVYQhkIc6AyO6YxdTYaqZXDg7OA7lzveoT8cvvyMHgHWkRLNXEFnoJBictgMB
	0C1XTvUamJVfTBMDcLuKkcSO698akyBo57VBpeAf9jtzYp15f4OiG9bQ+leEj52Kos
	e1bmSFrMg85Y1oi8kA+acYXiBfaQq27MUQVJfK0+4/mH1TGiiaBAofA+UN0wFzVJw6
	YUK1mbke9+Zvw==
Received: (from www@localhost)
	by webmail.leidinger.net (8.14.4/8.14.4/Submit) id p518JOs5063679;
	Wed, 1 Jun 2011 10:19:24 +0200 (CEST)
	(envelope-from Alexander@Leidinger.net)
X-Authentication-Warning: webmail.leidinger.net: www set sender to
	Alexander@Leidinger.net using -f
Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by
	webmail.leidinger.net (Horde Framework) with HTTP; Wed, 01 Jun 2011
	10:19:24 +0200
Message-ID: <20110601101924.11334wftbjfrvor0@webmail.leidinger.net>
Date: Wed, 01 Jun 2011 10:19:24 +0200
From: Alexander Leidinger <Alexander@Leidinger.net>
To: Negasi Gerima <negasi@decayingorbits.com>
References: <4DE5D7B1.5040902@decayingorbits.com>
In-Reply-To: <4DE5D7B1.5040902@decayingorbits.com>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=UTF-8;
 DelSp="Yes";
 format="flowed"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.6)
X-EBL-MailScanner-Information: Please contact the ISP for more information
X-EBL-MailScanner-ID: E7087844010.A0CDB
X-EBL-MailScanner: Found to be clean
X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN,
	SpamAssassin (not cached, score=0.054, required 6,
	autolearn=disabled, DKIM_SIGNED 0.10, DKIM_VALID -0.10,
	DKIM_VALID_AU -0.10, TW_ZD 0.08, TW_ZF 0.08)
X-EBL-MailScanner-From: alexander@leidinger.net
X-EBL-MailScanner-Watermark: 1307521168.89268@SQu9F5RP2TtpahR9Iv17BA
X-EBL-Spam-Status: No
Cc: freebsd-fs@freebsd.org
Subject: Re: ZFS on 4K drives - Expanding zpool and adding spares (gnop
 method)
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 01 Jun 2011 08:19:42 -0000

Quoting Negasi Gerima <negasi@decayingorbits.com> (from Wed, 01 Jun  
2011 02:09:53 -0400):

> Hello,
>
> I apologize if this topic is redundant, but I couldn't find an exact  
> answer to my specific question. So here goes.
>
> I recently decided to expand the storage in my home server with a  
> bunch of 1.5Tb Samsung 4K hard drives. Using the guide at the  
> following link I was able to use the "gnop method" in order to align  
> the drives for use with 4K sectors:
>
> http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html
>
> My question to the list is this, now that my new zpool is created  
> and I've made sure the drives are using the correct layout, how  
> would I go about expanding the zpool in the future with additional  
> drives and/or spares? I emailed the author of the article but  
> received a reply that I found terse at best.
>
> I'm assuming that I would have to create a new gnop device for each  
> additional hard drive, and then add those gnop devices to the pool.

The 4k-sector part results in a pool specific config setting. I would  
expect that if you add additional drives (no matter if 4k or not),  
that the pool will continue to use 4k sectors. So I would expect that  
you do not need gnop to expend an existing pool, but I have not  
tested/verified this.

You could test this with md(4) devices where you use gnop for the  
initial creation and no gnop during extending it. When it is extended  
you just run
   zdp <poolname) | grep ashift
and verify that it is set to 12.

> For example, if I had a pool called 'datastore' and three additional  
> drives /dev/ad10, /dev/ad12, & /dev/ad14. I'm assuming I should use  
> the following steps to achieve my goal:
>
> 1. Create the gnop devices /dev/ad10.nop, /dev/ad12.nop, and /dev/ad14.nop,

In general (I'm not sure if it applies to ZFS): If you use ad10pX.nop  
instead, it should work. If you use gnop on the entire disks instead  
of on partitions, you may get in trouble depending on where some  
important metadata is kept (e.g. gpt/gmirror/glabel/... save metadata  
in the last sector, but the last 4k-gnoped-sector is the 4th last  
512b-sector).

See
http://www.leidinger.net/blog/2011/05/03/another-root-on-zfs-howto-optimized-for-4k-sector-drives/
for how I create a 4k-safe pool.

> 2. Add these devices to the zpool with 'zpool add datastore  
> /dev/ad10.nop /dev/ad12.nop'
>
> 3. Export the zpool with 'zpool export datastore'
>
> 4. Destroy gnop devices with 'gnop destroy /dev/da10.nop  
> /dev/da12.nop /dev/da14.nop'
>
> 5. Re-import the zpool data with 'zpool import datastore'
>
> Can someone please confirm that this method is correct, or if there  
> is a more streamlined method for achieving the desired goal?

This (with partitions instead of the whole disks) is the safe way to  
do it. Technically it is not necessary to export the pool and destroy  
the gnop devices. The next reboot will not create the gnop devices and  
ZFS will pick up the drives based upon a zfs-meta-data search on all  
geom-providers.

Bye,
Alexander.

-- 
There is nothing so easy but that it becomes difficult
when you do it reluctantly.
		-- Publius Terentius Afer (Terence)

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137