From owner-freebsd-fs@FreeBSD.ORG Wed Jun 1 08:19:42 2011 Return-Path: <owner-freebsd-fs@FreeBSD.ORG> Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34503106566C for <freebsd-fs@freebsd.org>; Wed, 1 Jun 2011 08:19:42 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id BAB8A8FC34 for <freebsd-fs@freebsd.org>; Wed, 1 Jun 2011 08:19:41 +0000 (UTC) Received: from outgoing.leidinger.net (p5B155DF9.dip.t-dialin.net [91.21.93.249]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id E7087844010; Wed, 1 Jun 2011 10:19:27 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [IPv6:fd73:10c7:2053:1::3:102]) by outgoing.leidinger.net (Postfix) with ESMTP id 2A0CC33E0; Wed, 1 Jun 2011 10:19:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1306916365; bh=shvIaJugg/Lm7k465HqiPoGsF7+wTaOfyLdY0D9P8Gs=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=fTSAW+82QqTcY8piuQTYcFppW7F7a4eR6rxyxFURnywoWTXH/JPb5W0wvIXGPim0e U1yH9tL0GXTMZw7ElgAwXVsNzzeS3Z7BbyxrZoGfy6ghQo5YCRmN2JTj0WNsyfFZbG mojwVUVYQhkIc6AyO6YxdTYaqZXDg7OA7lzveoT8cvvyMHgHWkRLNXEFnoJBictgMB 0C1XTvUamJVfTBMDcLuKkcSO698akyBo57VBpeAf9jtzYp15f4OiG9bQ+leEj52Kos e1bmSFrMg85Y1oi8kA+acYXiBfaQq27MUQVJfK0+4/mH1TGiiaBAofA+UN0wFzVJw6 YUK1mbke9+Zvw== Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.14.4/Submit) id p518JOs5063679; Wed, 1 Jun 2011 10:19:24 +0200 (CEST) (envelope-from Alexander@Leidinger.net) X-Authentication-Warning: webmail.leidinger.net: www set sender to Alexander@Leidinger.net using -f Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 01 Jun 2011 10:19:24 +0200 Message-ID: <20110601101924.11334wftbjfrvor0@webmail.leidinger.net> Date: Wed, 01 Jun 2011 10:19:24 +0200 From: Alexander Leidinger <Alexander@Leidinger.net> To: Negasi Gerima <negasi@decayingorbits.com> References: <4DE5D7B1.5040902@decayingorbits.com> In-Reply-To: <4DE5D7B1.5040902@decayingorbits.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.6) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: E7087844010.A0CDB X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=0.054, required 6, autolearn=disabled, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, TW_ZD 0.08, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1307521168.89268@SQu9F5RP2TtpahR9Iv17BA X-EBL-Spam-Status: No Cc: freebsd-fs@freebsd.org Subject: Re: ZFS on 4K drives - Expanding zpool and adding spares (gnop method) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems <freebsd-fs.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs> List-Post: <mailto:freebsd-fs@freebsd.org> List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 01 Jun 2011 08:19:42 -0000 Quoting Negasi Gerima <negasi@decayingorbits.com> (from Wed, 01 Jun 2011 02:09:53 -0400): > Hello, > > I apologize if this topic is redundant, but I couldn't find an exact > answer to my specific question. So here goes. > > I recently decided to expand the storage in my home server with a > bunch of 1.5Tb Samsung 4K hard drives. Using the guide at the > following link I was able to use the "gnop method" in order to align > the drives for use with 4K sectors: > > http://ivoras.net/blog/tree/2011-01-01.freebsd-on-4k-sector-drives.html > > My question to the list is this, now that my new zpool is created > and I've made sure the drives are using the correct layout, how > would I go about expanding the zpool in the future with additional > drives and/or spares? I emailed the author of the article but > received a reply that I found terse at best. > > I'm assuming that I would have to create a new gnop device for each > additional hard drive, and then add those gnop devices to the pool. The 4k-sector part results in a pool specific config setting. I would expect that if you add additional drives (no matter if 4k or not), that the pool will continue to use 4k sectors. So I would expect that you do not need gnop to expend an existing pool, but I have not tested/verified this. You could test this with md(4) devices where you use gnop for the initial creation and no gnop during extending it. When it is extended you just run zdp <poolname) | grep ashift and verify that it is set to 12. > For example, if I had a pool called 'datastore' and three additional > drives /dev/ad10, /dev/ad12, & /dev/ad14. I'm assuming I should use > the following steps to achieve my goal: > > 1. Create the gnop devices /dev/ad10.nop, /dev/ad12.nop, and /dev/ad14.nop, In general (I'm not sure if it applies to ZFS): If you use ad10pX.nop instead, it should work. If you use gnop on the entire disks instead of on partitions, you may get in trouble depending on where some important metadata is kept (e.g. gpt/gmirror/glabel/... save metadata in the last sector, but the last 4k-gnoped-sector is the 4th last 512b-sector). See http://www.leidinger.net/blog/2011/05/03/another-root-on-zfs-howto-optimized-for-4k-sector-drives/ for how I create a 4k-safe pool. > 2. Add these devices to the zpool with 'zpool add datastore > /dev/ad10.nop /dev/ad12.nop' > > 3. Export the zpool with 'zpool export datastore' > > 4. Destroy gnop devices with 'gnop destroy /dev/da10.nop > /dev/da12.nop /dev/da14.nop' > > 5. Re-import the zpool data with 'zpool import datastore' > > Can someone please confirm that this method is correct, or if there > is a more streamlined method for achieving the desired goal? This (with partitions instead of the whole disks) is the safe way to do it. Technically it is not necessary to export the pool and destroy the gnop devices. The next reboot will not create the gnop devices and ZFS will pick up the drives based upon a zfs-meta-data search on all geom-providers. Bye, Alexander. -- There is nothing so easy but that it becomes difficult when you do it reluctantly. -- Publius Terentius Afer (Terence) http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137