From owner-freebsd-stable@FreeBSD.ORG Thu Jul 22 00:02:24 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A3ED106566B for ; Thu, 22 Jul 2010 00:02:24 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id 58B7A8FC1C for ; Thu, 22 Jul 2010 00:02:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id 9EF465093D; Thu, 22 Jul 2010 01:02:23 +0100 (BST) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TZHhkzzmnRnQ; Thu, 22 Jul 2010 01:02:22 +0100 (BST) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 8EF155089E ; Thu, 22 Jul 2010 01:02:22 +0100 (BST) Message-ID: <4C478A87.9070102@langille.org> Date: Wed, 21 Jul 2010 20:02:15 -0400 From: Dan Langille Organization: The FreeBSD Diary User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.7) Gecko/20100713 Thunderbird/3.1.1 MIME-Version: 1.0 To: Adam Vande More References: <4C4504DF.30602@langille.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable Subject: Re: Problems replacing failing drive in ZFS pool X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 00:02:24 -0000 On 7/19/2010 10:50 PM, Adam Vande More wrote: > On Mon, Jul 19, 2010 at 9:07 PM, Dan Langille wrote: > >> I think it's because you pull the old drive, boot with the new drive, >>> the controller re-numbers all the devices (ie da3 is now da2, da2 is >>> now da1, da1 is now da0, da0 is now da6, etc), and ZFS thinks that all >>> the drives have changed, thus corrupting the pool. I've had this >>> happen on our storage servers a couple of times before I started using >>> glabel(8) on all our drives (dead drive on RAID controller, remove >>> drive, reboot for whatever reason, all device nodes are renumbered, >>> everything goes kablooey). >>> >> >> >> Can you explain a bit about how you use glabel(8) in conjunction with ZFS? >> If I can retrofit this into an exist ZFS array to make things easier in the >> future... >> > > If you've used whole disks in ZFS, you can't retrofit it if by retrofit you > mean an almost painless method of resolving this. GEOM setup stuff > generally should happen BEFORE the file system is on it. > > You would create your partition(s) slightly smaller than the disk, label it, > then use the resulting device as your zfs device when creating the pool. If > you have an existing full disk install, that means restoring the data after > you've done those steps. It works just as well with MBR style partitioning, > there's nothing saying you have to use GPT. GPT is just better though in > terms of ease of use IMO among other things. FYI, this is exactly what I'm doing to do. I have obtained addition HDD to serve as temporary storage. I will also use them for practicing the commands before destroying the original array. I'll post my plan to the list for review. -- Dan Langille - http://langille.org/