From owner-freebsd-fs@FreeBSD.ORG Mon Sep 21 11:06:22 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DDAE106568D for ; Mon, 21 Sep 2009 11:06:22 +0000 (UTC) (envelope-from aaron@goflexitllc.com) Received: from mail.goflexitllc.com (mail.goflexitllc.com [70.38.81.12]) by mx1.freebsd.org (Postfix) with ESMTP id 175288FC14 for ; Mon, 21 Sep 2009 11:06:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=goflexitllc.com; h=message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type; s=gamma; bh=tJAXRL8a0UFmVuykMkwP/jTMP CY=; b=iXGS79H7Jq+HYBFzfHxQbdT3tnyp4eyINhVs5UEYeM5T1yjgFioVKR0G8 EHQxyz5BTbyFYd31xaWFsQmlgD528+fd2N2ttOriMweOxTywPGjNpLxzGQ+9dkz/ Q4E6Mr4 DomainKey-Signature: a=rsa-sha1; c=nofws; d=goflexitllc.com; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to: content-type; q=dns; s=gamma; b=KApVwUS7eZd4KuQsIMyWdHJCYreCtcxV yggg7iiDDkHV66yi9FLZ6UumT/4ccS3tBqLDJvzGDAhxKClP0DmyKkLrXCNo7Gzu nZ0nCdSjgfr8K7c15MurGp0pdvJEBKEX Received: (qmail 21572 invoked by uid 89); 21 Sep 2009 10:42:53 -0000 Received: (simscan 1.4.1 ppid 21548 pid 21554 t 0.2630s) (scanners: regex: 1.4.1 attach: 1.4.1 clamav: 0.95.1/m:); 21 Sep 0109 10:42:52 -0000 DomainKey-Status: no signature X-Originating-IP: 69.27.151.4 Received: from temp4.wavelinx.net (HELO ?172.16.1.128?) (aaron@goflexitllc.com@69.27.151.4) by mail.goflexitllc.com with ESMTPA; 21 Sep 2009 10:42:51 -0000 Message-ID: <4AB757E4.5060501@goflexitllc.com> Date: Mon, 21 Sep 2009 05:39:32 -0500 From: Aaron Hurt User-Agent: Thunderbird 2.0.0.22 (X11/20090719) MIME-Version: 1.0 To: Kurt Touet References: <2a5e326f0909201500w1513aeb5ra644f1c748e22f34@mail.gmail.com> In-Reply-To: <2a5e326f0909201500w1513aeb5ra644f1c748e22f34@mail.gmail.com> Content-Type: multipart/mixed; boundary="------------080102010205030309010709" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS - Unable to offline drive in raidz1 based pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2009 11:06:22 -0000 This is a multi-part message in MIME format. --------------080102010205030309010709 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Kurt Touet wrote: > I am using ZFS pool based on a 4-drive raidz1 setup for storage. I > believe that one of the drives is failing, and I'd like to > remove/replace it. The drive has been causing some issues (such as > becoming non-responsive and hanging the system with timeouts), so I'd > like to offline it, and then run in degraded mode until I can grab a > new drive (tomorrow). However, when I disconnected the drive (pulled > the plug, not using a zpool offline command), the following occurred: > > NAME STATE READ WRITE CKSUM > storage FAULTED 0 0 1 > raidz1 DEGRADED 0 0 0 > ad14 ONLINE 0 0 0 > ad6 UNAVAIL 0 0 0 > ad12 ONLINE 0 0 0 > ad4 ONLINE 0 0 0 > > Note: That's my recreation of the output... not the actual text. > > At this point, I was unable to to do anything with the pool... and all > data was inaccessible. Fortunately, the after sitting pulled for a > bit, I tried putting the failing drive back into the array, and it > booted properly. Of course, I still want to replace it, but this is > what happens when I try to take it offline: > > monolith# zpool status storage > pool: storage > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > storage ONLINE 0 0 0 > raidz1 ONLINE 0 0 0 > ad14 ONLINE 0 0 0 > ad6 ONLINE 0 0 0 > ad12 ONLINE 0 0 0 > ad4 ONLINE 0 0 0 > > errors: No known data errors > monolith# zpool offline storage ad6 > cannot offline ad6: no valid replicas > monolith# uname -a > FreeBSD monolith 8.0-RC1 FreeBSD 8.0-RC1 #2 r197370: Sun Sep 20 > 15:32:08 CST 2009 k@monolith:/usr/obj/usr/src/sys/MONOLITH amd64 > > If the array is online and healthy, why can't I simply offline a drive > and then replace it afterwards? Any thoughts? Also, how does a > degraded raidz1 array end up faulting the entire pool? > > Thanks, > -kurt > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > !DSPAM:2,4ab6ac55126167777521459! > > I'm not sure why it would be giving you that message. In a raidz1 you should be able to sustain one failure. The only thing that comes to mind this early in the morning would be that somehow your data replication across your discs isn't totally in sync. I would suggest you try a scrub and then see if you can remove the drive afterwards. Aaron Hurt Managing Partner Flex I.T., LLC 611 Commerce Street Suite 3117 Nashville, TN 37203 Phone: 615.438.7101 E-mail: aaron@goflexitllc.com --------------080102010205030309010709--