From owner-freebsd-fs@FreeBSD.ORG Mon Sep 21 17:21:46 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 643181065679 for ; Mon, 21 Sep 2009 17:21:46 +0000 (UTC) (envelope-from ktouet@gmail.com) Received: from mail-yw0-f178.google.com (mail-yw0-f178.google.com [209.85.211.178]) by mx1.freebsd.org (Postfix) with ESMTP id 1F6CA8FC14 for ; Mon, 21 Sep 2009 17:21:45 +0000 (UTC) Received: by ywh8 with SMTP id 8so3787579ywh.14 for ; Mon, 21 Sep 2009 10:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=WuWU7nWRa/yyStfgsl/DdUhBGOi94kChwctX7n3jYzo=; b=P2haug4c5/KCp47+O/iSCY2BIpv7Ldtb0/vEvtJpD93JGMa7nArHe8WVdAlLuMgdsI dmzPBr0S6hUyEXsW2B8GgWgcT5NLwVncQPLnjEXF40RWYhA8Z1wrcbWXwx+mo3HKyXdF x7fLPra8aAx8AhQ04ru58gwUADz/wKbhRXS9Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=xkredB7czGEz3hbCMOB3Hl0NnZ8hEj/BRvKFVxo0A4+4+Yu9Dk7DvneBBsveB5m3pZ M+IyZc2lz9gT9+F5qMpzOuLpX2jqLyDb3TRVNNqcZi1xZM174qH6inJ9owkXk7gFG+O6 u/qzrar8y0oRR/MYhY3dY8bMREjIshFwt7h4A= MIME-Version: 1.0 Received: by 10.91.22.6 with SMTP id z6mr3424606agi.65.1253553705272; Mon, 21 Sep 2009 10:21:45 -0700 (PDT) In-Reply-To: <4AB757E4.5060501@goflexitllc.com> References: <2a5e326f0909201500w1513aeb5ra644f1c748e22f34@mail.gmail.com> <4AB757E4.5060501@goflexitllc.com> Date: Mon, 21 Sep 2009 11:21:45 -0600 Message-ID: <2a5e326f0909211021o431ef53bh3077589efb0bed6c@mail.gmail.com> From: Kurt Touet To: Aaron Hurt Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: ZFS - Unable to offline drive in raidz1 based pool X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Sep 2009 17:21:46 -0000 I thought about that possibility as well.. but I had scrubbed the array within 10 days. I'll give it a shot again today and see if that brings up any other errors (or allows me to offline the drive afterwards). Cheers, -kurt On Mon, Sep 21, 2009 at 4:39 AM, Aaron Hurt wrote: > Kurt Touet wrote: >> >> I am using ZFS pool based on a 4-drive raidz1 setup for storage. =A0I >> believe that one of the drives is failing, and I'd like to >> remove/replace it. =A0The drive has been causing some issues (such as >> becoming non-responsive and hanging the system with timeouts), so I'd >> like to offline it, and then run in degraded mode until I can grab a >> new drive (tomorrow). =A0However, when I disconnected the drive (pulled >> the plug, not using a zpool offline command), the following occurred: >> >> =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0STATE =A0 =A0 READ WRITE CKSUM >> =A0 =A0 =A0 =A0storage =A0 =A0 FAULTED =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 1 >> =A0 =A0 =A0 =A0 =A0raidz1 =A0 =A0DEGRADED =A0 =A0 0 =A0 =A0 0 =A0 =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad14 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad6 =A0 =A0 UNAVAIL =A0 =A0 =A00 =A0 =A0 0 =A0 = =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad12 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad4 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> >> Note: That's my recreation of the output... not the actual text. >> >> At this point, I was unable to to do anything with the pool... and all >> data was inaccessible. =A0Fortunately, the after sitting pulled for a >> bit, I tried putting the failing drive back into the array, and it >> booted properly. =A0Of course, I still want to replace it, but this is >> what happens when I try to take it offline: >> >> monolith# zpool status storage >> =A0pool: storage >> =A0state: ONLINE >> =A0scrub: none requested >> config: >> >> =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0STATE =A0 =A0 READ WRITE CKSUM >> =A0 =A0 =A0 =A0storage =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 0 >> =A0 =A0 =A0 =A0 =A0raidz1 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 = 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad14 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad6 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad12 =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> =A0 =A0 =A0 =A0 =A0 =A0ad4 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >> >> errors: No known data errors >> monolith# zpool offline storage ad6 >> cannot offline ad6: no valid replicas >> monolith# uname -a >> FreeBSD monolith 8.0-RC1 FreeBSD 8.0-RC1 #2 r197370: Sun Sep 20 >> 15:32:08 CST 2009 =A0 =A0 k@monolith:/usr/obj/usr/src/sys/MONOLITH =A0am= d64 >> >> If the array is online and healthy, why can't I simply offline a drive >> and then replace it afterwards? =A0Any thoughts? =A0 Also, how does a >> degraded raidz1 array end up faulting the entire pool? >> >> Thanks, >> -kurt >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> >> !DSPAM:2,4ab6ac55126167777521459! >> >> > > I'm not sure why it would be giving you that message. =A0In a raidz1 you > should be able to sustain one failure. =A0The only thing that comes to mi= nd > this early in the morning would be that somehow your data replication acr= oss > your discs isn't totally in sync. =A0I would suggest you try a scrub and = then > see if you can remove the drive afterwards. > > Aaron Hurt > Managing Partner > Flex I.T., LLC > 611 Commerce Street > Suite 3117 > Nashville, TN =A037203 > Phone: 615.438.7101 > E-mail: aaron@goflexitllc.com > >