From owner-freebsd-fs@FreeBSD.ORG  Fri Jun 19 00:45:06 2009
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 78F241065672
	for <freebsd-fs@freebsd.org>; Fri, 19 Jun 2009 00:45:06 +0000 (UTC)
	(envelope-from morganw@chemikals.org)
Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2])
	by mx1.freebsd.org (Postfix) with ESMTP id 15B2A8FC0A
	for <freebsd-fs@freebsd.org>; Fri, 19 Jun 2009 00:45:06 +0000 (UTC)
	(envelope-from morganw@chemikals.org)
Received: from volatile.chemikals.org (adsl-67-127-7.shv.bellsouth.net
	[98.67.127.7])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client did not present a certificate)
	by warped.bluecherry.net (Postfix) with ESMTPSA id 7A9B78CA7115;
	Thu, 18 Jun 2009 19:45:04 -0500 (CDT)
Received: from localhost (morganw@localhost [127.0.0.1])
	by volatile.chemikals.org (8.14.3/8.14.3) with ESMTP id n5J0isRo043571; 
	Thu, 18 Jun 2009 19:44:58 -0500 (CDT)
	(envelope-from morganw@chemikals.org)
Date: Thu, 18 Jun 2009 19:44:54 -0500 (CDT)
From: Wes Morgan <morganw@chemikals.org>
To: Andrew Snow <andrew@modulus.org>
In-Reply-To: <4A3AC5B2.9010607@modulus.org>
Message-ID: <alpine.BSF.2.00.0906181941130.10212@ibyngvyr.purzvxnyf.bet>
References: <m2bpol6v8r.wl%randy@psg.com> <4A3ABF76.3020905@modulus.org>
	<cf9b1ee00906181540w746edafi4a560b0925b14e27@mail.gmail.com>
	<m28wjp6tp2.wl%randy@psg.com> <4A3AC5B2.9010607@modulus.org>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Cc: Randy Bush <randy@psg.com>, freebsd-fs@freebsd.org
Subject: Re: adding drive to raidz1
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Jun 2009 00:45:06 -0000

On Fri, 19 Jun 2009, Andrew Snow wrote:

>
>> 	    NAME        STATE     READ WRITE CKSUM
>> 	    tank        ONLINE       0     0     0
>> 	      raidz1    ONLINE       0     0     0
>> 		ad4s3   ONLINE       0     0     0
>> 		ad5s3   ONLINE       0     0     0
>> 		ad6s1   ONLINE       0     0     0
>> 	      ad7s1     ONLINE       0     0     0
>
> Here you have created a non-redundant stripe with two vdev members:
> 1.  a 3-disk RAIDZ1 and
> 2.a single disk.
>
> So you can't ever remove the ad7s1 without data loss.
>
> If you haven't written anything to the pool since adding ad7s1, you can 
> probably yank the disk out and ignore any errors but the error messages will 
> never go away until you recreate the whole pool from scratch

If you yank ad7s1 the pool will become unavailable. You could remove one 
of the slices in the raidz, though.

The only way to "fix" this is just what everyone has said... Back up the 
data, destroy the pool and recreate. When you do this, if you don't want 
to be using slices, just "don't" -- use

zpool create raidz somethingbesidestankfortheloveofgod ad4 ad5 ad6 ad7

And you'll be set. But you're using ad5s3 and ad6s3, are the first two 
slices in use?