Date: Tue, 22 Sep 2009 11:07:23 -0600 From: Kurt Touet <ktouet@gmail.com> To: Pawel Jakub Dawidek <pjd@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS - Unable to offline drive in raidz1 based pool Message-ID: <2a5e326f0909221007m71a84f34r8c07648f8bc8b1ac@mail.gmail.com> In-Reply-To: <20090922125625.GJ6038@garage.freebsd.pl> References: <2a5e326f0909201500w1513aeb5ra644f1c748e22f34@mail.gmail.com> <20090922125625.GJ6038@garage.freebsd.pl>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
On Tue, Sep 22, 2009 at 6:56 AM, Pawel Jakub Dawidek <pjd@freebsd.org> wrote:
> On Sun, Sep 20, 2009 at 04:00:10PM -0600, Kurt Touet wrote:
>> I am using ZFS pool based on a 4-drive raidz1 setup for storage. I
>> believe that one of the drives is failing, and I'd like to
>> remove/replace it. The drive has been causing some issues (such as
>> becoming non-responsive and hanging the system with timeouts), so I'd
>> like to offline it, and then run in degraded mode until I can grab a
>> new drive (tomorrow). However, when I disconnected the drive (pulled
>> the plug, not using a zpool offline command), the following occurred:
>>
>> NAME STATE READ WRITE CKSUM
>> storage FAULTED 0 0 1
>> raidz1 DEGRADED 0 0 0
>> ad14 ONLINE 0 0 0
>> ad6 UNAVAIL 0 0 0
>> ad12 ONLINE 0 0 0
>> ad4 ONLINE 0 0 0
>>
>> Note: That's my recreation of the output... not the actual text.
>>
>> At this point, I was unable to to do anything with the pool... and all
>> data was inaccessible. Fortunately, the after sitting pulled for a
>> bit, I tried putting the failing drive back into the array, and it
>> booted properly. Of course, I still want to replace it, but this is
>> what happens when I try to take it offline:
>>
>> monolith# zpool status storage
>> pool: storage
>> state: ONLINE
>> scrub: none requested
>> config:
>>
>> NAME STATE READ WRITE CKSUM
>> storage ONLINE 0 0 0
>> raidz1 ONLINE 0 0 0
>> ad14 ONLINE 0 0 0
>> ad6 ONLINE 0 0 0
>> ad12 ONLINE 0 0 0
>> ad4 ONLINE 0 0 0
>>
>> errors: No known data errors
>> monolith# zpool offline storage ad6
>> cannot offline ad6: no valid replicas
>
> Could you send the output of:
>
> # apply "zdb -l /dev/%1" ad{4,6,12,14}
>
Sure thing.
[-- Attachment #2 --]
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2a5e326f0909221007m71a84f34r8c07648f8bc8b1ac>