Date: Thu, 1 Nov 2012 02:56:15 -0000 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Steven Hartland" <killing@multiplay.co.uk>, "Peter Jeremy" <peter@rulingia.com> Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: ZFS corruption due to lack of space? Message-ID: <9DB937FEA7634C4BAC49EF5823F93CA3@multiplay.co.uk> References: <27087376D1C14132A3CC1B4016912F6D@multiplay.co.uk> <20121031212346.GL3309@server.rulingia.com> <F569C535F56843F3B16691C206AD6492@multiplay.co.uk> <A5515D2F4E974E619E24892D660E4DFE@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message -----
From: "Steven Hartland" <killing@multiplay.co.uk>
To: "Peter Jeremy" <peter@rulingia.com>
Cc: <freebsd-fs@freebsd.org>; <freebsd-stable@freebsd.org>
Sent: Thursday, November 01, 2012 12:19 AM
Subject: Re: ZFS corruption due to lack of space?
>
> ----- Original Message -----
> From: "Steven Hartland" <killing@multiplay.co.uk>
> To: "Peter Jeremy" <peter@rulingia.com>
> Cc: <freebsd-fs@freebsd.org>; <freebsd-stable@freebsd.org>
> Sent: Thursday, November 01, 2012 12:09 AM
> Subject: Re: ZFS corruption due to lack of space?
>
>
>> On 2012-Oct-31 17:25:09 -0000, Steven Hartland <steven@multiplay.co.uk> wrote:
>>>>Been running some tests on new hardware here to verify all
>>>>is good. One of the tests was to fill the zfs array which
>>>>seems like its totally corrupted the tank.
>>>
>>>I've accidently "filled" a pool, and had multiple processes try to
>>>write to the full pool, without either emptying the free space reserve
>>>(so I could still delete the offending files) or corrupting the pool.
>>
>> Same here but its the first time I've had ZIL in place at the time so
>> wondering if that may be playing a factor.
>>
>>> Had you tried to read/write the raw disks before you tried the
>>> ZFS testing?
>>
>> Yes, didn't see any issues but then it wasn't checksuming so tbh I
>> wouldn't have noticed if it was silently corrupting data.
>>
>>>Do you have compression and/or dedupe enabled on the pool?
>>
>> Nope bog standard raidz2 no additional settings
>>
>>>>1. Given the information it seems like the multiple writes filling
>>>>the disk may have caused metadata corruption?
>>>
>>> I don't recall seeing this reported before.
>>
>> Nore me and we've been using ZFS for years, but never filled a pool
>> with such known simultanious access + ZIL before
>>
>>>>2. Is there anyway to stop the scrub?
>>>
>>>Other than freeing up some space, I don't think so. If this is a test
>>>pool that you don't need, you could try destroying it and re-creating
>>>it - that may be quicker and easier than recovering the existing pool.
>>
>> Artems trick of cat /dev/null > /tank2/<bigfile> worked and I've now
>> managed to stop the scrub :)
>>
>>>>3. Surely low space should never prevent stopping a scrub?
>>>
>>> As Artem noted, ZFS is a copy-on-write filesystem. It is supposed to
>>> reserve some free space to allow metadata updates (stop scrubs, delete
>>> files, etc) even when it is "full" but I have seen reports of this not
>>> working correctly in the past. A truncate-in-place may work.
>>
>> Yes it did thanks, but as you said if this metadata update was failing
>> due to out of space lends credability to the fact that the same lack of
>> space and hence failure to update metadata could have also caused the
>> corruption in the first place.
>>
>> Its interesting to note that the zpool is reporting pleanty of free space
>> even when the root zfs volume was showing 0, so you would expect there
>> to be pleanty of space for it be able to stop the scrub but it appears
>> not which is definitely interesting and could point to the underlying
>> cause?
>>
>> zpool list tank2
>> NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
>> tank2 19T 18.7T 304G 98% 1.00x ONLINE -
>>
>> zfs list tank2
>> NAME USED AVAIL REFER MOUNTPOINT
>> tank2 13.3T 0 13.3T /tank2
>>
>> Current state is:-
>> scan: scrub in progress since Wed Oct 31 16:13:53 2012
>> 1.64T scanned out of 18.7T at 62.8M/s, 79h12m to go
>> 280M repaired, 8.76% done
>>
>> Something else that was interesting is while the scrub was running
>> devd was using a good amount of CPU 40% of a 3.3Ghz core, which I've
>> never seen before. Any ideas why its usage would be so high?
>
>
> In case its useful here's the output from a zdb tank2 so far:-
> zdb tank2
>
> Cached configuration:
> version: 28
> name: 'tank2'
> state: 0
> txg: 39502
> pool_guid: 15779146362913479443
> hostid: 1751781486
> vdev_children: 3
> vdev_tree:
> type: 'root'
> id: 0
> guid: 15779146362913479443
> create_txg: 4
> children[0]:
> type: 'raidz'
> id: 0
> guid: 8518972900227438019
> nparity: 2
> metaslab_array: 33
> metaslab_shift: 37
> ashift: 9
> asize: 21004116295680
> is_log: 0
> create_txg: 4
> children[0]:
> type: 'disk'
> id: 0
> guid: 15577380450172137060
> path: '/dev/mfisyspd0'
> phys_path: '/dev/mfisyspd0'
> whole_disk: 1
> DTL: 236
> create_txg: 4
> children[1]:
> type: 'disk'
> id: 1
> guid: 16940350228793267704
> path: '/dev/mfisyspd1'
> phys_path: '/dev/mfisyspd1'
> whole_disk: 1
> DTL: 235
> create_txg: 4
> children[2]:
> type: 'disk'
> id: 2
> guid: 9264743178245473794
> path: '/dev/mfisyspd2'
> phys_path: '/dev/mfisyspd2'
> whole_disk: 1
> DTL: 234
> create_txg: 4
> children[3]:
> type: 'disk'
> id: 3
> guid: 432716341673487166
> path: '/dev/mfisyspd3'
> phys_path: '/dev/mfisyspd3'
> whole_disk: 1
> DTL: 233
> create_txg: 4
> children[4]:
> type: 'disk'
> id: 4
> guid: 18217760646550913544
> path: '/dev/mfisyspd4'
> phys_path: '/dev/mfisyspd4'
> whole_disk: 1
> DTL: 232
> create_txg: 4
> children[5]:
> type: 'disk'
> id: 5
> guid: 6964614355298004256
> path: '/dev/mfisyspd5'
> phys_path: '/dev/mfisyspd5'
> whole_disk: 1
> DTL: 231
> create_txg: 4
> children[6]:
> type: 'disk'
> id: 6
> guid: 4397961270160308034
> path: '/dev/mfisyspd6'
> phys_path: '/dev/mfisyspd6'
> whole_disk: 1
> DTL: 230
> create_txg: 4
> children[1]:
> type: 'disk'
> id: 1
> guid: 13757670211012452260
> path: '/dev/mfisyspd7p3'
> phys_path: '/dev/mfisyspd7p3'
> whole_disk: 1
> metaslab_array: 32
> metaslab_shift: 27
> ashift: 9
> asize: 14125891584
> is_log: 1
> DTL: 237
> create_txg: 4
> children[2]:
> type: 'disk'
> id: 2
> guid: 7315839509249482920
> path: '/dev/mfisyspd8p3'
> phys_path: '/dev/mfisyspd8p3'
> whole_disk: 1
> metaslab_array: 31
> metaslab_shift: 27
> ashift: 9
> asize: 14125891584
> is_log: 1
> DTL: 229
> create_txg: 4
>
> MOS Configuration:
> version: 28
> name: 'tank2'
> state: 0
> txg: 39502
> pool_guid: 15779146362913479443
> hostid: 1751781486
> vdev_children: 3
> vdev_tree:
> type: 'root'
> id: 0
> guid: 15779146362913479443
> create_txg: 4
> children[0]:
> type: 'raidz'
> id: 0
> guid: 8518972900227438019
> nparity: 2
> metaslab_array: 33
> metaslab_shift: 37
> ashift: 9
> asize: 21004116295680
> is_log: 0
> create_txg: 4
> children[0]:
> type: 'disk'
> id: 0
> guid: 15577380450172137060
> path: '/dev/mfisyspd0'
> phys_path: '/dev/mfisyspd0'
> whole_disk: 1
> DTL: 236
> create_txg: 4
> children[1]:
> type: 'disk'
> id: 1
> guid: 16940350228793267704
> path: '/dev/mfisyspd1'
> phys_path: '/dev/mfisyspd1'
> whole_disk: 1
> DTL: 235
> create_txg: 4
> children[2]:
> type: 'disk'
> id: 2
> guid: 9264743178245473794
> path: '/dev/mfisyspd2'
> phys_path: '/dev/mfisyspd2'
> whole_disk: 1
> DTL: 234
> create_txg: 4
> children[3]:
> type: 'disk'
> id: 3
> guid: 432716341673487166
> path: '/dev/mfisyspd3'
> phys_path: '/dev/mfisyspd3'
> whole_disk: 1
> DTL: 233
> create_txg: 4
> children[4]:
> type: 'disk'
> id: 4
> guid: 18217760646550913544
> path: '/dev/mfisyspd4'
> phys_path: '/dev/mfisyspd4'
> whole_disk: 1
> DTL: 232
> create_txg: 4
> children[5]:
> type: 'disk'
> id: 5
> guid: 6964614355298004256
> path: '/dev/mfisyspd5'
> phys_path: '/dev/mfisyspd5'
> whole_disk: 1
> DTL: 231
> create_txg: 4
> children[6]:
> type: 'disk'
> id: 6
> guid: 4397961270160308034
> path: '/dev/mfisyspd6'
> phys_path: '/dev/mfisyspd6'
> whole_disk: 1
> DTL: 230
> create_txg: 4
> children[1]:
> type: 'disk'
> id: 1
> guid: 13757670211012452260
> path: '/dev/mfisyspd7p3'
> phys_path: '/dev/mfisyspd7p3'
> whole_disk: 1
> metaslab_array: 32
> metaslab_shift: 27
> ashift: 9
> asize: 14125891584
> is_log: 1
> DTL: 237
> create_txg: 4
> children[2]:
> type: 'disk'
> id: 2
> guid: 7315839509249482920
> path: '/dev/mfisyspd8p3'
> phys_path: '/dev/mfisyspd8p3'
> whole_disk: 1
> metaslab_array: 31
> metaslab_shift: 27
> ashift: 9
> asize: 14125891584
> is_log: 1
> DTL: 229
> create_txg: 4
>
> Uberblock:
> magic = 0000000000bab10c
> version = 28
> txg = 39547
> guid_sum = 6486691012039134504
> timestamp = 1351727718 UTC = Wed Oct 31 23:55:18 2012
>
> All DDTs are empty
>
> Metaslabs:
> vdev 0
> metaslabs 152 offset spacemap
> free --------------- ------------------- --------------- -------------
> metaslab 0 offset 0 spacemap 36 free 1.41G
> segments 4170 maxsize 1.38G freepct 1%
> metaslab 1 offset 2000000000 spacemap 65 free 1.45G
> segments 4508 maxsize 1.40G freepct 1%
> metaslab 2 offset 4000000000 spacemap 70 free 1.44G
> segments 5723 maxsize 1.39G freepct 1%
> metaslab 3 offset 6000000000 spacemap 77 free 1.41G
> segments 6635 maxsize 1.35G freepct 1%
> metaslab 4 offset 8000000000 spacemap 80 free 1.46G
> segments 6408 maxsize 1.40G freepct 1%
> metaslab 5 offset a000000000 spacemap 81 free 1.47G
> segments 6480 maxsize 1.42G freepct 1%
> metaslab 6 offset c000000000 spacemap 82 free 1.48G
> segments 6684 maxsize 1.42G freepct 1%
> metaslab 7 offset e000000000 spacemap 83 free 1.48G
> segments 6873 maxsize 1.42G freepct 1%
> metaslab 8 offset 10000000000 spacemap 84 free 1.48G
> segments 7298 maxsize 1.42G freepct 1%
> metaslab 9 offset 12000000000 spacemap 85 free 1.43G
> segments 6699 maxsize 1.37G freepct 1%
> metaslab 10 offset 14000000000 spacemap 88 free 1.50G
> segments 6781 maxsize 1.44G freepct 1%
> metaslab 11 offset 16000000000 spacemap 89 free 1.50G
> segments 6434 maxsize 1.44G freepct 1%
> metaslab 12 offset 18000000000 spacemap 90 free 1.51G
> segments 7188 maxsize 1.44G freepct 1%
> metaslab 13 offset 1a000000000 spacemap 91 free 1.49G
> segments 6712 maxsize 1.42G freepct 1%
> metaslab 14 offset 1c000000000 spacemap 92 free 1.52G
> segments 6810 maxsize 1.46G freepct 1%
> metaslab 15 offset 1e000000000 spacemap 93 free 1.52G
> segments 8306 maxsize 1.41G freepct 1%
> metaslab 16 offset 20000000000 spacemap 94 free 1.49G
> segments 21881 maxsize 660M freepct 1%
> metaslab 17 offset 22000000000 spacemap 97 free 1.50G
> segments 9590 maxsize 1.32G freepct 1%
> metaslab 18 offset 24000000000 spacemap 98 free 1.53G
> segments 6921 maxsize 1.47G freepct 1%
> metaslab 19 offset 26000000000 spacemap 99 free 1.53G
> segments 7982 maxsize 1.46G freepct 1%
> metaslab 20 offset 28000000000 spacemap 100 free 1.55G
> segments 7943 maxsize 1.48G freepct 1%
> metaslab 21 offset 2a000000000 spacemap 101 free 1.54G
> segments 8049 maxsize 1.47G freepct 1%
> metaslab 22 offset 2c000000000 spacemap 102 free 1.54G
> segments 8205 maxsize 1.46G freepct 1%
> metaslab 23 offset 2e000000000 spacemap 103 free 1.53G
> segments 11339 maxsize 1.37G freepct 1%
> metaslab 24 offset 30000000000 spacemap 104 free 1.55G
> segments 11536 maxsize 1.38G freepct 1%
> metaslab 25 offset 32000000000 spacemap 105 free 1.58G
> segments 7281 maxsize 1.50G freepct 1%
> metaslab 26 offset 34000000000 spacemap 108 free 1.57G
> segments 7917 maxsize 1.49G freepct 1%
> metaslab 27 offset 36000000000 spacemap 109 free 1.59G
> segments 8446 maxsize 1.51G freepct 1%
> metaslab 28 offset 38000000000 spacemap 110 free 1.59G
> segments 8437 maxsize 1.51G freepct 1%
> metaslab 29 offset 3a000000000 spacemap 35 free 1.60G
> segments 22991 maxsize 1.21G freepct 1%
> metaslab 30 offset 3c000000000 spacemap 111 free 1.57G
> segments 8358 maxsize 1.49G freepct 1%
> metaslab 31 offset 3e000000000 spacemap 112 free 1.62G
> segments 7724 maxsize 1.53G freepct 1%
> metaslab 32 offset 40000000000 spacemap 113 free 1.62G
> segments 8314 maxsize 1.53G freepct 1%
> metaslab 33 offset 42000000000 spacemap 114 free 1.61G
> segments 8294 maxsize 1.52G freepct 1%
> metaslab 34 offset 44000000000 spacemap 115 free 1.63G
> segments 8422 maxsize 1.54G freepct 1%
> metaslab 35 offset 46000000000 spacemap 116 free 1.60G
> segments 8125 maxsize 1.51G freepct 1%
> metaslab 36 offset 48000000000 spacemap 117 free 1.62G
> segments 8048 maxsize 1.53G freepct 1%
> metaslab 37 offset 4a000000000 spacemap 118 free 1.60G
> segments 8648 maxsize 1.51G freepct 1%
> metaslab 38 offset 4c000000000 spacemap 119 free 1.66G
> segments 8316 maxsize 1.57G freepct 1%
> metaslab 39 offset 4e000000000 spacemap 87 free 1.62G
> segments 27153 maxsize 1.05G freepct 1%
> metaslab 40 offset 50000000000 spacemap 121 free 1.66G
> segments 9314 maxsize 1.56G freepct 1%
> metaslab 41 offset 52000000000 spacemap 122 free 1.64G
> segments 9334 maxsize 1.56G freepct 1%
> metaslab 42 offset 54000000000 spacemap 123 free 1.67G
> segments 10391 maxsize 1.51G freepct 1%
> metaslab 43 offset 56000000000 spacemap 126 free 1.65G
> segments 12514 maxsize 1.49G freepct 1%
> metaslab 44 offset 58000000000 spacemap 127 free 1.67G
> segments 13441 maxsize 1.50G freepct 1%
> metaslab 45 offset 5a000000000 spacemap 129 free 1.70G
> segments 13288 maxsize 1.54G freepct 1%
> metaslab 46 offset 5c000000000 spacemap 96 free 1.71G
> segments 27184 maxsize 1.07G freepct 1%
> metaslab 47 offset 5e000000000 spacemap 130 free 1.69G
> segments 10019 maxsize 1.61G freepct 1%
> metaslab 48 offset 60000000000 spacemap 133 free 1.69G
> segments 13025 maxsize 1.53G freepct 1%
> metaslab 49 offset 62000000000 spacemap 135 free 1.71G
> segments 10562 maxsize 1.63G freepct 1%
> metaslab 50 offset 64000000000 spacemap 136 free 1.74G
> segments 9827 maxsize 1.66G freepct 1%
> metaslab 51 offset 66000000000 spacemap 137 free 1.73G
> segments 10206 maxsize 1.65G freepct 1%
> metaslab 52 offset 68000000000 spacemap 138 free 1.75G
> segments 9747 maxsize 1.67G freepct 1%
> metaslab 53 offset 6a000000000 spacemap 139 free 1.76G
> segments 14248 maxsize 1.57G freepct 1%
> metaslab 54 offset 6c000000000 spacemap 107 free 1.76G
> segments 29803 maxsize 987M freepct 1%
> metaslab 55 offset 6e000000000 spacemap 142 free 1.76G
> segments 9068 maxsize 1.68G freepct 1%
> metaslab 56 offset 70000000000 spacemap 143 free 1.76G
> segments 10561 maxsize 1.68G freepct 1%
> metaslab 57 offset 72000000000 spacemap 144 free 1.78G
> segments 10234 maxsize 1.70G freepct 1%
> metaslab 58 offset 74000000000 spacemap 34 free 1.79G
> segments 12737 maxsize 1.49G freepct 1%
> metaslab 59 offset 76000000000 spacemap 145 free 1.80G
> segments 10211 maxsize 1.71G freepct 1%
> metaslab 60 offset 78000000000 spacemap 146 free 1.77G
> segments 10696 maxsize 1.68G freepct 1%
> metaslab 61 offset 7a000000000 spacemap 147 free 1.81G
> segments 10934 maxsize 1.71G freepct 1%
> metaslab 62 offset 7c000000000 spacemap 148 free 1.81G
> segments 8698 maxsize 1.73G freepct 1%
> metaslab 63 offset 7e000000000 spacemap 149 free 1.82G
> segments 9165 maxsize 1.74G freepct 1%
> metaslab 64 offset 80000000000 spacemap 152 free 1.83G
> segments 9388 maxsize 1.74G freepct 1%
> metaslab 65 offset 82000000000 spacemap 154 free 1.84G
> segments 11321 maxsize 1.74G freepct 1%
> metaslab 66 offset 84000000000 spacemap 155 free 1.85G
> segments 10040 maxsize 1.76G freepct 1%
> metaslab 67 offset 86000000000 spacemap 156 free 1.86G
> segments 10531 maxsize 1.77G freepct 1%
> metaslab 68 offset 88000000000 spacemap 86 free 1.84G
> segments 8518 maxsize 1.73G freepct 1%
> metaslab 69 offset 8a000000000 spacemap 120 free 1.87G
> segments 18100 maxsize 1.51G freepct 1%
> metaslab 70 offset 8c000000000 spacemap 157 free 1.88G
> segments 12773 maxsize 1.70G freepct 1%
> metaslab 71 offset 8e000000000 spacemap 159 free 1.89G
> segments 11443 maxsize 1.79G freepct 1%
> metaslab 72 offset 90000000000 spacemap 125 free 1.90G
> segments 13633 maxsize 1.72G freepct 1%
> metaslab 73 offset 92000000000 spacemap 160 free 1.91G
> segments 10724 maxsize 1.81G freepct 1%
> metaslab 74 offset 94000000000 spacemap 161 free 1.92G
> segments 10550 maxsize 1.77G freepct 1%
> metaslab 75 offset 96000000000 spacemap 162 free 1.92G
> segments 10027 maxsize 1.83G freepct 1%
> metaslab 76 offset 98000000000 spacemap 132 free 1.93G
> segments 16007 maxsize 1.69G freepct 1%
> metaslab 77 offset 9a000000000 spacemap 164 free 1.94G
> segments 9721 maxsize 1.84G freepct 1%
> metaslab 78 offset 9c000000000 spacemap 134 free 1.95G
> segments 26262 maxsize 1.35G freepct 1%
> metaslab 79 offset 9e000000000 spacemap 165 free 1.95G
> segments 7968 maxsize 1.88G freepct 1%
> metaslab 80 offset a0000000000 spacemap 166 free 1.97G
> segments 7757 maxsize 1.89G freepct 1%
> metaslab 81 offset a2000000000 spacemap 167 free 1.97G
> segments 9206 maxsize 1.89G freepct 1%
> metaslab 82 offset a4000000000 spacemap 168 free 1.98G
> segments 9225 maxsize 1.89G freepct 1%
> metaslab 83 offset a6000000000 spacemap 106 free 1.99G
> segments 24197 maxsize 1.43G freepct 1%
> metaslab 84 offset a8000000000 spacemap 169 free 2.00G
> segments 9637 maxsize 1.91G freepct 1%
> metaslab 85 offset aa000000000 spacemap 170 free 2.01G
> segments 10167 maxsize 1.92G freepct 1%
> metaslab 86 offset ac000000000 spacemap 171 free 2.02G
> segments 12180 maxsize 1.85G freepct 1%
> metaslab 87 offset ae000000000 spacemap 174 free 2.02G
> segments 9716 maxsize 1.93G freepct 1%
> metaslab 88 offset b0000000000 spacemap 175 free 2.04G
> segments 10583 maxsize 1.94G freepct 1%
> metaslab 89 offset b2000000000 spacemap 176 free 2.05G
> segments 9935 maxsize 1.95G freepct 1%
> metaslab 90 offset b4000000000 spacemap 177 free 2.06G
> segments 10459 maxsize 1.96G freepct 1%
> metaslab 91 offset b6000000000 spacemap 178 free 2.07G
> segments 9396 maxsize 1.98G freepct 1%
> metaslab 92 offset b8000000000 spacemap 179 free 2.07G
> segments 8301 maxsize 1.96G freepct 1%
> metaslab 93 offset ba000000000 spacemap 151 free 2.08G
> segments 17800 maxsize 1.52G freepct 1%
> metaslab 94 offset bc000000000 spacemap 181 free 2.10G
> segments 10951 maxsize 2.00G freepct 1%
> metaslab 95 offset be000000000 spacemap 182 free 2.11G
> segments 11002 maxsize 2.01G freepct 1%
> metaslab 96 offset c0000000000 spacemap 183 free 2.12G
> segments 10855 maxsize 2.01G freepct 1%
> metaslab 97 offset c2000000000 spacemap 95 free 2.13G
> segments 13168 maxsize 1.96G freepct 1%
> metaslab 98 offset c4000000000 spacemap 184 free 2.13G
> segments 9408 maxsize 2.04G freepct 1%
> metaslab 99 offset c6000000000 spacemap 185 free 2.15G
> segments 10694 maxsize 2.04G freepct 1%
> metaslab 100 offset c8000000000 spacemap 186 free 2.16G
> segments 10563 maxsize 2.05G freepct 1%
> metaslab 101 offset ca000000000 spacemap 187 free 2.17G
> segments 11059 maxsize 2.06G freepct 1%
> metaslab 102 offset cc000000000 spacemap 188 free 2.18G
> segments 11516 maxsize 2.07G freepct 1%
> metaslab 103 offset ce000000000 spacemap 189 free 2.19G
> segments 10700 maxsize 2.08G freepct 1%
> metaslab 104 offset d0000000000 spacemap 163 free 2.20G
> segments 7869 maxsize 1.77G freepct 1%
> metaslab 105 offset d2000000000 spacemap 131 free 2.22G
> segments 11941 maxsize 2.08G freepct 1%
> metaslab 106 offset d4000000000 spacemap 190 free 9.24G
> segments 11407 maxsize 2.11G freepct 7%
> metaslab 107 offset d6000000000 spacemap 141 free 2.24G
> segments 13038 maxsize 2.00G freepct 1%
> metaslab 108 offset d8000000000 spacemap 192 free 9.7G
> segments 10472 maxsize 2.13G freepct 7%
> metaslab 109 offset da000000000 spacemap 193 free 8.61G
> segments 11389 maxsize 2.14G freepct 6%
> metaslab 110 offset dc000000000 spacemap 194 free 6.34G
> segments 10513 maxsize 2.16G freepct 4%
> metaslab 111 offset de000000000 spacemap 195 free 7.58G
> segments 10168 maxsize 2.17G freepct 5%
> metaslab 112 offset e0000000000 spacemap 197 free 7.14G
> segments 6922 maxsize 2.22G freepct 5%
> metaslab 113 offset e2000000000 spacemap 198 free 2.30G
> segments 6757 maxsize 2.24G freepct 1%
> metaslab 114 offset e4000000000 spacemap 199 free 2.32G
> segments 7008 maxsize 2.26G freepct 1%
> metaslab 115 offset e6000000000 spacemap 200 free 2.33G
> segments 6518 maxsize 2.27G freepct 1%
> metaslab 116 offset e8000000000 spacemap 173 free 2.35G
> segments 13857 maxsize 1.92G freepct 1%
> metaslab 117 offset ea000000000 spacemap 201 free 2.36G
> segments 7124 maxsize 2.30G freepct 1%
> metaslab 118 offset ec000000000 spacemap 202 free 2.37G
> segments 6936 maxsize 2.31G freepct 1%
> metaslab 119 offset ee000000000 spacemap 203 free 2.38G
> segments 6679 maxsize 2.32G freepct 1%
> metaslab 120 offset f0000000000 spacemap 204 free 2.39G
> segments 6818 maxsize 2.34G freepct 1%
> metaslab 121 offset f2000000000 spacemap 205 free 2.41G
> segments 7423 maxsize 2.34G freepct 1%
> metaslab 122 offset f4000000000 spacemap 150 free 2.42G
> segments 15678 maxsize 2.14G freepct 1%
> metaslab 123 offset f6000000000 spacemap 158 free 2.43G
> segments 9980 maxsize 2.28G freepct 1%
> metaslab 124 offset f8000000000 spacemap 180 free 2.45G
> segments 11702 maxsize 1.71G freepct 1%
> metaslab 125 offset fa000000000 spacemap 206 free 2.46G
> segments 7070 maxsize 2.40G freepct 1%
> metaslab 126 offset fc000000000 spacemap 124 free 2.47G
> segments 11485 maxsize 2.31G freepct 1%
> metaslab 127 offset fe000000000 spacemap 128 free 2.49G
> segments 2051 maxsize 2.42G freepct 1%
> metaslab 128 offset 100000000000 spacemap 207 free 2.50G
> segments 7309 maxsize 2.44G freepct 1%
> metaslab 129 offset 102000000000 spacemap 208 free 2.52G
> segments 7151 maxsize 2.46G freepct 1%
> metaslab 130 offset 104000000000 spacemap 209 free 2.52G
> segments 6041 maxsize 2.47G freepct 1%
> metaslab 131 offset 106000000000 spacemap 210 free 2.54G
> segments 6910 maxsize 2.47G freepct 1%
> metaslab 132 offset 108000000000 spacemap 211 free 2.56G
> segments 6816 maxsize 2.50G freepct 1%
> metaslab 133 offset 10a000000000 spacemap 212 free 2.57G
> segments 6182 maxsize 2.51G freepct 2%
> metaslab 134 offset 10c000000000 spacemap 213 free 2.59G
> segments 7541 maxsize 2.52G freepct 2%
> metaslab 135 offset 10e000000000 spacemap 214 free 2.61G
> segments 7810 maxsize 2.53G freepct 2%
> metaslab 136 offset 110000000000 spacemap 215 free 2.62G
> segments 6822 maxsize 2.56G freepct 2%
> metaslab 137 offset 112000000000 spacemap 191 free 9.39G
> segments 10891 maxsize 2.28G freepct 7%
> metaslab 138 offset 114000000000 spacemap 216 free 2.65G
> segments 7295 maxsize 2.58G freepct 2%
> metaslab 139 offset 116000000000 spacemap 217 free 2.67G
> segments 7435 maxsize 2.60G freepct 2%
> metaslab 140 offset 118000000000 spacemap 218 free 2.68G
> segments 6952 maxsize 2.62G freepct 2%
> metaslab 141 offset 11a000000000 spacemap 196 free 2.70G
> segments 5975 maxsize 2.37G freepct 2%
> metaslab 142 offset 11c000000000 spacemap 219 free 2.72G
> segments 7547 maxsize 2.65G freepct 2%
> metaslab 143 offset 11e000000000 spacemap 220 free 2.74G
> segments 7570 maxsize 2.67G freepct 2%
> metaslab 144 offset 120000000000 spacemap 221 free 2.75G
> segments 7321 maxsize 2.69G freepct 2%
> metaslab 145 offset 122000000000 spacemap 172 free 2.77G
> segments 3490 maxsize 2.68G freepct 2%
> metaslab 146 offset 124000000000 spacemap 222 free 2.79G
> segments 7448 maxsize 2.72G freepct 2%
> metaslab 147 offset 126000000000 spacemap 223 free 2.81G
> segments 7309 maxsize 2.74G freepct 2%
> metaslab 148 offset 128000000000 spacemap 224 free 2.82G
> segments 7367 maxsize 2.75G freepct 2%
> metaslab 149 offset 12a000000000 spacemap 225 free 2.84G
> segments 7249 maxsize 2.77G freepct 2%
> metaslab 150 offset 12c000000000 spacemap 226 free 2.86G
> segments 8091 maxsize 2.78G freepct 2%
> metaslab 151 offset 12e000000000 spacemap 153 free 2.88G
> segments 2771 maxsize 2.74G freepct 2%
>
> vdev 1
> metaslabs 105 offset spacemap
> free --------------- ------------------- --------------- -------------
> metaslab 0 offset 0 spacemap 38 free 128M
> segments 1 maxsize 128M freepct 99%
> metaslab 1 offset 8000000 spacemap 39 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 2 offset 10000000 spacemap 42 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 3 offset 18000000 spacemap 41 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 4 offset 20000000 spacemap 51 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 5 offset 28000000 spacemap 50 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 6 offset 30000000 spacemap 49 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 7 offset 38000000 spacemap 48 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 8 offset 40000000 spacemap 47 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 9 offset 48000000 spacemap 46 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 10 offset 50000000 spacemap 45 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 11 offset 58000000 spacemap 59 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 12 offset 60000000 spacemap 62 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 13 offset 68000000 spacemap 61 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 14 offset 70000000 spacemap 67 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 15 offset 78000000 spacemap 66 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 16 offset 80000000 spacemap 74 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 17 offset 88000000 spacemap 73 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 18 offset 90000000 spacemap 75 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 19 offset 98000000 spacemap 78 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 20 offset a0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 21 offset a8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 22 offset b0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 23 offset b8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 24 offset c0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 25 offset c8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 26 offset d0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 27 offset d8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 28 offset e0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 29 offset e8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 30 offset f0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 31 offset f8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 32 offset 100000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 33 offset 108000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 34 offset 110000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 35 offset 118000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 36 offset 120000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 37 offset 128000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 38 offset 130000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 39 offset 138000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 40 offset 140000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 41 offset 148000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 42 offset 150000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 43 offset 158000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 44 offset 160000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 45 offset 168000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 46 offset 170000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 47 offset 178000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 48 offset 180000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 49 offset 188000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 50 offset 190000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 51 offset 198000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 52 offset 1a0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 53 offset 1a8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 54 offset 1b0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 55 offset 1b8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 56 offset 1c0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 57 offset 1c8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 58 offset 1d0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 59 offset 1d8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 60 offset 1e0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 61 offset 1e8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 62 offset 1f0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 63 offset 1f8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 64 offset 200000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 65 offset 208000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 66 offset 210000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 67 offset 218000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 68 offset 220000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 69 offset 228000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 70 offset 230000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 71 offset 238000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 72 offset 240000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 73 offset 248000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 74 offset 250000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 75 offset 258000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 76 offset 260000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 77 offset 268000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 78 offset 270000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 79 offset 278000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 80 offset 280000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 81 offset 288000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 82 offset 290000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 83 offset 298000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 84 offset 2a0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 85 offset 2a8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 86 offset 2b0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 87 offset 2b8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 88 offset 2c0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 89 offset 2c8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 90 offset 2d0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 91 offset 2d8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 92 offset 2e0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 93 offset 2e8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 94 offset 2f0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 95 offset 2f8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 96 offset 300000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 97 offset 308000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 98 offset 310000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 99 offset 318000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 100 offset 320000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 101 offset 328000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 102 offset 330000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 103 offset 338000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 104 offset 340000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
>
> vdev 2
> metaslabs 105 offset spacemap
> free --------------- ------------------- --------------- -------------
> metaslab 0 offset 0 spacemap 37 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 1 offset 8000000 spacemap 40 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 2 offset 10000000 spacemap 44 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 3 offset 18000000 spacemap 43 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 4 offset 20000000 spacemap 58 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 5 offset 28000000 spacemap 57 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 6 offset 30000000 spacemap 56 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 7 offset 38000000 spacemap 55 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 8 offset 40000000 spacemap 54 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 9 offset 48000000 spacemap 53 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 10 offset 50000000 spacemap 52 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 11 offset 58000000 spacemap 60 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 12 offset 60000000 spacemap 64 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 13 offset 68000000 spacemap 63 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 14 offset 70000000 spacemap 69 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 15 offset 78000000 spacemap 68 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 16 offset 80000000 spacemap 72 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 17 offset 88000000 spacemap 71 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 18 offset 90000000 spacemap 76 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 19 offset 98000000 spacemap 79 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 20 offset a0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 21 offset a8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 22 offset b0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 23 offset b8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 24 offset c0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 25 offset c8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 26 offset d0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 27 offset d8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 28 offset e0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 29 offset e8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 30 offset f0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 31 offset f8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 32 offset 100000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 33 offset 108000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 34 offset 110000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 35 offset 118000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 36 offset 120000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 37 offset 128000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 38 offset 130000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 39 offset 138000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 40 offset 140000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 41 offset 148000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 42 offset 150000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 43 offset 158000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 44 offset 160000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 45 offset 168000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 46 offset 170000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 47 offset 178000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 48 offset 180000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 49 offset 188000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 50 offset 190000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 51 offset 198000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 52 offset 1a0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 53 offset 1a8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 54 offset 1b0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 55 offset 1b8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 56 offset 1c0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 57 offset 1c8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 58 offset 1d0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 59 offset 1d8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 60 offset 1e0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 61 offset 1e8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 62 offset 1f0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 63 offset 1f8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 64 offset 200000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 65 offset 208000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 66 offset 210000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 67 offset 218000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 68 offset 220000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 69 offset 228000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 70 offset 230000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 71 offset 238000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 72 offset 240000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 73 offset 248000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 74 offset 250000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 75 offset 258000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 76 offset 260000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 77 offset 268000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 78 offset 270000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 79 offset 278000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 80 offset 280000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 81 offset 288000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 82 offset 290000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 83 offset 298000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 84 offset 2a0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 85 offset 2a8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 86 offset 2b0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 87 offset 2b8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 88 offset 2c0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 89 offset 2c8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 90 offset 2d0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 91 offset 2d8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 92 offset 2e0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 93 offset 2e8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 94 offset 2f0000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 95 offset 2f8000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 96 offset 300000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 97 offset 308000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 98 offset 310000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 99 offset 318000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 100 offset 320000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 101 offset 328000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 102 offset 330000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 103 offset 338000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
> metaslab 104 offset 340000000 spacemap 0 free 128M
> segments 1 maxsize 128M freepct 100%
>
> Dataset mos [META], ID 0, cr_txg 4, 459M, 235 objects
>
> Object lvl iblk dblk dsize lsize %full type
> 0 2 16K 16K 256K 128K 91.80 DMU dnode
> 1 1 16K 16K 19.0K 32K 100.00 object directory
> 2 1 16K 512 0 512 0.00 DSL directory
> 3 1 16K 512 3.00K 512 100.00 DSL props
> 4 1 16K 512 3.00K 512 100.00 DSL directory child map
> 5 1 16K 512 0 512 0.00 DSL directory
> 6 1 16K 512 3.00K 512 100.00 DSL props
> 7 1 16K 512 3.00K 512 100.00 DSL directory child map
> 8 1 16K 512 0 512 0.00 DSL directory
> 9 1 16K 512 3.00K 512 100.00 DSL props
> 10 1 16K 512 3.00K 512 100.00 DSL directory child map
> 11 1 16K 128K 0 128K 0.00 bpobj
> 12 1 16K 512 0 512 0.00 DSL directory
> 13 1 16K 512 3.00K 512 100.00 DSL props
> 14 1 16K 512 3.00K 512 100.00 DSL directory child map
> 15 1 16K 512 0 512 0.00 DSL dataset
> 16 1 16K 512 3.00K 512 100.00 DSL dataset snap map
> 17 1 16K 512 3.00K 512 100.00 DSL deadlist map
> 18 1 16K 512 0 512 0.00 DSL dataset
> 19 1 16K 512 3.00K 512 100.00 DSL deadlist map
> 20 1 16K 128K 0 128K 0.00 bpobj
> 21 1 16K 512 0 512 0.00 DSL dataset
> 22 1 16K 512 3.00K 512 100.00 DSL dataset snap map
> 23 1 16K 512 3.00K 512 100.00 DSL deadlist map
> 24 1 16K 128K 0 128K 0.00 bpobj
> 25 1 16K 512 3.00K 512 100.00 DSL dataset next clones
> 26 1 16K 512 3.00K 512 100.00 DSL dir clones
> 27 1 16K 16K 51.0K 16K 100.00 packed nvlist
> 28 1 16K 16K 51.0K 16K 100.00 bpobj (Z=uncompressed)
> 29 1 16K 128K 16K 128K 100.00 SPA history
> 30 1 16K 16K 6.50K 16K 100.00 packed nvlist
> 31 1 16K 512 3.00K 512 100.00 object array
> 32 1 16K 512 3.00K 512 100.00 object array
> 33 1 16K 512 9.5K 1.50K 100.00 object array
> 34 2 16K 4K 467K 176K 100.00 SPA space map
> 35 2 16K 4K 851K 296K 100.00 SPA space map
> 36 2 16K 4K 275K 88.0K 100.00 SPA space map
> 37 1 16K 4K 0 4K 0.00 SPA space map
> 38 1 16K 4K 3.00K 4K 100.00 SPA space map
> 39 1 16K 4K 0 4K 0.00 SPA space map
> 40 1 16K 4K 0 4K 0.00 SPA space map
> 41 1 16K 4K 0 4K 0.00 SPA space map
> 42 1 16K 4K 0 4K 0.00 SPA space map
> 43 1 16K 4K 0 4K 0.00 SPA space map
> 44 1 16K 4K 0 4K 0.00 SPA space map
> 45 1 16K 4K 0 4K 0.00 SPA space map
> 46 1 16K 4K 0 4K 0.00 SPA space map
> 47 1 16K 4K 3.00K 4K 100.00 SPA space map
> 48 1 16K 4K 0 4K 0.00 SPA space map
> 49 1 16K 4K 0 4K 0.00 SPA space map
> 50 1 16K 4K 0 4K 0.00 SPA space map
> 51 1 16K 4K 0 4K 0.00 SPA space map
> 52 1 16K 4K 0 4K 0.00 SPA space map
> 53 1 16K 4K 0 4K 0.00 SPA space map
> 54 1 16K 4K 3.00K 4K 100.00 SPA space map
> 55 1 16K 4K 0 4K 0.00 SPA space map
> 56 1 16K 4K 0 4K 0.00 SPA space map
> 57 1 16K 4K 0 4K 0.00 SPA space map
> 58 1 16K 4K 0 4K 0.00 SPA space map
> 59 1 16K 4K 0 4K 0.00 SPA space map
> 60 1 16K 4K 0 4K 0.00 SPA space map
> 61 1 16K 4K 0 4K 0.00 SPA space map
> 62 1 16K 4K 0 4K 0.00 SPA space map
> 63 1 16K 4K 0 4K 0.00 SPA space map
> 64 1 16K 4K 0 4K 0.00 SPA space map
> 65 2 16K 4K 291K 92.0K 100.00 SPA space map
> 66 1 16K 4K 0 4K 0.00 SPA space map
> 67 1 16K 4K 0 4K 0.00 SPA space map
> 68 1 16K 4K 0 4K 0.00 SPA space map
> 69 1 16K 4K 0 4K 0.00 SPA space map
> 70 2 16K 4K 311K 100K 100.00 SPA space map
> 71 1 16K 4K 0 4K 0.00 SPA space map
> 72 1 16K 4K 0 4K 0.00 SPA space map
> 73 1 16K 4K 0 4K 0.00 SPA space map
> 74 1 16K 4K 0 4K 0.00 SPA space map
> 75 1 16K 4K 0 4K 0.00 SPA space map
> 76 1 16K 4K 0 4K 0.00 SPA space map
> 77 2 16K 4K 336K 104K 100.00 SPA space map
> 78 1 16K 4K 0 4K 0.00 SPA space map
> 79 1 16K 4K 0 4K 0.00 SPA space map
> 80 2 16K 4K 330K 104K 100.00 SPA space map
> 81 2 16K 4K 333K 104K 100.00 SPA space map
> 82 2 16K 4K 339K 108K 100.00 SPA space map
> 83 2 16K 4K 343K 108K 100.00 SPA space map
> 84 2 16K 4K 368K 116K 100.00 SPA space map
> 85 2 16K 4K 339K 108K 100.00 SPA space map
> 86 2 16K 4K 384K 120K 100.00 SPA space map
> 87 2 16K 4K 1.15M 368K 100.00 SPA space map
> 88 2 16K 4K 346K 108K 100.00 SPA space map
> 89 2 16K 4K 339K 108K 100.00 SPA space map
> 90 2 16K 4K 349K 108K 100.00 SPA space map
> 91 2 16K 4K 339K 108K 100.00 SPA space map
> 92 2 16K 4K 365K 116K 100.00 SPA space map
> 93 2 16K 4K 391K 120K 100.00 SPA space map
> 94 2 16K 4K 867K 268K 100.00 SPA space map
> 95 2 16K 4K 506K 156K 100.00 SPA space map
> 96 2 16K 4K 858K 268K 100.00 SPA space map
> 97 2 16K 4K 490K 152K 100.00 SPA space map
> 98 2 16K 4K 346K 112K 100.00 SPA space map
> 99 2 16K 4K 371K 116K 100.00 SPA space map
> 100 2 16K 4K 362K 112K 100.00 SPA space map
> 101 2 16K 4K 384K 120K 100.00 SPA space map
> 102 2 16K 4K 400K 124K 100.00 SPA space map
> 103 2 16K 4K 522K 160K 100.00 SPA space map
> 104 2 16K 4K 477K 148K 100.00 SPA space map
> 105 2 16K 4K 346K 112K 100.00 SPA space map
> 106 2 16K 4K 774K 240K 100.00 SPA space map
> 107 2 16K 4K 906K 312K 100.00 SPA space map
> 108 2 16K 4K 359K 112K 100.00 SPA space map
> 109 2 16K 4K 387K 120K 100.00 SPA space map
> 110 2 16K 4K 387K 120K 100.00 SPA space map
> 111 2 16K 4K 413K 128K 100.00 SPA space map
> 112 2 16K 4K 375K 120K 100.00 SPA space map
> 113 2 16K 4K 378K 116K 100.00 SPA space map
> 114 2 16K 4K 410K 128K 100.00 SPA space map
> 115 2 16K 4K 400K 124K 100.00 SPA space map
> 116 2 16K 4K 381K 116K 100.00 SPA space map
> 117 2 16K 4K 387K 120K 100.00 SPA space map
> 118 2 16K 4K 397K 124K 100.00 SPA space map
> 119 2 16K 4K 394K 120K 100.00 SPA space map
> 120 2 16K 4K 826K 256K 100.00 SPA space map
> 121 2 16K 4K 407K 124K 100.00 SPA space map
> 122 2 16K 4K 429K 132K 100.00 SPA space map
> 123 2 16K 4K 451K 140K 100.00 SPA space map
> 124 2 16K 4K 461K 144K 100.00 SPA space map
> 125 2 16K 4K 519K 160K 100.00 SPA space map
> 126 2 16K 4K 477K 148K 100.00 SPA space map
> 127 2 16K 4K 605K 188K 100.00 SPA space map
> 128 2 16K 4K 227K 100K 100.00 SPA space map
> 129 2 16K 4K 528K 164K 100.00 SPA space map
> 130 2 16K 4K 426K 132K 100.00 SPA space map
> 131 2 16K 4K 528K 164K 100.00 SPA space map
> 132 2 16K 4K 589K 184K 100.00 SPA space map
> 133 2 16K 4K 605K 188K 100.00 SPA space map
> 134 2 16K 4K 790K 276K 100.00 SPA space map
> 135 2 16K 4K 439K 136K 100.00 SPA space map
> 136 2 16K 4K 423K 132K 100.00 SPA space map
> 137 2 16K 4K 455K 140K 100.00 SPA space map
> 138 2 16K 4K 410K 128K 100.00 SPA space map
> 139 2 16K 4K 528K 164K 100.00 SPA space map
> 141 2 16K 4K 531K 164K 100.00 SPA space map
> 142 2 16K 4K 397K 124K 100.00 SPA space map
> 143 2 16K 4K 439K 136K 100.00 SPA space map
> 144 2 16K 4K 423K 132K 100.00 SPA space map
> 145 2 16K 4K 419K 132K 100.00 SPA space map
> 146 2 16K 4K 477K 148K 100.00 SPA space map
> 147 2 16K 4K 461K 144K 100.00 SPA space map
> 148 2 16K 4K 384K 120K 100.00 SPA space map
> 149 2 16K 4K 403K 124K 100.00 SPA space map
> 150 2 16K 4K 560K 172K 100.00 SPA space map
> 151 2 16K 4K 586K 212K 100.00 SPA space map
> 152 2 16K 4K 407K 128K 100.00 SPA space map
> 153 2 16K 4K 183K 88.0K 100.00 SPA space map
> 154 2 16K 4K 506K 156K 100.00 SPA space map
> 155 2 16K 4K 506K 156K 100.00 SPA space map
> 156 2 16K 4K 432K 136K 100.00 SPA space map
> 157 2 16K 4K 499K 156K 100.00 SPA space map
> 158 2 16K 4K 442K 140K 100.00 SPA space map
> 159 2 16K 4K 448K 140K 100.00 SPA space map
> 160 2 16K 4K 435K 136K 100.00 SPA space map
> 161 2 16K 4K 471K 148K 100.00 SPA space map
> 162 2 16K 4K 435K 140K 100.00 SPA space map
> 163 2 16K 4K 343K 136K 100.00 SPA space map
> 164 2 16K 4K 426K 132K 100.00 SPA space map
> 165 2 16K 4K 365K 124K 100.00 SPA space map
> 166 2 16K 4K 371K 120K 100.00 SPA space map
> 167 2 16K 4K 416K 136K 100.00 SPA space map
> 168 2 16K 4K 426K 132K 100.00 SPA space map
> 169 2 16K 4K 419K 132K 100.00 SPA space map
> 170 2 16K 4K 461K 144K 100.00 SPA space map
> 171 2 16K 4K 541K 156K 100.00 SPA space map
> 172 2 16K 4K 237K 100K 100.00 SPA space map
> 173 2 16K 4K 531K 176K 100.00 SPA space map
> 174 2 16K 4K 426K 136K 100.00 SPA space map
> 175 2 16K 4K 445K 140K 100.00 SPA space map
> 176 2 16K 4K 426K 136K 100.00 SPA space map
> 177 2 16K 4K 451K 140K 100.00 SPA space map
> 178 2 16K 4K 423K 136K 100.00 SPA space map
> 179 2 16K 4K 413K 136K 100.00 SPA space map
> 180 2 16K 4K 439K 164K 100.00 SPA space map
> 181 2 16K 4K 471K 148K 100.00 SPA space map
> 182 2 16K 4K 458K 144K 100.00 SPA space map
> 183 2 16K 4K 471K 148K 100.00 SPA space map
> 184 2 16K 4K 419K 132K 100.00 SPA space map
> 185 2 16K 4K 490K 152K 100.00 SPA space map
> 186 2 16K 4K 445K 140K 100.00 SPA space map
> 187 2 16K 4K 464K 144K 100.00 SPA space map
> 188 2 16K 4K 467K 148K 100.00 SPA space map
> 189 2 16K 4K 451K 140K 100.00 SPA space map
> 190 2 16K 4K 522K 164K 100.00 SPA space map
> 191 2 16K 4K 503K 144K 100.00 SPA space map
> 192 2 16K 4K 544K 156K 100.00 SPA space map
> 193 2 16K 4K 535K 168K 100.00 SPA space map
> 194 2 16K 4K 487K 152K 100.00 SPA space map
> 195 2 16K 4K 429K 136K 100.00 SPA space map
> 196 2 16K 4K 291K 120K 100.00 SPA space map
> 197 2 16K 4K 339K 112K 100.00 SPA space map
> 198 2 16K 4K 359K 124K 100.00 SPA space map
> 199 2 16K 4K 311K 120K 100.00 SPA space map
> 200 2 16K 4K 314K 116K 100.00 SPA space map
> 201 2 16K 4K 336K 128K 100.00 SPA space map
> 202 2 16K 4K 317K 120K 100.00 SPA space map
> 203 2 16K 4K 327K 120K 100.00 SPA space map
> 204 2 16K 4K 346K 124K 100.00 SPA space map
> 205 2 16K 4K 323K 124K 100.00 SPA space map
> 206 2 16K 4K 339K 116K 100.00 SPA space map
> 207 2 16K 4K 349K 124K 100.00 SPA space map
> 208 2 16K 4K 333K 124K 100.00 SPA space map
> 209 2 16K 4K 291K 112K 100.00 SPA space map
> 210 2 16K 4K 336K 120K 100.00 SPA space map
> 211 2 16K 4K 320K 120K 100.00 SPA space map
> 212 2 16K 4K 291K 112K 100.00 SPA space map
> 213 2 16K 4K 349K 128K 100.00 SPA space map
> 214 2 16K 4K 397K 132K 100.00 SPA space map
> 215 2 16K 4K 330K 116K 100.00 SPA space map
> 216 2 16K 4K 320K 120K 100.00 SPA space map
> 217 2 16K 4K 394K 132K 100.00 SPA space map
> 218 2 16K 4K 336K 120K 100.00 SPA space map
> 219 2 16K 4K 327K 124K 100.00 SPA space map
> 220 2 16K 4K 349K 128K 100.00 SPA space map
> 221 2 16K 4K 391K 128K 100.00 SPA space map
> 222 2 16K 4K 339K 124K 100.00 SPA space map
> 223 2 16K 4K 339K 124K 100.00 SPA space map
> 224 2 16K 4K 365K 124K 100.00 SPA space map
> 225 2 16K 4K 346K 124K 100.00 SPA space map
> 226 2 16K 4K 400K 136K 100.00 SPA space map
> 228 3 16K 16K 395M 433M 99.95 persistent error log
> 229 1 16K 4K 0 4K 0.00 SPA space map
> 230 1 16K 4K 0 4K 0.00 SPA space map
> 231 1 16K 4K 0 4K 0.00 SPA space map
> 232 1 16K 4K 0 4K 0.00 SPA space map
> 233 1 16K 4K 0 4K 0.00 SPA space map
> 234 1 16K 4K 0 4K 0.00 SPA space map
> 235 1 16K 4K 0 4K 0.00 SPA space map
> 236 1 16K 4K 0 4K 0.00 SPA space map
> 237 1 16K 4K 0 4K 0.00 SPA space map
>
> Dataset tank2 [ZPL], ID 21, cr_txg 1, 13.3T, 37 objects
>
> ZIL header: claim_txg 0, claim_blk_seq 0, claim_lr_seq 0 replay_seq 0, flags 0x0
>
>
> Object lvl iblk dblk dsize lsize %full type
> 0 7 16K 16K 40.5K 32K 57.81 DMU dnode
> -1 1 16K 512 2K 512 100.00 ZFS user/group used
> -2 1 16K 512 2K 512 100.00 ZFS user/group used
> 1 1 16K 512 2K 512 100.00 ZFS master node
> 2 1 16K 512 2K 512 100.00 SA master node
> 3 1 16K 512 2K 512 100.00 ZFS delete queue
> 4 1 16K 1.50K 2K 1.50K 100.00 ZFS directory
> 5 1 16K 1.50K 2K 1.50K 100.00 SA attr registration
> 6 1 16K 16K 10.5K 32K 100.00 SA attr layouts
> 7 1 16K 512 2K 512 100.00 ZFS directory
> 8 5 16K 128K 510G 510G 100.00 ZFS plain file
> 9 5 16K 128K 476G 476G 100.00 ZFS plain file
> 10 5 16K 128K 473G 473G 100.00 ZFS plain file
> 11 5 16K 128K 467G 467G 100.00 ZFS plain file
> 12 5 16K 128K 428G 428G 100.00 ZFS plain file
> 13 5 16K 128K 455G 455G 100.00 ZFS plain file
> 14 5 16K 128K 478G 478G 100.00 ZFS plain file
> 15 5 16K 128K 517G 517G 100.00 ZFS plain file
> 16 5 16K 128K 487G 487G 100.00 ZFS plain file
> 17 5 16K 128K 513G 513G 100.00 ZFS plain file
> 18 5 16K 128K 489G 489G 100.00 ZFS plain file
> 19 5 16K 128K 494G 493G 100.00 ZFS plain file
> 20 5 16K 128K 492G 492G 100.00 ZFS plain file
> 21 5 16K 128K 488G 487G 100.00 ZFS plain file
> 22 1 16K 1K 2K 1K 100.00 ZFS directory
> 23 4 16K 128K 107G 107G 100.00 ZFS plain file
> 24 4 16K 128K 92.4G 92.4G 100.00 ZFS plain file
> 25 4 16K 128K 97.2G 97.2G 100.00 ZFS plain file
> 26 4 16K 128K 0 128K 0.00 ZFS plain file
> 27 4 16K 128K 149G 149G 100.00 ZFS plain file
> 28 4 16K 128K 221G 221G 100.00 ZFS plain file
> 29 4 16K 128K 93.8G 93.8G 100.00 ZFS plain file
> 30 4 16K 128K 66.0G 66.0G 100.00 ZFS plain file
> 31 5 16K 128K 5.74T 5.74T 100.00 ZFS plain file
> 32 4 16K 128K 48.0G 48.0G 100.00 ZFS plain file
> 33 4 16K 128K 12.0G 12.0G 100.00 ZFS plain file
> 34 4 16K 128K 11.5G 11.5G 100.00 ZFS plain file
> 35 4 16K 128K 11.6G 11.5G 100.00 ZFS plain file
> 36 4 16K 128K 29.9G 29.8G 100.00 ZFS plain file
> 37 1 16K 512 0 512 0.00 ZFS plain file
After a long time waiting zdb started spaffing lots and lots of :-
zdb_blkptr_cb: Got error 122 reading <21, 8, 0, ea072> DVA[0]=<0:23c1e196400:2d000> [L0 ZFS plain file] fletcher4 uncompressed LE
contiguous unique single size=20000L/20000P birth=4507L/4507P fill=1
cksum=3f98d68a2453:fe4561392aca231:c327b800ff4c8ad5:25a834a4ff84cf0e -- skipping
zdb_blkptr_cb: Got error 122 reading <21, 8, 0, ea073> DVA[0]=<0:23c1e169400:2d000> [L0 ZFS plain file] fletcher4 uncompressed LE
contiguous unique single size=20000L/20000P birth=4507L/4507P fill=1
cksum=3ff53073c410:ff578ea3d9add37:56c79527f899fddd:c0f9a2751d3d4fe1 -- skipping
zdb_blkptr_cb: Got error 122 reading <21, 8, 0, ea074> DVA[0]=<0:23c1e1c3400:2d000> [L0 ZFS plain file] fletcher4 uncompressed LE
contiguous unique single size=20000L/20000P birth=4507L/4507P fill=1
cksum=403f20ed2647:100a131a2349650e:76e7f9a70f6d0b24:7329de5d1dfbd68b -- skipping
zdb_blkptr_cb: Got error 122 reading <21, 8, 0, ea075> DVA[0]=<0:23c1e1f0400:2d000> [L0 ZFS plain file] fletcher4 uncompressed LE
contiguous unique single size=20000L/20000P birth=4507L/4507P fill=1
cksum=3fa72784b6f2:fee8a882bc6849b:e90a52ef28fe1baa:8c84eb7b51c6d0ad -- skipping
zdb_blkptr_cb: Got error 122 reading <21, 8, 0, ea076> DVA[0]=<0:23c1e21d400:2d000> [L0 ZFS plain file] fletcher4 uncompressed LE
contiguous unique single size=20000L/20000P birth=4507L/4507P fill=1
cksum=4043be27b6cd:101d869585000e3c:52d438d1f3f14883:1f3e84a837d38701 -- skipping
zdb_blkptr_cb: Got error 122 reading <21, 8, 0, ea077> DVA[0]=<0:23c1e24a400:2d000> [L0 ZFS plain file] fletcher4 uncompressed LE
contiguous unique single size=20000L/20000P birth=4507L/4507P fill=1
cksum=402ba31a743e:10055a4e82d009fc:a6e97a0ebbaf4c80:6e60c0e410cf15e7 -- skipping
Regards
Steve
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9DB937FEA7634C4BAC49EF5823F93CA3>