Date: Thu, 05 Feb 2015 23:08:54 +0100 From: Michelle Sullivan <michelle@sorbs.net> To: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: ZFS pool faulted (corrupt metadata) but the disk data appears ok... Message-ID: <54D3E9F6.20702@sorbs.net>
next in thread | raw e-mail | index | archive | help
Any clues on this? root@colossus:~ # zpool import pool: storage id: 10618504954404185222 state: FAULTED status: The pool metadata is corrupted. action: The pool cannot be imported due to damaged devices or data. The pool may be active on another system, but can be imported using the '-f' flag. see: http://illumos.org/msg/ZFS-8000-72 config: storage FAULTED corrupted data raidz2-0 ONLINE mfid0 ONLINE mfid15 ONLINE mfid1 ONLINE mfid2 ONLINE mfid3 ONLINE mfid4 ONLINE mfid5 ONLINE replacing-7 ONLINE mfid13 ONLINE mfid14 ONLINE mfid6 ONLINE mfid7 ONLINE mfid8 ONLINE mfid9 ONLINE mfid10 ONLINE mfid11 ONLINE mfid12 ONLINE root@colossus:~ # zpool import -Ff storage cannot import 'storage': I/O error Destroy and re-create the pool from a backup source. root@colossus:~ # zdb -l /dev/mfid0 -------------------------------------------- LABEL 0 -------------------------------------------- version: 5000 name: 'storage' state: 0 txg: 1928241 pool_guid: 10618504954404185222 hostid: 4203774842 hostname: 'colossus' top_guid: 12489400212295803034 guid: 3998695725653225547 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 12489400212295803034 nparity: 2 metaslab_array: 34 metaslab_shift: 38 ashift: 9 asize: 45000449064960 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 3998695725653225547 path: '/dev/mfid0' phys_path: '/dev/mfid0' whole_disk: 1 DTL: 168 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 10795471632546545577 path: '/dev/mfid1' phys_path: '/dev/mfid1' whole_disk: 1 DTL: 167 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 15820272272734706674 path: '/dev/mfid2' phys_path: '/dev/mfid2' whole_disk: 1 DTL: 166 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 3928579496187019848 path: '/dev/mfid3' phys_path: '/dev/mfid3' whole_disk: 1 DTL: 165 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 7125052278051590304 path: '/dev/mfid4' phys_path: '/dev/mfid4' whole_disk: 1 DTL: 164 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 14370198745088794709 path: '/dev/mfid5' phys_path: '/dev/mfid5' whole_disk: 1 DTL: 163 create_txg: 4 children[6]: type: 'disk' id: 6 guid: 1843597351388951655 path: '/dev/mfid6' phys_path: '/dev/mfid6' whole_disk: 1 DTL: 162 create_txg: 4 children[7]: type: 'replacing' id: 7 guid: 2914889727426054645 whole_disk: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 10956220251832269421 path: '/dev/mfid15' phys_path: '/dev/mfid15' whole_disk: 1 DTL: 179 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2463756237300743131 path: '/dev/mfid13' phys_path: '/dev/mfid13' whole_disk: 1 DTL: 181 create_txg: 4 resilvering: 1 children[8]: type: 'disk' id: 8 guid: 8864096842672670007 path: '/dev/mfid7' phys_path: '/dev/mfid7' whole_disk: 1 DTL: 160 create_txg: 4 children[9]: type: 'disk' id: 9 guid: 4650681673751655245 path: '/dev/mfid8' phys_path: '/dev/mfid8' whole_disk: 1 DTL: 159 create_txg: 4 children[10]: type: 'disk' id: 10 guid: 8432109430432996813 path: '/dev/mfid9' phys_path: '/dev/mfid9' whole_disk: 1 DTL: 158 create_txg: 4 children[11]: type: 'disk' id: 11 guid: 414941847968750824 path: '/dev/mfid10' phys_path: '/dev/mfid10' whole_disk: 1 DTL: 157 create_txg: 4 children[12]: type: 'disk' id: 12 guid: 7335375930620195352 path: '/dev/mfid11' phys_path: '/dev/mfid11' whole_disk: 1 DTL: 156 create_txg: 4 children[13]: type: 'disk' id: 13 guid: 5100737174610362 path: '/dev/mfid12' phys_path: '/dev/mfid12' whole_disk: 1 DTL: 155 create_txg: 4 children[14]: type: 'disk' id: 14 guid: 15695558693726858796 path: '/dev/mfid14' phys_path: '/dev/mfid14' whole_disk: 1 DTL: 174 create_txg: 4 features_for_read: -------------------------------------------- LABEL 1 -------------------------------------------- version: 5000 name: 'storage' state: 0 txg: 1928241 pool_guid: 10618504954404185222 hostid: 4203774842 hostname: 'colossus' top_guid: 12489400212295803034 guid: 3998695725653225547 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 12489400212295803034 nparity: 2 metaslab_array: 34 metaslab_shift: 38 ashift: 9 asize: 45000449064960 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 3998695725653225547 path: '/dev/mfid0' phys_path: '/dev/mfid0' whole_disk: 1 DTL: 168 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 10795471632546545577 path: '/dev/mfid1' phys_path: '/dev/mfid1' whole_disk: 1 DTL: 167 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 15820272272734706674 path: '/dev/mfid2' phys_path: '/dev/mfid2' whole_disk: 1 DTL: 166 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 3928579496187019848 path: '/dev/mfid3' phys_path: '/dev/mfid3' whole_disk: 1 DTL: 165 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 7125052278051590304 path: '/dev/mfid4' phys_path: '/dev/mfid4' whole_disk: 1 DTL: 164 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 14370198745088794709 path: '/dev/mfid5' phys_path: '/dev/mfid5' whole_disk: 1 DTL: 163 create_txg: 4 children[6]: type: 'disk' id: 6 guid: 1843597351388951655 path: '/dev/mfid6' phys_path: '/dev/mfid6' whole_disk: 1 DTL: 162 create_txg: 4 children[7]: type: 'replacing' id: 7 guid: 2914889727426054645 whole_disk: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 10956220251832269421 path: '/dev/mfid15' phys_path: '/dev/mfid15' whole_disk: 1 DTL: 179 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2463756237300743131 path: '/dev/mfid13' phys_path: '/dev/mfid13' whole_disk: 1 DTL: 181 create_txg: 4 resilvering: 1 children[8]: type: 'disk' id: 8 guid: 8864096842672670007 path: '/dev/mfid7' phys_path: '/dev/mfid7' whole_disk: 1 DTL: 160 create_txg: 4 children[9]: type: 'disk' id: 9 guid: 4650681673751655245 path: '/dev/mfid8' phys_path: '/dev/mfid8' whole_disk: 1 DTL: 159 create_txg: 4 children[10]: type: 'disk' id: 10 guid: 8432109430432996813 path: '/dev/mfid9' phys_path: '/dev/mfid9' whole_disk: 1 DTL: 158 create_txg: 4 children[11]: type: 'disk' id: 11 guid: 414941847968750824 path: '/dev/mfid10' phys_path: '/dev/mfid10' whole_disk: 1 DTL: 157 create_txg: 4 children[12]: type: 'disk' id: 12 guid: 7335375930620195352 path: '/dev/mfid11' phys_path: '/dev/mfid11' whole_disk: 1 DTL: 156 create_txg: 4 children[13]: type: 'disk' id: 13 guid: 5100737174610362 path: '/dev/mfid12' phys_path: '/dev/mfid12' whole_disk: 1 DTL: 155 create_txg: 4 children[14]: type: 'disk' id: 14 guid: 15695558693726858796 path: '/dev/mfid14' phys_path: '/dev/mfid14' whole_disk: 1 DTL: 174 create_txg: 4 features_for_read: -------------------------------------------- LABEL 2 -------------------------------------------- version: 5000 name: 'storage' state: 0 txg: 1928241 pool_guid: 10618504954404185222 hostid: 4203774842 hostname: 'colossus' top_guid: 12489400212295803034 guid: 3998695725653225547 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 12489400212295803034 nparity: 2 metaslab_array: 34 metaslab_shift: 38 ashift: 9 asize: 45000449064960 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 3998695725653225547 path: '/dev/mfid0' phys_path: '/dev/mfid0' whole_disk: 1 DTL: 168 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 10795471632546545577 path: '/dev/mfid1' phys_path: '/dev/mfid1' whole_disk: 1 DTL: 167 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 15820272272734706674 path: '/dev/mfid2' phys_path: '/dev/mfid2' whole_disk: 1 DTL: 166 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 3928579496187019848 path: '/dev/mfid3' phys_path: '/dev/mfid3' whole_disk: 1 DTL: 165 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 7125052278051590304 path: '/dev/mfid4' phys_path: '/dev/mfid4' whole_disk: 1 DTL: 164 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 14370198745088794709 path: '/dev/mfid5' phys_path: '/dev/mfid5' whole_disk: 1 DTL: 163 create_txg: 4 children[6]: type: 'disk' id: 6 guid: 1843597351388951655 path: '/dev/mfid6' phys_path: '/dev/mfid6' whole_disk: 1 DTL: 162 create_txg: 4 children[7]: type: 'replacing' id: 7 guid: 2914889727426054645 whole_disk: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 10956220251832269421 path: '/dev/mfid15' phys_path: '/dev/mfid15' whole_disk: 1 DTL: 179 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2463756237300743131 path: '/dev/mfid13' phys_path: '/dev/mfid13' whole_disk: 1 DTL: 181 create_txg: 4 resilvering: 1 children[8]: type: 'disk' id: 8 guid: 8864096842672670007 path: '/dev/mfid7' phys_path: '/dev/mfid7' whole_disk: 1 DTL: 160 create_txg: 4 children[9]: type: 'disk' id: 9 guid: 4650681673751655245 path: '/dev/mfid8' phys_path: '/dev/mfid8' whole_disk: 1 DTL: 159 create_txg: 4 children[10]: type: 'disk' id: 10 guid: 8432109430432996813 path: '/dev/mfid9' phys_path: '/dev/mfid9' whole_disk: 1 DTL: 158 create_txg: 4 children[11]: type: 'disk' id: 11 guid: 414941847968750824 path: '/dev/mfid10' phys_path: '/dev/mfid10' whole_disk: 1 DTL: 157 create_txg: 4 children[12]: type: 'disk' id: 12 guid: 7335375930620195352 path: '/dev/mfid11' phys_path: '/dev/mfid11' whole_disk: 1 DTL: 156 create_txg: 4 children[13]: type: 'disk' id: 13 guid: 5100737174610362 path: '/dev/mfid12' phys_path: '/dev/mfid12' whole_disk: 1 DTL: 155 create_txg: 4 children[14]: type: 'disk' id: 14 guid: 15695558693726858796 path: '/dev/mfid14' phys_path: '/dev/mfid14' whole_disk: 1 DTL: 174 create_txg: 4 features_for_read: -------------------------------------------- LABEL 3 -------------------------------------------- version: 5000 name: 'storage' state: 0 txg: 1928241 pool_guid: 10618504954404185222 hostid: 4203774842 hostname: 'colossus' top_guid: 12489400212295803034 guid: 3998695725653225547 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 12489400212295803034 nparity: 2 metaslab_array: 34 metaslab_shift: 38 ashift: 9 asize: 45000449064960 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 3998695725653225547 path: '/dev/mfid0' phys_path: '/dev/mfid0' whole_disk: 1 DTL: 168 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 10795471632546545577 path: '/dev/mfid1' phys_path: '/dev/mfid1' whole_disk: 1 DTL: 167 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 15820272272734706674 path: '/dev/mfid2' phys_path: '/dev/mfid2' whole_disk: 1 DTL: 166 create_txg: 4 children[3]: type: 'disk' id: 3 guid: 3928579496187019848 path: '/dev/mfid3' phys_path: '/dev/mfid3' whole_disk: 1 DTL: 165 create_txg: 4 children[4]: type: 'disk' id: 4 guid: 7125052278051590304 path: '/dev/mfid4' phys_path: '/dev/mfid4' whole_disk: 1 DTL: 164 create_txg: 4 children[5]: type: 'disk' id: 5 guid: 14370198745088794709 path: '/dev/mfid5' phys_path: '/dev/mfid5' whole_disk: 1 DTL: 163 create_txg: 4 children[6]: type: 'disk' id: 6 guid: 1843597351388951655 path: '/dev/mfid6' phys_path: '/dev/mfid6' whole_disk: 1 DTL: 162 create_txg: 4 children[7]: type: 'replacing' id: 7 guid: 2914889727426054645 whole_disk: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 10956220251832269421 path: '/dev/mfid15' phys_path: '/dev/mfid15' whole_disk: 1 DTL: 179 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2463756237300743131 path: '/dev/mfid13' phys_path: '/dev/mfid13' whole_disk: 1 DTL: 181 create_txg: 4 resilvering: 1 children[8]: type: 'disk' id: 8 guid: 8864096842672670007 path: '/dev/mfid7' phys_path: '/dev/mfid7' whole_disk: 1 DTL: 160 create_txg: 4 children[9]: type: 'disk' id: 9 guid: 4650681673751655245 path: '/dev/mfid8' phys_path: '/dev/mfid8' whole_disk: 1 DTL: 159 create_txg: 4 children[10]: type: 'disk' id: 10 guid: 8432109430432996813 path: '/dev/mfid9' phys_path: '/dev/mfid9' whole_disk: 1 DTL: 158 create_txg: 4 children[11]: type: 'disk' id: 11 guid: 414941847968750824 path: '/dev/mfid10' phys_path: '/dev/mfid10' whole_disk: 1 DTL: 157 create_txg: 4 children[12]: type: 'disk' id: 12 guid: 7335375930620195352 path: '/dev/mfid11' phys_path: '/dev/mfid11' whole_disk: 1 DTL: 156 create_txg: 4 children[13]: type: 'disk' id: 13 guid: 5100737174610362 path: '/dev/mfid12' phys_path: '/dev/mfid12' whole_disk: 1 DTL: 155 create_txg: 4 children[14]: type: 'disk' id: 14 guid: 15695558693726858796 path: '/dev/mfid14' phys_path: '/dev/mfid14' whole_disk: 1 DTL: 174 create_txg: 4 features_for_read: LSI9260-16i raid controller all drives in single disk RAID0, system wasn't busy (=in use for writing, however was resilvering a replacement drive after the HSP had taken over due to failed drive) and it got rebooted... faulted on reboot. Recoverable? Regards, -- Michelle Sullivan http://www.mhix.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54D3E9F6.20702>