From owner-freebsd-fs@freebsd.org Wed May 29 13:22:27 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4C8815C2B9E for ; Wed, 29 May 2019 13:22:26 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F09D56C118 for ; Wed, 29 May 2019 13:22:25 +0000 (UTC) (envelope-from mike@sentex.net) Received: by mailman.ysv.freebsd.org (Postfix) id AA7E515C2B9D; Wed, 29 May 2019 13:22:25 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8577D15C2B9C for ; Wed, 29 May 2019 13:22:25 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene.sentex.ca (unknown [IPv6:2607:f3e0:0:3::18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B0446C114 for ; Wed, 29 May 2019 13:22:25 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] ([192.168.43.26]) by pyroxene.sentex.ca (8.15.2/8.15.2) with ESMTP id x4TDMLo3048821; Wed, 29 May 2019 09:22:22 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: Crashed ZFS To: =?UTF-8?B?0JDQu9C10LrRgdCw0L3QtNGAINCf0L7QstC+0LvQvtGG0LrQuNC5?= , fs@freebsd.org References: From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= mQENBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAG0HG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT6JATgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnm5AQ0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAYkBHwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: <460609b1-35a6-6bfc-318c-bd796a3e3239@sentex.net> Date: Wed, 29 May 2019 09:22:18 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 1B0446C114 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 May 2019 13:22:27 -0000 I would wait for a few more people to chime with what to do, but I had a similar issue (same error IIRC) last week after physically moving the disks to a new controller.  I did zpool clear -F zpool export zpool import The clear gave an error but after the export / import, it came back online.  A scrub was done, but showed no errors. Good luck!     ---Mike On 5/29/2019 7:28 AM, Александр Поволоцкий wrote: > Hello > > After power surge, one of my zpools yields errors > > |root@stor:/home/tarkhil # zpool status -v big_fast_one|| > ||  pool: big_fast_one|| > || state: FAULTED|| > ||status: The pool metadata is corrupted and the pool cannot be opened.|| > ||action: Recovery is possible, but will result in some data loss.|| > ||        Returning the pool to its state as of Tue May 28 02:00:35 > 2019|| > ||        should correct the problem.  Approximately 5 seconds of data|| > ||        must be discarded, irreversibly.  Recovery can be attempted|| > ||        by executing 'zpool clear -F big_fast_one'. A scrub of the > pool|| > ||        is strongly recommended after recovery.|| > ||   see: http://illumos.org/msg/ZFS-8000-72|| > ||  scan: none requested|| > ||config:|| > || > ||        NAME              STATE     READ WRITE CKSUM|| > ||        big_fast_one      FAULTED      0     0     1|| > ||          raidz1-0        ONLINE       0     0     7|| > ||            gpt/ZA21TJA7  ONLINE       0     0     0|| > ||            gpt/ZA21P6JQ  ONLINE       0     0     0|| > ||            gpt/ZA21PJZY  ONLINE       0     0     0|| > ||            gpt/ZA21T6L6  ONLINE       0     0     0|| > ||            gpt/ZA21TN3R  ONLINE       0     0     0|| > | > > |root@stor:/home/tarkhil # zpool clear -Fn big_fast_one|| > ||internal error: out of memory||| > > while there are plenty of RAM|(96 Gb)| > > |gpart shows everything OK| > > |root@stor:/home/tarkhil # zdb -AAA -L -e big_fast_one > > Configuration for import: >         vdev_children: 1 >         version: 5000 >         pool_guid: 4972776226197917949 >         name: 'big_fast_one' >         state: 0 >         hostid: 773241384 >         hostname: 'stor.inf.sudo.su' >         vdev_tree: >             type: 'root' >             id: 0 >             guid: 4972776226197917949 >             children[0]: >                 type: 'raidz' >                 id: 0 >                 guid: 58821498572043303 >                 nparity: 1 >                 metaslab_array: 41 >                 metaslab_shift: 38 >                 ashift: 12 >                 asize: 50004131840000 >                 is_log: 0 >                 create_txg: 4 >                 children[0]: >                     type: 'disk' >                     id: 0 >                     guid: 13318923208485210326 >                     phys_path: > 'id1,enc@n50030480005d387f/type@0/slot@e/elmdesc@013/p1' >                     whole_disk: 1 >                     DTL: 57 >                     create_txg: 4 >                     path: '/dev/gpt/ZA21TJA7' >                 children[1]: >                     type: 'disk' >                     id: 1 >                     guid: 5421240647062683539 >                     phys_path: > 'id1,enc@n50030480005d387f/type@0/slot@1/elmdesc@000/p1' >                     whole_disk: 1 >                     DTL: 56 >                     create_txg: 4 >                     path: '/dev/gpt/ZA21P6JQ' >                 children[2]: >                     type: 'disk' >                     id: 2 >                     guid: 17788210514601115893 >                     phys_path: > 'id1,enc@n50030480005d387f/type@0/slot@5/elmdesc@004/p1' >                     whole_disk: 1 >                     DTL: 55 >                     create_txg: 4 >                     path: '/dev/gpt/ZA21PJZY' >                 children[3]: >                     type: 'disk' >                     id: 3 >                     guid: 11411950711187621765 >                     phys_path: > 'id1,enc@n50030480005d387f/type@0/slot@9/elmdesc@008/p1' >                     whole_disk: 1 >                     DTL: 54 >                     create_txg: 4 >                     path: '/dev/gpt/ZA21T6L6' >                 children[4]: >                     type: 'disk' >                     id: 4 >                     guid: 6486033012937503138 >                     phys_path: > 'id1,enc@n50030480005d387f/type@0/slot@d/elmdesc@012/p1' >                     whole_disk: 1 >                     DTL: 52 >                     create_txg: 4 >                     path: '/dev/gpt/ZA21TN3R' > zdb: can't open 'big_fast_one': File exists > > ZFS_DBGMSG(zdb): > | > > |root@stor:/home/tarkhil # zdb -AAA -L -u -e big_fast_one > zdb: can't open 'big_fast_one': File exists > root@stor:/home/tarkhil # zdb -AAA -L -d -e big_fast_one > zdb: can't open 'big_fast_one': File exists > root@stor:/home/tarkhil # zdb -AAA -L -h -e big_fast_one > zdb: can't open 'big_fast_one': File exists > | > > |What should I do? Export and import? Rename zpool.cache and import > (it's a remote box, I cannot afford another 3 hours to and from it)? > Something else?| > > |--| > > |Alex > | > > > > > --- > Это сообщение проверено на вирусы антивирусом Avast. > https://www.avast.com/antivirus > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada