From owner-freebsd-questions@FreeBSD.ORG Fri May 21 06:12:06 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B333106564A for ; Fri, 21 May 2010 06:12:06 +0000 (UTC) (envelope-from lordofhyphens@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id F38C08FC0A for ; Fri, 21 May 2010 06:12:05 +0000 (UTC) Received: by gwj16 with SMTP id 16so396363gwj.13 for ; Thu, 20 May 2010 23:12:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=9nSbvYdazN6+BdR7dpNuAR35KNSMj2XF993NABpfOB8=; b=E+7kVGGHYxw+R3CjBNIaAOOvmypFfl3g265wRzXNDmsdLl1SGrmi0Oy8jxW/4WCrmb 4NxOywm4kgliP8qBe1pkKcyGWB/cTiaeMMTudphuSforGkrS9lMz72lVk7DS/BXQ4l9e ohX6Hhtj2HlvbyTJzaMz7/usEPDevuGZsmLsE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; b=nMfxofGmNvG0BgRqOKAzQqXrP7aRcY1bLc/kSUpY/S3fpQfRAvmaiwwvAk/bplIxdN Ts/S3tPPrnnDMbq7f2o2Y7KHSwxHZH/3kXR1Yfs+MaxhFCfM3dQPsmDzvi58i0RYRupL j6uxCU0fHn+EHRtzL8mcjbIZxv4Z9lelVEDVU= Received: by 10.150.62.14 with SMTP id k14mr3016856yba.35.1274422324902; Thu, 20 May 2010 23:12:04 -0700 (PDT) Received: from [192.168.0.250] (173-24-125-201.client.mchsi.com [173.24.125.201]) by mx.google.com with ESMTPS id w18sm16490109ybe.10.2010.05.20.23.12.03 (version=SSLv3 cipher=RC4-MD5); Thu, 20 May 2010 23:12:03 -0700 (PDT) Message-ID: <4BF6242F.8010801@gmail.com> Date: Fri, 21 May 2010 01:11:59 -0500 From: Joseph Lenox User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.9) Gecko/20100317 Lightning/1.0b1 Thunderbird/3.0.4 MIME-Version: 1.0 To: "freebsd-questions@freebsd.org" References: <4BF4D41E.3000107@gmail.com> In-Reply-To: <4BF4D41E.3000107@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Zpool import failure, metadata checksum fails 8.0-RELEASE X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2010 06:12:06 -0000 Update -- Updated to 8.0-STABLE (tagged 8.1-PRERELEASE), zpool is sitting at tx->tx state (according to top). The root cause of this was apparently an enabled write cache on the sata controller (an adaptec model, can handle its own RAID5)+inopportune power failure; I've disabled it going forward. I've done some searching through the 'net and haven't found any useful info for this type of situation--apparently Solaris just flags the pool as having soft errors and the user is expected to move on from there. As far as I can tell, the pool MUST be imported to perform any other kind of operation on it--you can't even destroy the pool. I don't know if recreating the pool from the underlying system destroys the data. I'm going to leave it for 24 hours and double check to make sure the process is truly stuck (although I'm pretty sure it is). The current system install is a rebuild on different media--I do have the original HDD with the OS install on it, but any attempts to access the pool on that drive gives the same symptoms. --Joseph Lenox Command: /sbin/zpool import -p failmode=panic -f valkyrie PID username THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND #### root 1 44 0 15668K 1936K tx->tx 1 0:00 0.00% zpool /var/log/messages excerpt: May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=494926778368 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=494926778368 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=494926778368 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=494926778368 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=494926778880 size=512 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=494926778880 size=512 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=129568120320 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=129568120320 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=129568120320 size=1024 May 21 00:58:31 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=129568120320 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=129568120320 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=129568120320 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=289968832000 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=289968832000 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=289968832512 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=289968832512 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=289968832512 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=289968832512 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=494926778368 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=494926778368 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=494926778368 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=494926778368 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=494926778880 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=494926778880 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=129568120320 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=129568120320 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=129568120320 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=129568120320 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=129568120320 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=129568120320 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd4 offset=289968832000 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd5 offset=289968832000 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd0 offset=289968832512 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd2 offset=289968832512 size=1024 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd1 offset=289968832512 size=512 May 21 00:58:32 silmeria root: ZFS: checksum mismatch, zpool=valkyrie path=/dev/aacd3 offset=289968832512 size=512 May 21 00:58:32 silmeria root: ZFS: zpool I/O failure, zpool=valkyrie error=86 May 21 00:58:32 silmeria root: ZFS: vdev I/O failure, zpool=valkyrie path= offset= size= error= On 5/20/2010 1:18 AM, Joseph Lenox wrote: > Hello, all-- > > An unexpected powercycle apparently introduced (thanks to my system's > RAID controller) metadata checksum errors on the system. Attempts to > import that pool on any system hangs the command (such that it cannot > even be killed). > > I tried pulling out the OpenSolaris (2009.07) cd that I had on-hand, > but the livecd couldn't find any of the pools in the system--so no > help there. > > I'm certain a scrub will fix the metadata problems (with or without > dataloss, the data isn't critical enough to worry about losing a file > or ten -- losing everything is more of a pain). > > Anyone have any ideas for how I can get this pool fixed? I'm working > on getting 8.0-STABLE sources downloaded (via cvsup) for testing. > > --Joseph Lenox