From owner-freebsd-stable@FreeBSD.ORG Sat Jan 24 13:17:58 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC3E7106566C for ; Sat, 24 Jan 2009 13:17:58 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from warped.bluecherry.net (unknown [IPv6:2001:440:eeee:fffb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 50C1F8FC1F for ; Sat, 24 Jan 2009 13:17:58 +0000 (UTC) (envelope-from morganw@chemikals.org) Received: from volatile.chemikals.org (morganw-1-pt.tunnel.tserv8.dal1.ipv6.he.net [IPv6:2001:470:1f0e:47e::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by warped.bluecherry.net (Postfix) with ESMTPSA id AFD3684CE05A; Sat, 24 Jan 2009 07:17:56 -0600 (CST) Received: from localhost (morganw@localhost [127.0.0.1]) by volatile.chemikals.org (8.14.3/8.14.3) with ESMTP id n0ODHn8R072581; Sat, 24 Jan 2009 07:17:50 -0600 (CST) (envelope-from morganw@chemikals.org) Date: Sat, 24 Jan 2009 07:17:49 -0600 (CST) From: Wes Morgan To: David Ehrmann In-Reply-To: <6e0e5340901240026o39eb4554u6d8eb8c00ee2adbb@mail.gmail.com> Message-ID: References: <6e0e5340901151158n5108ba8ct6af8fb270b10b75b@mail.gmail.com> <6e0e5340901161521t30845197s9529fb5a55dbba13@mail.gmail.com> <6e0e5340901221324o33f1e2b1l53c842ebf9dad9a8@mail.gmail.com> <6e0e5340901240026o39eb4554u6d8eb8c00ee2adbb@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-stable@freebsd.org Subject: Re: zfs drive keeps failing between export and import X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jan 2009 13:17:59 -0000 On Sat, 24 Jan 2009, David Ehrmann wrote: > On Thu, Jan 22, 2009 at 8:07 PM, Wes Morgan wrote: >> On Thu, 22 Jan 2009, David Ehrmann wrote: >> >>> On Fri, Jan 16, 2009 at 3:21 PM, David Ehrmann wrote: >>> >>> In the /dev/ad8.eli that zfs doesn't recognize, I found a 16 byte >>> string that was repeated a lot, but it was also repeated in another >>> place: the good /dev/ad10.eli (though the offsets were different). >>> The other weird thing: the good and bad /dev/ad8.eli look a lot alike: >>> one 16 byte string, then another that gets repeated, then another 16 >>> byte string randomly shows up at 0x200. >> >> The "xxx.eli" devices are the decrypted versions, aren't they? ZFS vdev >> labels and uberblocks occupy the first 512k of the device, and consist of >> virtually identical data, differing only by the GUID that the label claims >> to be and a sha256 checksum... So, decrypted, they should all be very, very >> similar. You could actually use the label from any device in a pool to >> reconstruct the label for any other device. > > Let me clarify one thing: when zfs has problems reading the device, > the data resemble the data when it's read fine, but by resemble, I > don't mean values as much as structure. The values are all wrong, but > if you overlaid hexdumps, both share repeated patterns. That makes me > think it's an encryption problem, but I haven't been able to reproduce > it with other configurations. You might try creating the pool, saving the first 512k of each block device to a file, then export the pool and repeat, then import (or try to). Run zdb on each file and compare the output. From creation to export to import they should only differ by the "state" in the top level of the label nvlist. If the entire label is corrupted, then likely it's a crypto problem. Although, it really sounds like you've been able to eliminate zfs as a culprit.