From owner-freebsd-stable@FreeBSD.ORG Thu Jan 22 21:24:18 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E392106564A for ; Thu, 22 Jan 2009 21:24:18 +0000 (UTC) (envelope-from ehrmann@gmail.com) Received: from yx-out-2324.google.com (yx-out-2324.google.com [74.125.44.28]) by mx1.freebsd.org (Postfix) with ESMTP id CD2768FC1E for ; Thu, 22 Jan 2009 21:24:17 +0000 (UTC) (envelope-from ehrmann@gmail.com) Received: by yx-out-2324.google.com with SMTP id 8so1848492yxb.13 for ; Thu, 22 Jan 2009 13:24:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=OyL2LS3IKle/C6+bHREMpnylDnuuz1/FhytqXHkATpU=; b=uo/wgLpgsNxAC7dqm36Sre+ey2/cNzziRzoR26WawUJireULzJTwIm9j3/tKaFin4q bN3CGwMYJuZK2tRTfOD18RfIox45CIpl8GVa12UfGHkYEtIB1xgb9SjyXhWdO/q1+9v4 XFF+BS6t9F9ZvRPnEAxkTbgZsH39wV6HjgBls= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=io2WvLxQ6SsgI3kn/VR8bmO3TU8njlXxdoV1aYAsX8pS3TSlXfzQtSAmIUw9MdWgHj pChbR6orRxuXObSkPydcweFjGMqwh+3uBkLjEFWHt6kFp07ghyrgfBZmW2Hq9yYKsKEc PIaA+56Dh1Kmy1lROWiMh1PxHWZXjTdtxcUac= MIME-Version: 1.0 Received: by 10.100.164.20 with SMTP id m20mr221531ane.97.1232659457230; Thu, 22 Jan 2009 13:24:17 -0800 (PST) In-Reply-To: <6e0e5340901161521t30845197s9529fb5a55dbba13@mail.gmail.com> References: <6e0e5340901151158n5108ba8ct6af8fb270b10b75b@mail.gmail.com> <6e0e5340901161521t30845197s9529fb5a55dbba13@mail.gmail.com> Date: Thu, 22 Jan 2009 13:24:17 -0800 Message-ID: <6e0e5340901221324o33f1e2b1l53c842ebf9dad9a8@mail.gmail.com> From: David Ehrmann To: Pete French Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: zfs drive keeps failing between export and import X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2009 21:24:18 -0000 On Fri, Jan 16, 2009 at 3:21 PM, David Ehrmann wrote: > On Fri, Jan 16, 2009 at 3:33 AM, Pete French > wrote: >>> a software problem before hardware. Both drives are encrypted geli >>> devices. I tried to reproduce the error with 1GB disk images (vs >> >> This is probably a silly question, but are you sure that the drives >> are not auto detaching ? I had big problems with a zfs mirror on top >> of geli which turned out to be that drives mounted using "geli_devices" >> in rc.conf will auto detach unless you set "geli_autodetach" to NO. > > Not silly at all. I didn't know that could be an issue, but they > weren't mounted with "geli_devices," they were mounted by hand with > "geli attach /dev/ad." I did not set the -d flag on attach, and > I don't think I used the -l flag on detach, either. Listing the > device says this: > > Geom name: ad10.eli > EncryptionAlgorithm: AES-CBC > KeyLength: 128 > Crypto: hardware > UsedKey: 0 > Flags: NONE > > (and more stuff) > > One more interesting thing: I accidentally rebooted the system without > any detaching/exporting (it involved a different, bad drive). When it > came up, I was able to re-import tank without any problems. > Ok, here's where it gets interesting: The next time I saw the import error, I ran zdb -l on the actual dev. It couldn't find the labels. So I used dd to grab the first 4k of the .eli device and the actual device. Once I got it working, I repeated. The data in the first 4k of /dev/ad8 were all 0x00 both times. I'm guessing this is reserved, or something. The data in the first 4k of /dev/ad8.eli differed between runs (so zdb -l is probably right about not finding the label). In the /dev/ad8.eli that zfs doesn't recognize, I found a 16 byte string that was repeated a lot, but it was also repeated in another place: the good /dev/ad10.eli (though the offsets were different). The other weird thing: the good and bad /dev/ad8.eli look a lot alike: one 16 byte string, then another that gets repeated, then another 16 byte string randomly shows up at 0x200. Why the same data appear in the bad ad8.eli as the good ad10.eli, I'm not sure (I do have the same password and no keyfile with geli), but the patterns of data looking the same make me think something's wrong with the encryption. It's using 128 bit AES-CBC, and these patterns would not be hidden by it (128 bits == 16 bytes). I'm using a Via C7 CPU's padlock cryptographic accelerator, and geli reports this. I'm guessing this is either a padlock or a geli bug. I can't reliably reproduce this problem, but doing it with padlock off might be a good test.