From owner-freebsd-stable@FreeBSD.ORG Sat Jan 24 04:57:38 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F6401065672 for ; Sat, 24 Jan 2009 04:57:38 +0000 (UTC) (envelope-from ehrmann@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.240]) by mx1.freebsd.org (Postfix) with ESMTP id 23EF88FC13 for ; Sat, 24 Jan 2009 04:57:38 +0000 (UTC) (envelope-from ehrmann@gmail.com) Received: by an-out-0708.google.com with SMTP id b38so753995ana.13 for ; Fri, 23 Jan 2009 20:57:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=cJ2u46faDA+MWW3uloMR9phzOG52jJ07xk+0VigtTfM=; b=fnA0fF0fUQsozkMwHQICdEgCLWMT6CrRmROC4+EhRFVgJwInqFLprR76hn55YrO3Qd fyZq9GI83NDAtUfkiBF9kyYO5xEEGN01H1Yn1q2EQQTQP5Sy23Ts9GUSDVkjHTG4XU5E 6IYXoAPIqizMkRCy1MI/b57Wg3e5p5I1ueLYc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Ikb0vS56r7iMzM4njXKJEVNq5HUHPmQwsHGcII4/WPLNNnD3zXbHdBaPk37dbzdEng H+I8MG9VoJIq5cmtHDdaDhyNKSBBZ87EfK8i3qEab1zQbR1DmxYUuCZN661No6ahfzmR uyiHo5RI3sZ0cc1MuW5TOhPln5sdeQ/Jt6w8c= MIME-Version: 1.0 Received: by 10.100.93.17 with SMTP id q17mr307168anb.104.1232773057637; Fri, 23 Jan 2009 20:57:37 -0800 (PST) In-Reply-To: <6e0e5340901222112x159409c5xd2fd93e32b020c0f@mail.gmail.com> References: <6e0e5340901151158n5108ba8ct6af8fb270b10b75b@mail.gmail.com> <6e0e5340901161521t30845197s9529fb5a55dbba13@mail.gmail.com> <6e0e5340901221324o33f1e2b1l53c842ebf9dad9a8@mail.gmail.com> <1de79840901221745r4149dc30yfcfcb8c8a24ad8ce@mail.gmail.com> <6e0e5340901222108i6a5e300fte7fdd1a517fbe049@mail.gmail.com> <6e0e5340901222112x159409c5xd2fd93e32b020c0f@mail.gmail.com> Date: Fri, 23 Jan 2009 20:57:37 -0800 Message-ID: <6e0e5340901232057p4f696455lbbe0bb4e38248837@mail.gmail.com> From: David Ehrmann To: Michael Proto Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Pete French Subject: Re: zfs drive keeps failing between export and import X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jan 2009 04:57:38 -0000 On Thu, Jan 22, 2009 at 9:12 PM, David Ehrmann wrote: > On Thu, Jan 22, 2009 at 9:08 PM, David Ehrmann wrote: >> On Thu, Jan 22, 2009 at 5:45 PM, Michael Proto wrote: >>> On Thu, Jan 22, 2009 at 4:24 PM, David Ehrmann wrote: >>>> On Fri, Jan 16, 2009 at 3:21 PM, David Ehrmann wrote: >>>>> On Fri, Jan 16, 2009 at 3:33 AM, Pete French >>>>> wrote: >>>>>>> a software problem before hardware. Both drives are encrypted geli >>>>>>> devices. I tried to reproduce the error with 1GB disk images (vs >>>>>> >>>>>> This is probably a silly question, but are you sure that the drives >>>>>> are not auto detaching ? I had big problems with a zfs mirror on top >>>>>> of geli which turned out to be that drives mounted using "geli_devices" >>>>>> in rc.conf will auto detach unless you set "geli_autodetach" to NO. >>>>> >>>>> Not silly at all. I didn't know that could be an issue, but they >>>>> weren't mounted with "geli_devices," they were mounted by hand with >>>>> "geli attach /dev/ad." I did not set the -d flag on attach, and >>>>> I don't think I used the -l flag on detach, either. Listing the >>>>> device says this: >>>>> >>>>> Geom name: ad10.eli >>>>> EncryptionAlgorithm: AES-CBC >>>>> KeyLength: 128 >>>>> Crypto: hardware >>>>> UsedKey: 0 >>>>> Flags: NONE >>>>> >>>>> (and more stuff) >>>>> >>>>> One more interesting thing: I accidentally rebooted the system without >>>>> any detaching/exporting (it involved a different, bad drive). When it >>>>> came up, I was able to re-import tank without any problems. >>>>> >>>> >>>> Ok, here's where it gets interesting: >>>> >>>> The next time I saw the import error, I ran zdb -l on the actual dev. >>>> It couldn't find the labels. So I used dd to grab the first 4k of the >>>> .eli device and the actual device. Once I got it working, I repeated. >>>> The data in the first 4k of /dev/ad8 were all 0x00 both times. I'm >>>> guessing this is reserved, or something. The data in the first 4k of >>>> /dev/ad8.eli differed between runs (so zdb -l is probably right about >>>> not finding the label). >>>> >>>> In the /dev/ad8.eli that zfs doesn't recognize, I found a 16 byte >>>> string that was repeated a lot, but it was also repeated in another >>>> place: the good /dev/ad10.eli (though the offsets were different). >>>> The other weird thing: the good and bad /dev/ad8.eli look a lot alike: >>>> one 16 byte string, then another that gets repeated, then another 16 >>>> byte string randomly shows up at 0x200. >>>> >>>> Why the same data appear in the bad ad8.eli as the good ad10.eli, I'm >>>> not sure (I do have the same password and no keyfile with geli), but >>>> the patterns of data looking the same make me think something's wrong >>>> with the encryption. It's using 128 bit AES-CBC, and these patterns >>>> would not be hidden by it (128 bits == 16 bytes). >>>> >>>> I'm using a Via C7 CPU's padlock cryptographic accelerator, and geli >>>> reports this. I'm guessing this is either a padlock or a geli bug. >>>> >>>> I can't reliably reproduce this problem, but doing it with padlock off >>>> might be a good test. >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >>>> >>> >>> I saw something similar (minus zfs) when I was playing with padlock >>> and geli on my C7-Esther fileserver. When trying to mount a geli >>> partition I'd intermittently get a bad decryption key error. Run the >>> same command again to mount the partition and it'd work fine. This was >>> using both password and key-file operations. IIRC when I disabled >>> padlock acceleration it worked fine in my limited testing. That was >>> 6.4, now that I'm on 7.1 it might be worth looking at again. >> >> I just got around to trying it without padlock. I tried to replicate >> the problem 5 or 6 times, but no luck. >> >> This is 7.1. >> >> It *sounds* like a padlock problem, but I'd like to see it make the >> same mistake with a file or memory backed md device. Anyway, that >> this point, I can pretty much rule out zfs as the culprit. >> > > Or geli... Any success (not intermittent) reports with a hifn or > broadcom accelerator and geli? > I wasn't able to reproduce the problem with two ~100MB disk-backed md devices and just geli. Next up is a real disk.