From owner-freebsd-fs@FreeBSD.ORG Thu Oct 4 20:59:14 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 019A11065672 for ; Thu, 4 Oct 2012 20:59:14 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 832448FC08 for ; Thu, 4 Oct 2012 20:59:12 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id 16so797930wgi.31 for ; Thu, 04 Oct 2012 13:59:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=rI0bpUvstetRIOAl72uzXQ3QHp3cxC0Z4qZIZZ+Th+U=; b=ywsKjJTSJFTir1UhipGalqi5YnSQnz4WsCn/5awKB9hkYWHGZ7N1Z6+/pf+OLjO9TS 2P7HKS4V0KUEevNXe/2z433sUqSPVqgkdzY59XVda6YL/oSzQeKloIberVH3/6nrGk0u I/oOYOULor4c0bK0C+oCYoJ7OQWssLsT6whzEnprJaItRW0j5+56OyWgD1pSX1YdrT/U JDYNXKwzjs9XvxtmPquPKTNPZ7xF+3nIvmZQu7e1FzxYTiisCdFngVf3B7XXOtva1P8Z n5mIqtQ/PJbOlu2eQyq9qO2lAPg6Nmi4Eee8bgQgA7L6IFfurVh9f9AxtsY9dhHyoi8d 5oaw== Received: by 10.216.140.73 with SMTP id d51mr3914906wej.217.1349384351948; Thu, 04 Oct 2012 13:59:11 -0700 (PDT) Received: from [10.0.0.86] ([93.152.184.10]) by mx.google.com with ESMTPS id j8sm6881714wiy.9.2012.10.04.13.59.06 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 04 Oct 2012 13:59:10 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Content-Type: text/plain; charset=windows-1252 From: Nikolay Denev In-Reply-To: <20121004222422.68d176ec@fabiankeil.de> Date: Thu, 4 Oct 2012 23:59:04 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5A5FE35F-7D68-4E83-A88D-3002B51F2E00@gmail.com> <20121004222422.68d176ec@fabiankeil.de> To: Fabian Keil X-Mailer: Apple Mail (2.1498) Cc: freebsd-fs@freebsd.org Subject: Re: zpool scrub on pool from geli devices offlines the pool? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 20:59:14 -0000 On Oct 4, 2012, at 11:24 PM, Fabian Keil = wrote: > Nikolay Denev wrote: >=20 >> I have a zfs pool from 24 disks encrypted with geli. >>=20 >> I just did a zpool scrub tank, and that probably reopened all of the = devices, >> but this caused geli "detach on last close" to kick in=20 >> which resulted in offline pool from UNAVAILABLE devices.=20 >=20 > This is a known issue: > http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/117158 >=20 > The fact that the system didn't panic seems like an improvement, > although this might be the result of the different pool layout. >=20 >> pool: tank >> state: UNAVAIL >> status: One or more devices are faulted in response to IO failures. >> action: Make sure the affected devices are connected, then run 'zpool = clear'. >> see: http://illumos.org/msg/ZFS-8000-HC >> scan: scrub in progress since Thu Oct 4 21:19:15 2012 >> 1 scanned out of 8.29T at 1/s, (scan is slow, no estimated = time) >> 0 repaired, 0.00% done >> config: >>=20 >> NAME STATE READ WRITE CKSUM >> tank UNAVAIL 0 0 0 > [...] >>=20 >> errors: 1 data errors, use '-v' for a list >>=20 >> Dmesg shows : >>=20 >> GEOM_ELI: Detached mfid1.eli on last close. >> =85 >> GEOM_ELI: Detached mfid24.eli on last close. >>=20 >> I then did /etc/rc.d/geli restart and zpool clear tank, and it is = back online, >> but shows permanent metadata errors=85 >=20 > I'd expect the "permanent" metadata errors to be gone after > the scrubbing is completed. >=20 >> Any ideas why this happned from a simple zpool scrub, and how it can = be prevented? >> Just disable "detach on last close" for the geli devices? >=20 > At least that was Pawel's recommendation in 2007: > = http://lists.freebsd.org/pipermail/freebsd-current/2007-October/078107.htm= l >=20 > Fabian Thanks for the information, I have missed that. And yep, the pool reports as ONLINE without errors after the reboot. I'll add geli_autodetach=3D"NO" to rc.conf. Regards, Nikolay