From owner-freebsd-fs@FreeBSD.ORG Thu Jun 6 20:14:54 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 92880C4A for ; Thu, 6 Jun 2013 20:14:54 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from slow1-d.mail.gandi.net (slow1-d.mail.gandi.net [217.70.178.86]) by mx1.freebsd.org (Postfix) with ESMTP id 5122311AC for ; Thu, 6 Jun 2013 20:14:53 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by slow1-d.mail.gandi.net (Postfix) with ESMTP id E21455348B3 for ; Thu, 6 Jun 2013 21:33:58 +0200 (CEST) Received: from mfilter23-d.gandi.net (mfilter23-d.gandi.net [217.70.178.151]) by relay4-d.mail.gandi.net (Postfix) with ESMTP id 72357172080; Thu, 6 Jun 2013 21:33:48 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mfilter23-d.gandi.net Received: from relay4-d.mail.gandi.net ([217.70.183.196]) by mfilter23-d.gandi.net (mfilter23-d.gandi.net [10.0.15.180]) (amavisd-new, port 10024) with ESMTP id 8xztYmocPT9n; Thu, 6 Jun 2013 21:33:46 +0200 (CEST) X-Originating-IP: 67.180.84.87 Received: from jdc.koitsu.org (c-67-180-84-87.hsd1.ca.comcast.net [67.180.84.87]) (Authenticated sender: jdc@koitsu.org) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 943B9172095; Thu, 6 Jun 2013 21:33:42 +0200 (CEST) Received: by icarus.home.lan (Postfix, from userid 1000) id A18C273A1C; Thu, 6 Jun 2013 12:33:40 -0700 (PDT) Date: Thu, 6 Jun 2013 12:33:40 -0700 From: Jeremy Chadwick To: mxb Subject: Re: zpool export/import on failover - The pool metadata is corrupted Message-ID: <20130606193340.GA42205@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jun 2013 20:14:54 -0000 On Thu, Jun 06, 2013 at 09:24:34PM +0200, mxb wrote: > > Hello list, > > I have two-head ZFS setup with external disk enclosure over SAS expander. > This is a failover setup with CARP and devd triggering spool export/import. > One of two nodes is preferred master. > > Then master is rebooted, devd kicks in as of CARP becomes master and the second node picks up ZFS-disks from external enclosure. > Then master comes back, CARP becomes master, devd kicks in and pool gets exported from the second node and imported on the first one. > > However, I have experienced metadata corruption several times with this setup. > Note, that ZIL(mirrored) resides on external enclosure. Only L2ARC is both local and external - da1,da2, da13s2, da14s2 > > root@nfs2:/root # zpool import > pool: jbod > id: 17635654860276652744 > state: FAULTED > status: The pool metadata is corrupted. > action: The pool cannot be imported due to damaged devices or data. > see: http://illumos.org/msg/ZFS-8000-72 > config: > > jbod FAULTED corrupted data > raidz3-0 ONLINE > da3 ONLINE > da4 ONLINE > da5 ONLINE > da6 ONLINE > da7 ONLINE > da8 ONLINE > da9 ONLINE > da10 ONLINE > da11 ONLINE > da12 ONLINE > cache > da1 > da2 > da13s2 > da14s2 > logs > mirror-1 ONLINE > da13s1 ONLINE > da14s1 ONLINE > > Any ideas what is going on? uname -a please. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |