From owner-freebsd-fs@FreeBSD.ORG Thu Jun 6 19:24:39 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0D09C2FF for ; Thu, 6 Jun 2013 19:24:39 +0000 (UTC) (envelope-from mxb@alumni.chalmers.se) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) by mx1.freebsd.org (Postfix) with ESMTP id 8B1C51353 for ; Thu, 6 Jun 2013 19:24:37 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id fp12so2293034lab.8 for ; Thu, 06 Jun 2013 12:24:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version:x-mailer:x-gm-message-state; bh=mQzCLsyEFPEpej4gVREKQYl2PKQeIZbLHInE8rRkbOA=; b=FrUp2SXsJd9wqvAnhSsvrwFix8GNv0Soy6HjIA9jAZ/6q4RtcB2Zd9Ml4FYWJbJmtb 1UUCtQas3qnp6/1ioIxA084Tx/IVYZ97AziDFQN2Q6J/t5yIE0ijQPy0ZdQQsaUuOInS 4ab6fEkc5JAm+tn7Zov2MOrBrUlVHuHRTkTtzgS4n7Kmtn5olaNiisAONGmQ+T7uDf+m uFpf5/hC9Og8XpZ6kLhQpK73013H6cErON8TlH4dSWSyMGUrQ7+ZAV8i8VU3K6O0qv9u uUJYHz/B60mQPIQZg4BwC+AkAUiU7tswk95YqXBWADhzbSUjCrNKo6yky8WfQIgUGzAn yMeQ== X-Received: by 10.112.61.199 with SMTP id s7mr528986lbr.53.1370546676512; Thu, 06 Jun 2013 12:24:36 -0700 (PDT) Received: from grey.home.unixconn.com (h-75-17.a183.priv.bahnhof.se. [46.59.75.17]) by mx.google.com with ESMTPSA id f9sm29294600lbf.4.2013.06.06.12.24.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Jun 2013 12:24:35 -0700 (PDT) From: mxb Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: zpool export/import on failover - The pool metadata is corrupted Message-Id: Date: Thu, 6 Jun 2013 21:24:34 +0200 To: "freebsd-fs@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-Gm-Message-State: ALoCoQk3OIbYPTDY6EDV//ouiDNI+tTWduHPfwi2lCGlO0VjFu6kk3zwm6OI2M/8DajFgz9QeswK X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jun 2013 19:24:39 -0000 Hello list, I have two-head ZFS setup with external disk enclosure over SAS = expander. This is a failover setup with CARP and devd triggering spool = export/import. One of two nodes is preferred master. Then master is rebooted, devd kicks in as of CARP becomes master and the = second node picks up ZFS-disks from external enclosure. Then master comes back, CARP becomes master, devd kicks in and pool gets = exported from the second node and imported on the first one. However, I have experienced metadata corruption several times with this = setup. Note, that ZIL(mirrored) resides on external enclosure. Only L2ARC is = both local and external - da1,da2, da13s2, da14s2 root@nfs2:/root # zpool import pool: jbod id: 17635654860276652744 state: FAULTED status: The pool metadata is corrupted. action: The pool cannot be imported due to damaged devices or data. see: http://illumos.org/msg/ZFS-8000-72 config: jbod FAULTED corrupted data raidz3-0 ONLINE da3 ONLINE da4 ONLINE da5 ONLINE da6 ONLINE da7 ONLINE da8 ONLINE da9 ONLINE da10 ONLINE da11 ONLINE da12 ONLINE cache da1 da2 da13s2 da14s2 logs mirror-1 ONLINE da13s1 ONLINE da14s1 ONLINE Any ideas what is going on? //mxb =20=