From owner-freebsd-fs@FreeBSD.ORG Sat May 11 08:44:44 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8117E662 for ; Sat, 11 May 2013 08:44:44 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.78]) by mx1.freebsd.org (Postfix) with ESMTP id 1A40AEB2 for ; Sat, 11 May 2013 08:44:43 +0000 (UTC) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1Ub5Q1-0002Lp-6y for freebsd-fs@freebsd.org; Sat, 11 May 2013 10:44:41 +0200 Received: from dhcp-077-251-158-153.chello.nl ([77.251.158.153] helo=ronaldradial) by smtp.greenhost.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1Ub5Pz-0007D2-Vm for freebsd-fs@freebsd.org; Sat, 11 May 2013 10:44:40 +0200 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-fs@freebsd.org Subject: Re: Corrupted zpool import -f FAILS state FAULTED References: <518CFA05.6090706@gmail.com> Date: Sat, 11 May 2013 10:44:39 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.15 (Win32) X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: 0.8 X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.1 X-Scan-Signature: 4cc6a862e0a753e674eb374334b394fd X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 May 2013 08:44:44 -0000 On Fri, 10 May 2013 16:07:36 +0200, Outback Dingo wrote: > On Fri, May 10, 2013 at 9:45 AM, Volodymyr Kostyrko > wrote: > >> 09.05.2013 15:31, Outback Dingo: >> >> ok zfsgurus, FreeBSD 9.1-STABLE box zpool import -f reports pool status >>> Faulted, one of more devices contains corrupted data, however its >>> showing >>> the guid as faulted in the poll, and not the actual disk device >>> /dev/daX, >>> the pool is a single vdev 24 disk raidz3. Essentially the hardward >>> platform >>> is a dual node system, with 8 enclosures connected to 24 SAS drives >>> via 4 >>> LSI cards. I am not currently using geom_multipath, but the box is >>> zoned >>> so >>> that each node can see 50% of the drives, >>> in case of Failure, carp kicks in and migrates "zpool import -af" the >>> pools >>> onto the other node. it seems as though somehow the pool is now seeing >>> guid >>> and not devices, not sure if they have switched devices ids due to a >>> reboot. >>> >> >> Am not a zfs guru, but I'll try to help. >> >> Any console log snippets are welcome. What does "showing the guid as >> faulted in the pool" looks like. >> >> What are the guids for all partitions? Do they interlap for different >> nodes? >> >> ZFS recognizes devices by tasting they vdev labels and not by their >> logical location and naming. It can safely report any vdev location - >> but >> it requires the same set vdevs to bring pool online. > > > zdb shows valid data on the drives, no drives have been removed from the > box > whats confusing is why its using guids and not devices daX is what > puzzles > me > camcontrol devlist and dmesg clearly show the devices are there. The SAS > bus is shared > so both nodes with 2 LSI controllers each see all drives. We were > utilizing > a failover script > if nodeA dies, carp would kick the script to import the pool to nodeB, > both > nodes are in the > same chassis and see all the enclosures and all the drives Are the machines configured the same? As in _exactly_ the same. Glabel modules, hint files, sysctls, etc. Ronald.