From owner-freebsd-fs@FreeBSD.ORG Fri May 10 14:07:37 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7FC0A26F for ; Fri, 10 May 2013 14:07:37 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) by mx1.freebsd.org (Postfix) with ESMTP id 4EC8AB8B for ; Fri, 10 May 2013 14:07:37 +0000 (UTC) Received: by mail-ob0-f178.google.com with SMTP id v19so477216obq.23 for ; Fri, 10 May 2013 07:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=2ex/hC6BCSEtCOqKcqsySJcpN6CT15rtqR0G8A2G434=; b=d3kvOEMg08Kc+NjofY3AZBM6OF7pHopb9jGOxS/oLYwuYW6Qs8JlBTF0a9vS1EboND xO2BCsjk9CO+qQ/ioE2tiGQRCoTtOw3eZksqBXfuCjEkzI8vw1Res1eIr32K8ih1vZRR tG80iATz+0LxxCj6QfeHshUkyY+E2LDIOJ1JYWDP/E8wJvNY4OBU4j4aJn3hA37m1ICc CuVk0qbwPPdVeK3mMGfeywi6e1yXHnTGq2slTER3Ul70jVigxFIMGBU/EWLJ4NcE+SIx Q9cBazQ02Q1+5kuSoyt5JnOs1kyjX0/UdZ8V6OaIEnjgnSWAygv+MddZYxW4aGhZViOg quQg== MIME-Version: 1.0 X-Received: by 10.60.92.201 with SMTP id co9mr6912546oeb.113.1368194856899; Fri, 10 May 2013 07:07:36 -0700 (PDT) Received: by 10.76.96.49 with HTTP; Fri, 10 May 2013 07:07:36 -0700 (PDT) In-Reply-To: <518CFA05.6090706@gmail.com> References: <518CFA05.6090706@gmail.com> Date: Fri, 10 May 2013 10:07:36 -0400 Message-ID: Subject: Re: Corrupted zpool import -f FAILS state FAULTED From: Outback Dingo To: Volodymyr Kostyrko Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 14:07:37 -0000 On Fri, May 10, 2013 at 9:45 AM, Volodymyr Kostyrko wrote: > 09.05.2013 15:31, Outback Dingo: > > ok zfsgurus, FreeBSD 9.1-STABLE box zpool import -f reports pool status >> Faulted, one of more devices contains corrupted data, however its showing >> the guid as faulted in the poll, and not the actual disk device /dev/daX, >> the pool is a single vdev 24 disk raidz3. Essentially the hardward >> platform >> is a dual node system, with 8 enclosures connected to 24 SAS drives via 4 >> LSI cards. I am not currently using geom_multipath, but the box is zoned >> so >> that each node can see 50% of the drives, >> in case of Failure, carp kicks in and migrates "zpool import -af" the >> pools >> onto the other node. it seems as though somehow the pool is now seeing >> guid >> and not devices, not sure if they have switched devices ids due to a >> reboot. >> > > Am not a zfs guru, but I'll try to help. > > Any console log snippets are welcome. What does "showing the guid as > faulted in the pool" looks like. > > What are the guids for all partitions? Do they interlap for different > nodes? > > ZFS recognizes devices by tasting they vdev labels and not by their > logical location and naming. It can safely report any vdev location - but > it requires the same set vdevs to bring pool online. zdb shows valid data on the drives, no drives have been removed from the box whats confusing is why its using guids and not devices daX is what puzzles me camcontrol devlist and dmesg clearly show the devices are there. The SAS bus is shared so both nodes with 2 LSI controllers each see all drives. We were utilizing a failover script if nodeA dies, carp would kick the script to import the pool to nodeB, both nodes are in the same chassis and see all the enclosures and all the drives > > -- > Sphinx of black quartz, judge my vow. >