From owner-freebsd-geom@FreeBSD.ORG Sun Jan 29 01:15:08 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26D591065670 for ; Sun, 29 Jan 2012 01:15:08 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id E1F2E8FC08 for ; Sun, 29 Jan 2012 01:15:06 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id D1BBE28424; Sun, 29 Jan 2012 01:58:02 +0100 (CET) Received: from [192.168.1.2] (ip-86-49-61-235.net.upcbroadband.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id A8C9328423; Sun, 29 Jan 2012 01:58:01 +0100 (CET) Message-ID: <4F249997.1010502@quip.cz> Date: Sun, 29 Jan 2012 01:57:59 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: Lee Dilkie References: <4F24785F.20607@dilkie.com> <4F247D69.6000105@dilkie.com> In-Reply-To: <4F247D69.6000105@dilkie.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-geom Subject: Re: gmirror question, drive missing X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 01:15:08 -0000 Lee Dilkie wrote: > additional. > > like I said, the original setup had the drives swapped. ad10, now ad11, > was the source of the failure. > > from the log files.. > > +ad10: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=1740673583 > +ad10: FAILURE - READ_DMA48 status=51 error=10 LBA=1740673583 > +GEOM_MIRROR: Request failed (error=5). ad10[READ(offset=891224874496, length=4096)] > +GEOM_MIRROR: Device gm0: provider ad10 disconnected. Your problem is exactly the above error - the disk is marked as BROKEN by gmirror and thus not connected (re-synchronized) to gm0 anymore. If you are really sure you want to re-add this broken disk into gmirror gm0, you must clear metadata on it, then remove info about this provider from gmirror configuration and then insert it in to gmirror again. Example (if ad11 is the broken disk): gmirror clear -v ad11 gmirror forget -v gm0 gmirror insert -v gm0 ad11 Maybe you can use gmirror remove instead of clear and forget, I am not sure. PS: I recommend you tu check the disk with sysutils/smartmontools: smartctl -a /dev/ad11 > On 1/28/2012 5:36 PM, Lee Dilkie wrote: >> Hi Folks, >> >> I'm hoping freebsd-goem is the correct place for this question. >> >> I had a server crash that had a gmirror array of two drives. It was my >> bootable disk and it failed in such a way that it wouldn't recover >> without swapping cables. >> >> So the original gmirror, gm0, had /dev/ad10 and /dev/ad11 as providers. >> >> Now, gmirror shows. >> >> $ gmirror status gm0 >> Name Status Components >> mirror/gm0 DEGRADED ad10 >> >> $ gmirror list gm0 >> Geom name: gm0 >> State: DEGRADED >> Components: 2 >> Balance: round-robin >> Slice: 4096 >> Flags: NONE >> GenID: 1 >> SyncID: 2 >> ID: 1837199655 >> Providers: >> 1. Name: mirror/gm0 >> Mediasize: 1000204885504 (931G) >> Sectorsize: 512 >> Mode: r5w5e6 >> Consumers: >> 1. Name: ad10 >> Mediasize: 1000204886016 (931G) >> Sectorsize: 512 >> Mode: r1w1e1 >> State: ACTIVE >> Priority: 0 >> Flags: DIRTY >> GenID: 1 >> SyncID: 2 >> ID: 3835414526 >> >> no mention of /dev/ad11 at all. >> >> however, I can dump the metadata and it's there, as far as I can tell. >> >> $ gmirror dump /dev/ad11 >> Metadata on /dev/ad11: >> magic: GEOM::MIRROR >> version: 4 >> name: gm0 >> mid: 1837199655 >> did: 1856327785 >> all: 2 >> genid: 0 >> syncid: 2 >> priority: 0 >> slice: 4096 >> balance: round-robin >> mediasize: 1000204885504 >> sectorsize: 512 >> syncoffset: 0 >> mflags: NONE >> dflags: DIRTY >> hcprovider: >> provsize: 1000204886016 >> MD5 hash: 5ac5a19c7e0b99b4852ba535297d413a >> >> and the working, ad10... >> >> $ gmirror dump /dev/ad10 >> Metadata on /dev/ad10: >> magic: GEOM::MIRROR >> version: 4 >> name: gm0 >> mid: 1837199655 >> did: 3835414526 >> all: 2 >> genid: 1 >> syncid: 2 >> priority: 0 >> slice: 4096 >> balance: round-robin >> mediasize: 1000204885504 >> sectorsize: 512 >> syncoffset: 0 >> mflags: NONE >> dflags: DIRTY >> hcprovider: >> provsize: 1000204886016 >> MD5 hash: 13cc9220591eb360cbdaea72edbe87d9 >> >> so what's happened? Do I simply perform a "gmirror insert gmo >> /dev/ad11" to re-setup the mirror or should something else be done? >> >> BTW, when I performed all my failure mode tests before, I never saw a >> provider disappear, gmirror simply rebuilt the array when the disk was >> plugged in. And I did move disks around the sata cables so they were >> both masters instead of the master/slave I had originally used with >> the mirror was set up, so I don't *think* swapping the sata cables did >> this but....