From owner-freebsd-geom@FreeBSD.ORG Sun Jan 29 01:15:08 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26D591065670 for ; Sun, 29 Jan 2012 01:15:08 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id E1F2E8FC08 for ; Sun, 29 Jan 2012 01:15:06 +0000 (UTC) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id D1BBE28424; Sun, 29 Jan 2012 01:58:02 +0100 (CET) Received: from [192.168.1.2] (ip-86-49-61-235.net.upcbroadband.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id A8C9328423; Sun, 29 Jan 2012 01:58:01 +0100 (CET) Message-ID: <4F249997.1010502@quip.cz> Date: Sun, 29 Jan 2012 01:57:59 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.19) Gecko/20110420 Lightning/1.0b1 SeaMonkey/2.0.14 MIME-Version: 1.0 To: Lee Dilkie References: <4F24785F.20607@dilkie.com> <4F247D69.6000105@dilkie.com> In-Reply-To: <4F247D69.6000105@dilkie.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-geom Subject: Re: gmirror question, drive missing X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 01:15:08 -0000 Lee Dilkie wrote: > additional. > > like I said, the original setup had the drives swapped. ad10, now ad11, > was the source of the failure. > > from the log files.. > > +ad10: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=1740673583 > +ad10: FAILURE - READ_DMA48 status=51 error=10 LBA=1740673583 > +GEOM_MIRROR: Request failed (error=5). ad10[READ(offset=891224874496, length=4096)] > +GEOM_MIRROR: Device gm0: provider ad10 disconnected. Your problem is exactly the above error - the disk is marked as BROKEN by gmirror and thus not connected (re-synchronized) to gm0 anymore. If you are really sure you want to re-add this broken disk into gmirror gm0, you must clear metadata on it, then remove info about this provider from gmirror configuration and then insert it in to gmirror again. Example (if ad11 is the broken disk): gmirror clear -v ad11 gmirror forget -v gm0 gmirror insert -v gm0 ad11 Maybe you can use gmirror remove instead of clear and forget, I am not sure. PS: I recommend you tu check the disk with sysutils/smartmontools: smartctl -a /dev/ad11 > On 1/28/2012 5:36 PM, Lee Dilkie wrote: >> Hi Folks, >> >> I'm hoping freebsd-goem is the correct place for this question. >> >> I had a server crash that had a gmirror array of two drives. It was my >> bootable disk and it failed in such a way that it wouldn't recover >> without swapping cables. >> >> So the original gmirror, gm0, had /dev/ad10 and /dev/ad11 as providers. >> >> Now, gmirror shows. >> >> $ gmirror status gm0 >> Name Status Components >> mirror/gm0 DEGRADED ad10 >> >> $ gmirror list gm0 >> Geom name: gm0 >> State: DEGRADED >> Components: 2 >> Balance: round-robin >> Slice: 4096 >> Flags: NONE >> GenID: 1 >> SyncID: 2 >> ID: 1837199655 >> Providers: >> 1. Name: mirror/gm0 >> Mediasize: 1000204885504 (931G) >> Sectorsize: 512 >> Mode: r5w5e6 >> Consumers: >> 1. Name: ad10 >> Mediasize: 1000204886016 (931G) >> Sectorsize: 512 >> Mode: r1w1e1 >> State: ACTIVE >> Priority: 0 >> Flags: DIRTY >> GenID: 1 >> SyncID: 2 >> ID: 3835414526 >> >> no mention of /dev/ad11 at all. >> >> however, I can dump the metadata and it's there, as far as I can tell. >> >> $ gmirror dump /dev/ad11 >> Metadata on /dev/ad11: >> magic: GEOM::MIRROR >> version: 4 >> name: gm0 >> mid: 1837199655 >> did: 1856327785 >> all: 2 >> genid: 0 >> syncid: 2 >> priority: 0 >> slice: 4096 >> balance: round-robin >> mediasize: 1000204885504 >> sectorsize: 512 >> syncoffset: 0 >> mflags: NONE >> dflags: DIRTY >> hcprovider: >> provsize: 1000204886016 >> MD5 hash: 5ac5a19c7e0b99b4852ba535297d413a >> >> and the working, ad10... >> >> $ gmirror dump /dev/ad10 >> Metadata on /dev/ad10: >> magic: GEOM::MIRROR >> version: 4 >> name: gm0 >> mid: 1837199655 >> did: 3835414526 >> all: 2 >> genid: 1 >> syncid: 2 >> priority: 0 >> slice: 4096 >> balance: round-robin >> mediasize: 1000204885504 >> sectorsize: 512 >> syncoffset: 0 >> mflags: NONE >> dflags: DIRTY >> hcprovider: >> provsize: 1000204886016 >> MD5 hash: 13cc9220591eb360cbdaea72edbe87d9 >> >> so what's happened? Do I simply perform a "gmirror insert gmo >> /dev/ad11" to re-setup the mirror or should something else be done? >> >> BTW, when I performed all my failure mode tests before, I never saw a >> provider disappear, gmirror simply rebuilt the array when the disk was >> plugged in. And I did move disks around the sata cables so they were >> both masters instead of the master/slave I had originally used with >> the mirror was set up, so I don't *think* swapping the sata cables did >> this but.... From owner-freebsd-geom@FreeBSD.ORG Sun Jan 29 03:30:16 2012 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C8CF1065672 for ; Sun, 29 Jan 2012 03:30:16 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 51FD68FC15 for ; Sun, 29 Jan 2012 03:30:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0T3UGI7096615 for ; Sun, 29 Jan 2012 03:30:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0T3UGJH096609; Sun, 29 Jan 2012 03:30:16 GMT (envelope-from gnats) Date: Sun, 29 Jan 2012 03:30:16 GMT Message-Id: <201201290330.q0T3UGJH096609@freefall.freebsd.org> To: freebsd-geom@FreeBSD.org From: Yoshisato YANAGISAWA Cc: Subject: Re: kern/163020: [patch] enable the Camellia-XTS on GEOM ELI (geli). X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Yoshisato YANAGISAWA List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 03:30:16 -0000 The following reply was made to PR kern/163020; it has been noted by GNATS. From: Yoshisato YANAGISAWA To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org Cc: Subject: Re: kern/163020: [patch] enable the Camellia-XTS on GEOM ELI (geli). Date: Sun, 29 Jan 2012 12:07:25 +0900 I tried to apply the same patch to the latest FreeBSD current, and confirm it still works. I can apply the patch and all the test has passed. From owner-freebsd-geom@FreeBSD.ORG Mon Jan 30 07:23:32 2012 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75560106564A; Mon, 30 Jan 2012 07:23:32 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4B7428FC12; Mon, 30 Jan 2012 07:23:32 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0U7NW7t072210; Mon, 30 Jan 2012 07:23:32 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0U7NVE6072206; Mon, 30 Jan 2012 07:23:31 GMT (envelope-from linimon) Date: Mon, 30 Jan 2012 07:23:31 GMT Message-Id: <201201300723.q0U7NVE6072206@freefall.freebsd.org> To: osho@pcc-software.org, linimon@FreeBSD.org, freebsd-geom@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/163020: [geli] [patch] enable the Camellia-XTS on GEOM ELI X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 07:23:32 -0000 Synopsis: [geli] [patch] enable the Camellia-XTS on GEOM ELI State-Changed-From-To: open->analyzed State-Changed-By: linimon State-Changed-When: Mon Jan 30 07:22:51 UTC 2012 State-Changed-Why: Submitter notes this patch is still valid for -current. http://www.freebsd.org/cgi/query-pr.cgi?pr=163020 From owner-freebsd-geom@FreeBSD.ORG Mon Jan 30 11:07:34 2012 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91A331065675 for ; Mon, 30 Jan 2012 11:07:34 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7E26F8FC16 for ; Mon, 30 Jan 2012 11:07:34 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0UB7Y6D005402 for ; Mon, 30 Jan 2012 11:07:34 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0UB7XPG005400 for freebsd-geom@FreeBSD.org; Mon, 30 Jan 2012 11:07:33 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 30 Jan 2012 11:07:33 GMT Message-Id: <201201301107.q0UB7XPG005400@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-geom@FreeBSD.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 11:07:34 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/164254 geom [geom] gjournal not stopping on GPT partitions o kern/164252 geom [geom] gjournal overflow o kern/164143 geom [geom] Partition table not recognized after upgrade R8 a kern/163020 geom [geli] [patch] enable the Camellia-XTS on GEOM ELI o kern/162010 geom [geli] panic: Provider's error should be set (error=0) o kern/161979 geom [geom] glabel doesn't update after newfs, and glabel s o kern/161752 geom [geom] glabel(8) doesn't get gpt label change o bin/161677 geom gpart(8) Probably bug in gptboot o kern/160562 geom [geom][patch] Allow to insert new component to geom_ra o kern/160409 geom [geli] failed to attach provider f kern/159595 geom [geom] [panic] panic on gmirror unload in vbox [regres p kern/158398 geom [headers] [patch] includes o kern/158197 geom [geom] geom_cache with size>1000 leads to panics o kern/157879 geom [libgeom] [regression] ABI change without version bump o kern/157863 geom [geli] kbdmux prevents geli passwords from being enter o kern/157739 geom [geom] GPT labels with geom_multipath o kern/157724 geom [geom] gpart(8) 'add' command must preserve gap for sc o kern/157723 geom [geom] GEOM should not process 'c' (raw) partitions fo o kern/157108 geom [gjournal] dumpon(8) fails on gjournal providers o kern/155994 geom [geom] Long "Suspend time" when reading large files fr o kern/154226 geom [geom] GEOM label does not change when you modify them o kern/150858 geom [geom] [geom_label] [patch] glabel(8) is not compatibl o kern/150626 geom [geom] [gjournal] gjournal(8) destroys label o kern/150555 geom [geom] gjournal unusable on GPT partitions o kern/150334 geom [geom] [udf] [patch] geom label does not support UDF o kern/149762 geom volume labels with rogue characters o bin/149215 geom [panic] [geom_part] gpart(8): Delete linux's slice via o kern/147667 geom [gmirror] Booting with one component of a gmirror, the o kern/145818 geom [geom] geom_stat_open showing cached information for n o kern/145042 geom [geom] System stops booting after printing message "GE o kern/143455 geom gstripe(8) in RELENG_8 (31st Jan 2010) broken o kern/142563 geom [geom] [hang] ioctl freeze in zpool o kern/141740 geom [geom] gjournal(8): g_journal_destroy concurrent error o kern/140352 geom [geom] gjournal + glabel not working o kern/135898 geom [geom] Severe filesystem corruption - large files or l o kern/134922 geom [gmirror] [panic] kernel panic when use fdisk on disk o kern/134113 geom [geli] Problem setting secondary GELI key o kern/133931 geom [geli] [request] intentionally wrong password to destr o bin/132845 geom [geom] [patch] ggated(8) does not close files opened a o bin/131415 geom [geli] keystrokes are unregulary sent to Geli when typ o kern/131353 geom [geom] gjournal(8) kernel lock o kern/129674 geom [geom] gjournal root did not mount on boot o kern/129645 geom gjournal(8): GEOM_JOURNAL causes system to fail to boo o kern/129245 geom [geom] gcache is more suitable for suffix based provid f kern/128276 geom [gmirror] machine lock up when gmirror module is used o kern/127420 geom [geom] [gjournal] [panic] Journal overflow on gmirrore o kern/124973 geom [gjournal] [patch] boot order affects geom_journal con o kern/124969 geom gvinum(8): gvinum raid5 plex does not detect missing s o kern/123962 geom [panic] [gjournal] gjournal (455Gb data, 8Gb journal), o kern/123122 geom [geom] GEOM / gjournal kernel lock o kern/122738 geom [geom] gmirror list "losts consumers" after gmirror de o kern/122067 geom [geom] [panic] Geom crashed during boot o kern/121364 geom [gmirror] Removing all providers create a "zombie" mir o kern/120091 geom [geom] [geli] [gjournal] geli does not prompt for pass o kern/115856 geom [geli] ZFS thought it was degraded when it should have o kern/115547 geom [geom] [patch] [request] let GEOM Eli get password fro o kern/114532 geom [geom] GEOM_MIRROR shows up in kldstat even if compile f kern/113957 geom [gmirror] gmirror is intermittently reporting a degrad o kern/113837 geom [geom] unable to access 1024 sector size storage o kern/113419 geom [geom] geom fox multipathing not failing back o kern/107707 geom [geom] [patch] [request] add new class geom_xbox360 to o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/90582 geom [geom] [panic] Restore cause panic string (ffs_blkfree o bin/90093 geom fdisk(8) incapable of altering in-core geometry o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o bin/86388 geom [geom] [geom_part] periodic(8) daily should backup gpa o kern/84556 geom [geom] [panic] GBDE-encrypted swap causes panic at shu o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/79035 geom [vinum] gvinum unable to create a striped set of mirro o bin/78131 geom gbde(8) "destroy" not working. 70 problems total. From owner-freebsd-geom@FreeBSD.ORG Mon Jan 30 16:14:07 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D65D81065672 for ; Mon, 30 Jan 2012 16:14:07 +0000 (UTC) (envelope-from vivek@mailermailer.com) Received: from yertle.kcilink.com (yertle-comcast.kcilink.com [74.92.149.58]) by mx1.freebsd.org (Postfix) with ESMTP id 8E6078FC15 for ; Mon, 30 Jan 2012 16:14:07 +0000 (UTC) Received: from vick.int.kcilink.com (vick.int.kcilink.com [192.168.7.80]) by yertle.kcilink.com (Postfix) with ESMTP id F08588A00B for ; Mon, 30 Jan 2012 10:55:19 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mailermailer.com; s=office0709; t=1327938919; bh=r4StdKSuo0iJbOr3xuPDw2botHsD1XSj7BEJoDV+9nM=; h=From:Content-Type:Content-Transfer-Encoding:Subject:Date: Message-Id:To:Mime-Version; b=DDl5Ro99+J+YHnOsIqNs81bFfkwZSzl+1VMAugPllC8pFsCeL1WB8Iv72RwvTA29o paKS7Q09UXxI214ci2HxUtU7u5vOyAjjR1j2HbvnzyvlzI5OpqZ9Hnsgzwzx6RKG40 gWhEqzCwzl6HQ1ryaYXeQpX/mj9GRy9d6PP43esw= From: Vick Khera Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Mon, 30 Jan 2012 10:55:19 -0500 Message-Id: To: freebsd-geom@freebsd.org Mime-Version: 1.0 (Apple Message framework v1251.1) X-Mailer: Apple Mail (2.1251.1) Subject: whole-disk gmirror and upgrade from 8.2 to 9.0 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 16:14:07 -0000 Earlier on questions@ I asked about the gmirror failure on booting into = a 9.0 kernel when the whole-disk was mirrored on an 8.2 system. I was = pointed to this post = = , which helped me get it to boot, and I verified that I have the exact = same issue mentioned there with the difference in gpart. So I figured what I should do is destroy my existing gmirror while = booted in 8.2, boot into 9.0 and install world. This went extremely = well.... that is until I try to recreate the new gmirror. I destroyed the gmirror in 8.2 with these commands: gmirror remove gm0 ad6 reboot gmirror remove gm0 ad4 reboot into 9.0 kernel, install world, merge master, etc. So now that my ada0 (old ad4) disk has on it a full 9.0 install, I want = to set up gmirror again, like I always do: boot to single user, then: # uname -a FreeBSD 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 17 14:49:58 EST = 2012 = vivek@lorax.kcilink.com:/u/lorax1/usr9/obj.amd64/u/lorax1/usr9/src/sys/KCI= 64 amd64 # gmirror list # gmirror status # gmirror dump ada1 Can't read metadata from ada1: Invalid argument. gmirror: Not fully done. # gmirror dump ada0 Can't read metadata from ada0: Invalid argument. gmirror: Not fully done. # sysctl kern.geom.debugflags=3D16 kern.geom.debugflags: 0 -> 16 # gmirror label -v gm0 ada0 Metadata value stored on ada0. Done. # reboot On reboot, I again get the same "GEOM_PART: integrity check failed = (mirror/gm0, MBR)" error from before, and then the system proceeds to = boot the ada1 disk, giving me 9.0 kernel with 8.2 world. gmirror = reports the mirror is fine, though: # gmirror status Name Status Components mirror/gm0 COMPLETE ada0 # gmirror list =20 Geom name: gm0 State: COMPLETE Components: 1 Balance: load Slice: 4096 Flags: NONE GenID: 0 SyncID: 1 ID: 168727457 Providers: 1. Name: mirror/gm0 Mediasize: 32016530944 (30G) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: ada0 Mediasize: 32016531456 (30G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 0 SyncID: 1 ID: 2335861329 If I set kern.geom.part.check_integrity=3D0 on the boot, I boot fine on = the gm0 device, and gmirror reports the same status "COMPLETE". If I = clear the gmirror by removing ada0, it will boot and run fine from the = ada0 device. What do I need to do to make this gmirror work again? The disk is = partitioned MBR so there should be no problems with GPT interference. From owner-freebsd-geom@FreeBSD.ORG Mon Jan 30 17:07:43 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFC3B1065670 for ; Mon, 30 Jan 2012 17:07:43 +0000 (UTC) (envelope-from lee@dilkie.com) Received: from data.snhdns.com (data.snhdns.com [208.76.82.136]) by mx1.freebsd.org (Postfix) with ESMTP id B75578FC0C for ; Mon, 30 Jan 2012 17:07:42 +0000 (UTC) Received: from 66-46-196-229.dedicated.allstream.net ([66.46.196.229] helo=[127.0.0.1]) by data.snhdns.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1Rruhf-0002Ke-66; Mon, 30 Jan 2012 12:07:39 -0500 Message-ID: <4F26CE5C.20003@dilkie.com> Date: Mon, 30 Jan 2012 12:07:40 -0500 From: Lee Dilkie User-Agent: Mozilla/5.0 (Windows NT 5.2; WOW64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <4F24785F.20607@dilkie.com> <4F247D69.6000105@dilkie.com> <4F249997.1010502@quip.cz> In-Reply-To: <4F249997.1010502@quip.cz> X-Enigmail-Version: 1.3.5 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - data.snhdns.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dilkie.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-geom Subject: Re: gmirror question, drive missing X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 17:07:44 -0000 On 1/28/2012 7:57 PM, Miroslav Lachman wrote: > Lee Dilkie wrote: >> additional. >> >> like I said, the original setup had the drives swapped. ad10, now ad11, >> was the source of the failure. >> >> from the log files.. >> >> +ad10: TIMEOUT - READ_DMA48 retrying (1 retry left) LBA=1740673583 >> +ad10: FAILURE - READ_DMA48 status=51 >> error=10 LBA=1740673583 >> +GEOM_MIRROR: Request failed (error=5). >> ad10[READ(offset=891224874496, length=4096)] >> +GEOM_MIRROR: Device gm0: provider ad10 disconnected. > > Your problem is exactly the above error - the disk is marked as BROKEN > by gmirror and thus not connected (re-synchronized) to gm0 anymore. > > If you are really sure you want to re-add this broken disk into > gmirror gm0, you must clear metadata on it, then remove info about > this provider from gmirror configuration and then insert it in to > gmirror again. > > Example (if ad11 is the broken disk): > gmirror clear -v ad11 > gmirror forget -v gm0 > gmirror insert -v gm0 ad11 > > Maybe you can use gmirror remove instead of clear and forget, I am not > sure. > > PS: I recommend you tu check the disk with sysutils/smartmontools: > > smartctl -a /dev/ad11 Thanks Miroslav (sorry for the late reply, my home internet connection went down on the weekend and hasn't recovered). Yes, I did do a smartctl "long" test on the drive and it came back clean so I'm not sure it was a drive failure. It was a very odd failure actually. The system should have continued to run with the one drive gone but it didn't It stayed up and I was able to access it using ssh for a while, but then it became clear that the filesystem had "gone". Running applications couldn't access the filesystem and eventually even ssh refused connections. Didn't happen all at once though, first indication was imap complaining that it couldn't access user mailboxes... when I got someone to go in and reboot, the server wouldn't come up, the failed drive, ad10 at the time, had no boot loader available... don't know what happened to that drive but it was corrupted somehow (still is in the same state if someone has an idea of what I could look for?). I swapped drive cables to the other drive from the mirror and it came right up and has been running fine ever since (I did a manual fsck to fix the unclean shutdown). what I *think* happened was some sort of system h/w failure (maybe) that made both drives not work properly. There's no indication in the logs but the logs seem to have stopped after the one indicated so I think the remaining drive was no longer writable. just guessing. -lee From owner-freebsd-geom@FreeBSD.ORG Mon Jan 30 17:19:08 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E50DE1065672 for ; Mon, 30 Jan 2012 17:19:08 +0000 (UTC) (envelope-from khera@kcilink.com) Received: from yertle.kcilink.com (yertle-comcast.kcilink.com [74.92.149.58]) by mx1.freebsd.org (Postfix) with ESMTP id 9C1CC8FC1E for ; Mon, 30 Jan 2012 17:19:08 +0000 (UTC) Received: from vick.int.kcilink.com (vick.int.kcilink.com [192.168.7.80]) by yertle.kcilink.com (Postfix) with ESMTP id 071DB8A00C; Mon, 30 Jan 2012 12:02:06 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kcilink.com; s=kci0709; t=1327942926; bh=ZClTCw5h3JDGKxea39kJ3HWIiL1M9VcXbp+97UaAtx0=; h=From:Content-Type:Content-Transfer-Encoding:Subject:Date: Message-Id:Cc:To:Mime-Version; b=BiZzdLxRLixQgLVfTydu5Sb10ub1xsXU/gw0P++DVz2a3mIAN2n/DAuQP1vNIebyt /gGH7nXEQ2uocyV23z1XajAdf+6zWDJZiJybpF0EZiUuS2raumEohGbLYk88yOmh+t 0JV7fWqdgZoIDFj2GBe3u0UOQgAPXSY1tyhlFVWw= From: Vick Khera Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Mon, 30 Jan 2012 12:02:05 -0500 Message-Id: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> To: freebsd-geom@freebsd.org Mime-Version: 1.0 (Apple Message framework v1251.1) X-Mailer: Apple Mail (2.1251.1) Cc: sascha@trimind.de Subject: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 17:19:09 -0000 A couple of weeks ago I discovered this = = which seemed to identically match a problem I was having with one = server upgrading to 9.0. However, another system I have survived an upgrad through this change = just fine with zero errors. It also exhibited the same change in part = output. The relevant bits of change are: -last: 156249953 +last: 156249998 where the latter number is from the 9.0 boot of this system. So, it seems that something other than just the "last" reported from = gpart is causing gmirror to fail its integrity check. From owner-freebsd-geom@FreeBSD.ORG Tue Jan 31 04:45:55 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68E0C106566B for ; Tue, 31 Jan 2012 04:45:55 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward5.mail.yandex.net (forward5.mail.yandex.net [IPv6:2a02:6b8:0:602::5]) by mx1.freebsd.org (Postfix) with ESMTP id D55008FC0A for ; Tue, 31 Jan 2012 04:45:54 +0000 (UTC) Received: from smtp1.mail.yandex.net (smtp1.mail.yandex.net [77.88.46.101]) by forward5.mail.yandex.net (Yandex) with ESMTP id 120381201EF4; Tue, 31 Jan 2012 08:45:53 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1327985153; bh=8sz6ktvHk3W6IJiqythEWSpMJoj0hELvtKJlwVjxmPI=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=qNZOmxnuv7hGGx0tr/6tZ8T3kom9tIdTRbB5/vBcAt2yogt3sTHcuCS8QsS0Lm82O UslNR2e//bbQMXewkMn5wSUjqlnb5PKDxitDIUP8SNuuTIq5YXBZEoUAX85jsjz6O/ WGbUM6sHCBg67+fWxRmYociznAEzvNylsB3VBwVc= Received: from smtp1.mail.yandex.net (localhost [127.0.0.1]) by smtp1.mail.yandex.net (Yandex) with ESMTP id D3826AA0176; Tue, 31 Jan 2012 08:45:52 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1327985152; bh=8sz6ktvHk3W6IJiqythEWSpMJoj0hELvtKJlwVjxmPI=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=EIIu/X5Qc46aY9TdKttEKjTq3OAyD68h0K4ypE2BVuxUMPuof9skMhi47GBjVPLLq iG3fMMthIf7V05c1rLBPCP/JrLv5qHjSB2a443TgUUI/nMyOETndyg+exDT9RvvZnP O67dU9tc2/AQzhVTPF+EYib+Vg6uUjpED5kt5lxQ= Received: from ns.kirov.so-ups.ru (ns.kirov.so-ups.ru [178.74.170.1]) by smtp1.mail.yandex.net (nwsmtp/Yandex) with ESMTP id jqUeZ2S2-jqUmejRU; Tue, 31 Jan 2012 08:45:52 +0400 X-Yandex-Spam: 1 Message-ID: <4F277200.8020003@yandex.ru> Date: Tue, 31 Jan 2012 08:45:52 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: Vick Khera References: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> In-Reply-To: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> X-Enigmail-Version: 1.3.4 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: sascha@trimind.de, freebsd-geom@freebsd.org Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 04:45:55 -0000 On 30.01.2012 21:02, Vick Khera wrote: > The relevant bits of change are: > > -last: 156249953 +last: 156249998 > > where the latter number is from the 9.0 boot of this system. > > So, it seems that something other than just the "last" reported from gpart is causing gmirror to > fail its integrity check. First of you can enable verbose boot mode and you will see why your partition table was lost. Pay attention to the messages which begins from "GEOM_PART:". Usually the cause is that you have created the mirror after partition table. And partition size stored in the partition table metadata is greater than available sectors in this mirror. Now it is detected by the gpart's integrity checks and gpart on the mirror refuses partition table. One way to fix the problem: 1. Disable integrity checks: kern.geom.part.check_integrity=0 2. Detach one component from the mirror and destroy partition table on it. 3. Create new mirror on this disk and recreate partition table ATOP of the mirror. 4. dump+restore the data from old mirror to new one 5. destroy old mirror and attach the disk to new mirror. -- WBR, Andrey V. Elsukov From owner-freebsd-geom@FreeBSD.ORG Tue Jan 31 13:40:40 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 688161065675 for ; Tue, 31 Jan 2012 13:40:40 +0000 (UTC) (envelope-from vivek@khera.org) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id ED18E8FC13 for ; Tue, 31 Jan 2012 13:40:39 +0000 (UTC) Received: by bkbzx1 with SMTP id zx1so705621bkb.13 for ; Tue, 31 Jan 2012 05:40:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=khera.org; s=google11; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=n+WCUmY4DLhLOQBrLoGCK5EWCdHXLP20l+RkJ64/mgI=; b=RmHYp1KaWLxG+Gz4mSZtRWAaSVIYI1XkgwYW9DUJp2kraJz3lD5q/AHDqpn1MuOg1a PFr05BueFmCf3AJ0yuIVkFfJYajyyH42WkTJd8CG41X/39yVEw3M6KONw4RmtDD7M23W cW6VQmyfO3jG5rdWAYCKxnHEe8lCJprz1+3Ew= MIME-Version: 1.0 Received: by 10.204.156.155 with SMTP id x27mr10608155bkw.4.1328015740572; Tue, 31 Jan 2012 05:15:40 -0800 (PST) Received: by 10.204.148.11 with HTTP; Tue, 31 Jan 2012 05:15:40 -0800 (PST) In-Reply-To: <4F277200.8020003@yandex.ru> References: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> <4F277200.8020003@yandex.ru> Date: Tue, 31 Jan 2012 08:15:40 -0500 Message-ID: From: Vick Khera To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 13:40:40 -0000 2012/1/30 Andrey V. Elsukov : > Usually the cause is that you have created the mirror after partition table. > And partition size stored in the partition table metadata is greater than available sectors > in this mirror. Now it is detected by the gpart's integrity checks and gpart on the mirror > refuses partition table. Yes, this is exactly what it seems to be. My solution was to re-break the gmirror from FreeBSD 8 kernel, boot into FreeBSD 9 kernel and world, then reformat the second drive using GPT, dump/restore from first drive, boot to second drive, reformat first drive, set up gmirror per partition, then reboot. I'm *really* glad I had an ILOM on this server to let me reset it many many times as I made mistakes and locked up or panicked the system to the point of lockups. The procedure I have now is, I believe, panic-free and lockup-free. I'll be cleaning up my notes and posting them somewhere for others. From owner-freebsd-geom@FreeBSD.ORG Tue Jan 31 16:52:28 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7568106566C for ; Tue, 31 Jan 2012 16:52:28 +0000 (UTC) (envelope-from vivek@khera.org) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3992A8FC0A for ; Tue, 31 Jan 2012 16:52:27 +0000 (UTC) Received: by bkbzx1 with SMTP id zx1so211211bkb.13 for ; Tue, 31 Jan 2012 08:52:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=khera.org; s=google11; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=mKvAsyzFK7IPyfBTp9C/S6HqrsxSCtTCWrFdGuIvspg=; b=ZczR2UQadDI+TSjCMyciGek84IOByvuo/LiPUhpO6tc/nxA41q9S+eCfh8KgjvoKuB j1Hwa/z4jJLXtiZDCWslN+nTAClaPDM342v04IZfzWEQkxQK8/f4lqjzYmewXnZXHnZy 5kwjQ0WfXLiw9LZdflsNjzweLLBFGDPp5HCDM= MIME-Version: 1.0 Received: by 10.204.154.136 with SMTP id o8mr11414320bkw.112.1328028746725; Tue, 31 Jan 2012 08:52:26 -0800 (PST) Received: by 10.204.148.11 with HTTP; Tue, 31 Jan 2012 08:52:26 -0800 (PST) In-Reply-To: References: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> <4F277200.8020003@yandex.ru> Date: Tue, 31 Jan 2012 11:52:26 -0500 Message-ID: From: Vick Khera To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 16:52:28 -0000 On Tue, Jan 31, 2012 at 8:15 AM, Vick Khera wrote: > My solution was to re-break the gmirror from FreeBSD 8 kernel, boot > into FreeBSD 9 kernel and world, then reformat the second drive using > GPT, dump/restore from first drive, boot to second drive, reformat > first drive, set up gmirror per partition, then reboot. My step-by-step is posted. Hope it helps someone get over the busted gmirror situation where you get this error: GEOM_PART: integrity check failed (mirror/gm0, MBR) http://vivek.khera.org/mini-blog/upgradefreebsd82to90withgmirrorbootdisk From owner-freebsd-geom@FreeBSD.ORG Wed Feb 1 08:21:53 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49952106564A for ; Wed, 1 Feb 2012 08:21:53 +0000 (UTC) (envelope-from sascha@trimind.de) Received: from mikako.shopkeeper.de (mikako.shopkeeper.de [82.119.175.20]) by mx1.freebsd.org (Postfix) with ESMTP id 9FF118FC0A for ; Wed, 1 Feb 2012 08:21:52 +0000 (UTC) Received: from avalon.dobu.local (p508D4F8B.dip.t-dialin.net [80.141.79.139]) (authenticated bits=0) by mikako.shopkeeper.de (8.14.3/8.14.3) with ESMTP id q1182tMr057669 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 1 Feb 2012 09:02:57 +0100 (CET) (envelope-from sascha@trimind.de) Received: from avalon.dobu.local (localhost [127.0.0.1]) by avalon.dobu.local (8.14.5/8.14.2) with ESMTP id q1182qx2001133; Wed, 1 Feb 2012 09:02:54 +0100 (CET) (envelope-from sascha@avalon.dobu.local) Received: (from sascha@localhost) by avalon.dobu.local (8.14.5/8.14.5/Submit) id q0VEcIXg002396; Tue, 31 Jan 2012 15:38:18 +0100 (CET) (envelope-from sascha) Date: Tue, 31 Jan 2012 15:38:18 +0100 From: Sascha Klauder To: "Andrey V. Elsukov" Message-ID: <20120131143818.GA2320@trimind.de> References: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> <4F277200.8020003@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F277200.8020003@yandex.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.97.2 at mikako.shopkeeper.de X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (mikako.shopkeeper.de [82.119.175.20]); Wed, 01 Feb 2012 09:02:58 +0100 (CET) Cc: Vick Khera , freebsd-geom@freebsd.org Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 08:21:53 -0000 On Tue, 2012-01-31 08:45 +0400, Andrey V. Elsukov wrote: > Usually the cause is that you have created the mirror after partition table. > And partition size stored in the partition table metadata is greater than available sectors > in this mirror. Now it is detected by the gpart's integrity checks and gpart on the mirror > refuses partition table. > > One way to fix the problem: > 1. Disable integrity checks: kern.geom.part.check_integrity=0 > 2. Detach one component from the mirror and destroy partition table on it. > 3. Create new mirror on this disk and recreate partition table ATOP of the mirror. > 4. dump+restore the data from old mirror to new one > 5. destroy old mirror and attach the disk to new mirror. Thanks for the followup and the explanation Andrey. I've just tried and found this method working fine when upgrading another identical configured system from 8.2 to 9.0. Given geom_mirror provider gm0 consisting of two disks ada0 and ada1, in short, here is what I did: # gmirror remove gm0 ada1 # dd if=/dev/zero of=/dev/ada1 bs=512 count=10 # gmirror label gm1 /dev/ada1 # gpart create -s gpt /dev/mirror/gm1 # gpart add -t freebsd-boot -l gptboot -s 128K /dev/mirror/gm1 # gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 /dev/mirror/gm1 # gpart add -t freebsd-ufs -l root -s 2G /dev/mirror/gm1 [adding other fs, newfs, dump/restore filesystems] [rebooting from disk ada1] # gmirror remove gm0 ada0 # dd if=/dev/zero of=/dev/ada0 bs=512 count=10 # gmirror insert gm1 /dev/ada0 The only problem so far is this warning message when booting: gptboot: invalid backup GPT header whereas gpart(8) "show", "status" and "list" does not indicate any problem. Is this caused by geom_mirror usage? Cheers, -sascha From owner-freebsd-geom@FreeBSD.ORG Wed Feb 1 08:29:29 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60EE81065673 for ; Wed, 1 Feb 2012 08:29:29 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward4.mail.yandex.net (forward4.mail.yandex.net [IPv6:2a02:6b8:0:602::4]) by mx1.freebsd.org (Postfix) with ESMTP id C81828FC0C for ; Wed, 1 Feb 2012 08:29:28 +0000 (UTC) Received: from smtp1.mail.yandex.net (smtp1.mail.yandex.net [77.88.46.101]) by forward4.mail.yandex.net (Yandex) with ESMTP id 05B425004ED; Wed, 1 Feb 2012 12:29:26 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1328084967; bh=8oiMgnmgE8OWWsXx/rlbwHpMfTWztNn1yFLG6E58tY0=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=U+Vrm/g5Lk9Zb9Jdnd8lJKEsZvyHu40nkVtvFv7R1mH4ZUT3EBfTppSYG9AuC3cDd 9hYg75hS1z6CpYwym1rsAbC/6Ulp4QEsKU7SFxEtDHjygniQl7+kQA/DtqC29NU5gl FZfNrraBZxp0gECTWZAtLKgSO9YKjwgypwrHQkyo= Received: from smtp1.mail.yandex.net (localhost [127.0.0.1]) by smtp1.mail.yandex.net (Yandex) with ESMTP id C4629AA0343; Wed, 1 Feb 2012 12:29:26 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1328084966; bh=8oiMgnmgE8OWWsXx/rlbwHpMfTWztNn1yFLG6E58tY0=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=wUNPgsYU2P4SefzHAae5f534SOQ1EhYhUa3WDlqvlOOjkdAOQULzoJchUHxbR+I6T N+Q17ePpn7ArZtwzbXhNTZHG/hsydrCDpqmbu3KIbS6FBc3JFXDwY8373vdYiJPVrA ovqYgPQ4EPFZZe8TgcnspZZbf8EzSHTq0lmIRS5U= Received: from ns.kirov.so-ups.ru (ns.kirov.so-ups.ru [178.74.170.1]) by smtp1.mail.yandex.net (nwsmtp/Yandex) with ESMTP id TQU0CCCS-TQUm8G5p; Wed, 1 Feb 2012 12:29:26 +0400 X-Yandex-Spam: 1 Message-ID: <4F28F7E2.3070504@yandex.ru> Date: Wed, 01 Feb 2012 12:29:22 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: Sascha Klauder References: <9DB745B1-AA3D-4EF4-9A5E-4159EBFBC3E7@kcilink.com> <4F277200.8020003@yandex.ru> <20120131143818.GA2320@trimind.de> In-Reply-To: <20120131143818.GA2320@trimind.de> X-Enigmail-Version: 1.3.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigD9ECE1A7D28C9BE80A34F13F" Cc: Vick Khera , freebsd-geom@freebsd.org Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 08:29:29 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigD9ECE1A7D28C9BE80A34F13F Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable On 31.01.2012 18:38, Sascha Klauder wrote: > The only problem so far is this warning message when booting: >=20 > gptboot: invalid backup GPT header >=20 > whereas gpart(8) "show", "status" and "list" does not indicate > any problem. Is this caused by geom_mirror usage? It's because gptboot doesn't know about gmirror and it could not find GPT backup header in the last LBA. --=20 WBR, Andrey V. Elsukov --------------enigD9ECE1A7D28C9BE80A34F13F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) iQEcBAEBAgAGBQJPKPfmAAoJEAHF6gQQyKF6IHwIALi7bdti+bTN6dMvkgmmU8wh gcNv3OyWlgFC0J8LMKpy8ztM93TDZ3AMg+WM/Fe/ZU8p4KT1Ie4CMH1X6NbtLkQJ N3GWbdNF8asa1PnFk4Wl1hOhDFdjpvqew9yCl4IzFEdykkC+WrECc78pemkWy2sT HAeijt0YUtKD+aIx0q+4E8xOlOvBUkc7e6XH3lNbGXZk346jklkOxl83SV9I7Cq3 JhlEJKCLtUDOjfLVPa+/ZXvUI7IlHTh/f3vQUncqrQfEn4O8B9JYBWnRjcg+OaNM RualyO8ZM6wXbZNOeoetHYCiBl+eRX0ABcSw3O4SE0FiUc1Gbn2SbnmgiTPtK0o= =1oq7 -----END PGP SIGNATURE----- --------------enigD9ECE1A7D28C9BE80A34F13F-- From owner-freebsd-geom@FreeBSD.ORG Thu Feb 2 22:30:25 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E33ED106566B for ; Thu, 2 Feb 2012 22:30:25 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7C2FE8FC13 for ; Thu, 2 Feb 2012 22:30:25 +0000 (UTC) Received: by eekb47 with SMTP id b47so1057504eek.13 for ; Thu, 02 Feb 2012 14:30:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :content-type:content-transfer-encoding; bh=IQncnPQs+tYhMK3AvJs8qcveZfwB0tEe7+xpFvQq+1c=; b=yBmz9xjirZ51etZJWjhXBgYlTYGmd8iXa4TUmbGkGK8hVZdpCTMDAv+8LrWNNGWRo1 jgKlzNfw2aCAxz6EkcYexy9K0hegOYxt1Q++aHNOmYwTXvIJdQlUUqiB4Y3Q3AUm6Wfa NTncFdTbxfinwyttyL3vqudffbgBcHhwn24nU= Received: by 10.14.100.142 with SMTP id z14mr1495241eef.58.1328221824342; Thu, 02 Feb 2012 14:30:24 -0800 (PST) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id o49sm14243838eeb.7.2012.02.02.14.30.22 (version=SSLv3 cipher=OTHER); Thu, 02 Feb 2012 14:30:23 -0800 (PST) Sender: Alexander Motin Message-ID: <4F2B0E74.9070809@FreeBSD.org> Date: Fri, 03 Feb 2012 00:30:12 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120116 Thunderbird/9.0 MIME-Version: 1.0 To: Ravi Pokala Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-geom@FreeBSD.org Subject: Re: Bootable RAID10 on 9.0-RELEASE X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2012 22:30:26 -0000 Hi. > I expect there must be an easier way. If your system has one of supported software RAID BIOSes (Intel, AMD/Promise, NVIDIA, SiI, JMicron), you may just use geom_raid to boot from it's RAID10 volume with no additional magic. -- Alexander Motin From owner-freebsd-geom@FreeBSD.ORG Fri Feb 3 07:09:28 2012 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 896D0106566B; Fri, 3 Feb 2012 07:09:28 +0000 (UTC) (envelope-from rp_freebsd@mac.com) Received: from asmtpout017.mac.com (asmtpout017.mac.com [17.148.16.92]) by mx1.freebsd.org (Postfix) with ESMTP id 70C998FC0A; Fri, 3 Feb 2012 07:09:28 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from [192.168.1.3] ([98.210.97.211]) by asmtp017.mac.com (Oracle Communications Messaging Server 7u4-23.01 (7.0.4.23.0) 64bit (built Aug 10 2011)) with ESMTPA id <0LYT0024K2JQZ290@asmtp017.mac.com>; Fri, 03 Feb 2012 07:09:27 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.6.7361,1.0.260,0.0.0000 definitions=2012-02-03_02:2012-02-02, 2012-02-03, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 suspectscore=2 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=6.0.2-1012030000 definitions=main-1202020435 User-Agent: Microsoft-MacOutlook/14.14.0.111121 Date: Thu, 02 Feb 2012 21:35:35 -0800 From: Ravi Pokala Sender: Ravi Pokala To: Alexander Motin Message-id: Thread-topic: Bootable RAID10 on 9.0-RELEASE In-reply-to: <4F2B0E74.9070809@FreeBSD.org> Cc: freebsd-geom@FreeBSD.org Subject: Re: Bootable RAID10 on 9.0-RELEASE X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2012 07:09:28 -0000 -----Original Message----- From: Alexander Motin Date: Fri, 03 Feb 2012 00:30:12 +0200 To: Ravi Pokala Cc: Subject: Re: Bootable RAID10 on 9.0-RELEASE >Hi. > > > I expect there must be an easier way. > >If your system has one of supported software RAID BIOSes (Intel, >AMD/Promise, NVIDIA, SiI, JMicron), you may just use geom_raid to boot >from it's RAID10 volume with no additional magic. I played around with a VM (with all the IO I was doing, I'm real glad it was backed by SSDs!) before I had the actual hardware, and managed to get "close" to what I wanted: 1) Install on a temporary drive. 2) Set up GPT partitions, *including writing the pmbr and gptboot*, on each of the final drives. It took me a few attempts before I realized I was creating the partition table but not installing the bootcode. 3) Use `gmirror' to create a mirror of the partition which will hold the / filesystem, *on all four of the final drives*. That means you will end up with a quadruplicated /. 4) Use `gmirror' to create mirrors of da0pX and da1pX, and da2pX and da3pX on the final drives. 5) Use `gstripe' to create stripes across the corresponding mirrors. 6) Create filesystems on the quadruplicated mirror and the stripes. 7) Add the right entries to /boot/loader.conf, update /etc/fstab, etc. 8) Use 'dump | restore' to copy everything from the temporary drive to the four-way-mirror and the striped-mirrors. 9) Change the boot-order, and make sure you can boot off the four-way mirror and mount the striped-mirrors. When I got a chance to play with the actual hardware, I found that it has an LSI SAS controller which is supported by mfi(4). I ended up setting up the RAID10 in the pre-boot environment, then just creating GPT partitions on mfid0 and going from there. *Much* easier (once I dug up the documentation on LSI's website), works fine, and the interface offered by `mfiutil' looks pretty reasonable. Thanks, rp > >-- >Alexander Motin >_______________________________________________ >freebsd-geom@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-geom >To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org"