From owner-freebsd-geom@FreeBSD.ORG Thu Apr 27 21:22:09 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 631E716A402 for ; Thu, 27 Apr 2006 21:22:09 +0000 (UTC) (envelope-from ender@tog.net) Received: from tog.net (tog.net [216.89.226.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 604F843D70 for ; Thu, 27 Apr 2006 21:22:07 +0000 (GMT) (envelope-from ender@tog.net) Received: from [192.168.1.101] (unknown [216.89.225.137]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tog.net (Postfix) with ESMTP id E6B2C29B5A2 for ; Thu, 27 Apr 2006 17:22:06 -0400 (EDT) Message-ID: <4451355E.801@tog.net> Date: Thu, 27 Apr 2006 17:19:26 -0400 From: Ender User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: GEOM_MIRROR: Component concat/leg2 (device data) broken, skipping X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Apr 2006 21:22:09 -0000 I have a gmirror provider named "data" with 2 consumers "leg1" and "leg2". Each gconcat provider has 4 consumers. The gconcat consumers are actually ggatec devices. Geom name: leg1 State: UP Status: Total=4, Online=4 Type: AUTOMATIC ID: 1899079681 Providers: 1. Name: concat/leg1 Mediasize: 675976717312 (630G) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: ggate14 Mediasize: 95865781248 (89G) Sectorsize: 512 Mode: r1w1e2 Start: 580110936576 End: 675976717312 2. Name: ggate11 Mediasize: 120000000000 (112G) Sectorsize: 512 Mode: r1w1e2 Start: 0 End: 119999999488 3. Name: ggate13 Mediasize: 300069052416 (279G) Sectorsize: 512 Mode: r1w1e2 Start: 280041884672 End: 580110936576 4. Name: ggate12 Mediasize: 160041885696 (149G) Sectorsize: 512 Mode: r1w1e2 Start: 119999999488 End: 280041884672 Geom name: leg2 State: UP Status: Total=4, Online=4 Type: AUTOMATIC ID: 260049114 Providers: 1. Name: concat/leg2 Mediasize: 676009618432 (630G) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: ggate21 Mediasize: 120000000000 (112G) Sectorsize: 512 Mode: r0w0e0 Start: 0 End: 119999999488 2. Name: ggate22 Mediasize: 160041885696 (149G) Sectorsize: 512 Mode: r0w0e0 Start: 119999999488 End: 280041884672 3. Name: ggate23 Mediasize: 300069052416 (279G) Sectorsize: 512 Mode: r0w0e0 Start: 280041884672 End: 580110936576 4. Name: ggate24 Mediasize: 95898682368 (89G) Sectorsize: 512 Mode: r0w0e0 Start: 580110936576 End: 676009618432 # ggatec list ggate14 ggate11 ggate13 ggate12 ggate21 ggate22 ggate23 ggate24 The problem is my gmiror degraded. # gmirror list Geom name: data State: DEGRADED Components: 2 Balance: round-robin Slice: 4096 Flags: NONE GenID: 1 SyncID: 1 ID: 3615665854 Providers: 1. Name: mirror/data Mediasize: 675976716800 (630G) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: concat/leg1 Mediasize: 675976717312 (630G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 1 SyncID: 1 ID: 1250101353 And when i try andn insert a new disk i get this> GEOM_MIRROR: Component concat/leg2 (device data) broken, skipping. GEOM_MIRROR: Cannot add disk concat/leg2 to data (error=22). I set kern.geom.mirror.debug: 1, and i tryed to dd if=/dev/zero of=/dev/ad* for each disk in leg2 as shown here: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2005-01/2574.html no luck, any suggestions? Here is what i did to start this setup: #ON 10.10.1.10 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 11 10.10.1.101 /dev/ad1 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 12 10.10.1.101 /dev/ad2 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 13 10.10.1.101 /dev/ad3 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 14 10.10.1.101 /dev/ad0s1f ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 21 10.10.1.102 /dev/ad1 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 22 10.10.1.102 /dev/ad2 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 23 10.10.1.102 /dev/ad3 ggatec create -o rw -q 2048 -S 4194304 -R 4194304 -t 600 -u 24 10.10.1.102 /dev/ad0s1f #ON 10.10.1.10 gconcat label -v leg1 /dev/ggate11 /dev/ggate12 /dev/ggate13 /dev/ggate14 gconcat label -v leg2 /dev/ggate21 /dev/ggate22 /dev/ggate23 /dev/ggate24 gmirror label -v -b round-robin data /dev/concat/leg1 /dev/concat/leg2 newfs -U /dev/mirror/data mount /dev/mirror/data /data #ON 10.10.1.101 and 10.10.1.102 #!/bin/sh # # PROVIDE: ggated # REQUIRE: bar_service_required_to_precede_foo . /etc/rc.subr name="ggated" rcvar=`set_rcvar` command="/sbin/ggated" command_args="-R 4194304 -S 4194304" load_rc_config $name run_rc_command "$1"