From owner-freebsd-stable@FreeBSD.ORG Fri Jul 7 14:35:55 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BC9B716A4DF; Fri, 7 Jul 2006 14:35:55 +0000 (UTC) (envelope-from goran.lowkrantz@ismobile.com) Received: from mail.ismobile.com (tandgrisner.ismobile.com [213.88.244.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 157D643D46; Fri, 7 Jul 2006 14:35:54 +0000 (GMT) (envelope-from goran.lowkrantz@ismobile.com) Received: from [172.16.2.106] (viglaf.hq.ismobile.com [172.16.2.106]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.ismobile.com (Postfix) with ESMTP id E47C92280B; Fri, 7 Jul 2006 16:35:53 +0200 (CEST) Message-ID: <44AE7149.6080102@ismobile.com> Date: Fri, 07 Jul 2006 16:35:53 +0200 From: glz Organization: isMobile AB User-Agent: Thunderbird 1.5.0.4 (X11/20060605) MIME-Version: 1.0 To: le@FreeBSD.org References: <24C282E4D260624D50275B43@[10.255.253.2]> In-Reply-To: <24C282E4D260624D50275B43@[10.255.253.2]> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org Subject: Re: Problem restarting gvinum raid-5 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jul 2006 14:35:55 -0000 First, missed some info: > uname -a FreeBSD byleist.hq.ismobile.com 6.1-STABLE FreeBSD 6.1-STABLE #1: Mon Jun 26 20:37:45 CEST 2006 root@byleist.hq.ismobile.com:/usr/obj/usr/src/sys/BYLEISTSMP i386 So I continued to dig into this and it seems that the plex is not rebuilding because the geom is open, i.e. this routine: /* Check if any consumer of the given geom is open. */ int gv_is_open(struct g_geom *gp) { struct g_consumer *cp; if (gp == NULL) return (0); LIST_FOREACH(cp, &gp->consumer, consumer) { if (cp->acr || cp->acw || cp->ace) return (1); } return (0); } What does this mean? How do I make sure the geom is not opened until I can start the plex? I have tested single user and not mounted file system but it does not help. From the code I have read, this state should be the proper for allowing a rebuild of the plex: 5 drives: D disk5 State: up /dev/da6s1a A: 0/17492 MB (0%) D disk4 State: up /dev/da5s1a A: 0/17492 MB (0%) D disk3 State: up /dev/da4s1a A: 0/17492 MB (0%) D disk2 State: up /dev/da3s1a A: 0/17492 MB (0%) D disk1 State: up /dev/da2s1a A: 0/17492 MB (0%) 1 volume: V imap State: up Plexes: 1 Size: 68 GB 1 plex: P imap.p0 R5 State: degraded Subdisks: 5 Size: 68 GB 5 subdisks: S imap.p0.s0 State: up D: disk1 Size: 17 GB S imap.p0.s1 State: up D: disk2 Size: 17 GB S imap.p0.s2 State: up D: disk3 Size: 17 GB S imap.p0.s3 State: up D: disk4 Size: 17 GB S imap.p0.s4 State: stale D: disk5 Size: 17 GB /glz Goran Lowkrantz wrote: > Hi, > > We have a gvinum raid-5 volume that that we had to replace a disk on and > after that we cant get the new subdisk starting. > > Here are the things we did: > 1: Replace disk and boot singleuser to fdisk and lable new disk: > gvinum -> list > 5 drives: > D disk4 State: up /dev/da5s1a A: 0/17492 MB (0%) > D disk3 State: up /dev/da4s1a A: 0/17492 MB (0%) > D disk2 State: up /dev/da3s1a A: 0/17492 MB (0%) > D disk1 State: up /dev/da2s1a A: 0/17492 MB (0%) > > 1 volume: > V imap State: up Plexes: 1 Size: 68 GB > > 1 plex: > P imap.p0 R5 State: up Subdisks: 5 Size: 68 GB > > 5 subdisks: > S imap.p0.s0 State: up D: disk1 Size: 17 GB > S imap.p0.s1 State: up D: disk2 Size: 17 GB > S imap.p0.s2 State: up D: disk3 Size: 17 GB > S imap.p0.s3 State: up D: disk4 Size: 17 GB > S imap.p0.s4 State: up D: disk5 Size: 17 GB > > After fixing the new disk partition we did a saveconfig and reboot: > gvinum -> list > 5 drives: > D disk5 State: up /dev/da6s1a A: 0/17492 MB (0%) > D disk4 State: up /dev/da5s1a A: 0/17492 MB (0%) > D disk3 State: up /dev/da4s1a A: 0/17492 MB (0%) > D disk2 State: up /dev/da3s1a A: 0/17492 MB (0%) > D disk1 State: up /dev/da2s1a A: 0/17492 MB (0%) > > 1 volume: > V imap State: up Plexes: 1 Size: 68 GB > > 1 plex: > P imap.p0 R5 State: up Subdisks: 5 Size: 68 GB > > 5 subdisks: > S imap.p0.s4 State: stale D: disk5 Size: 17 GB > S imap.p0.s3 State: up D: disk4 Size: 17 GB > S imap.p0.s2 State: up D: disk3 Size: 17 GB > S imap.p0.s1 State: up D: disk2 Size: 17 GB > S imap.p0.s0 State: up D: disk1 Size: 17 GB > > Tried start on plex and subdisk, nnot working. Finally, to get plex into > degraded mode we did a setstate down imap.p0.s4. > gvinum -> list > 5 drives: > D disk5 State: up /dev/da6s1a A: 0/17492 MB (0%) > D disk4 State: up /dev/da5s1a A: 0/17492 MB (0%) > D disk3 State: up /dev/da4s1a A: 0/17492 MB (0%) > D disk2 State: up /dev/da3s1a A: 0/17492 MB (0%) > D disk1 State: up /dev/da2s1a A: 0/17492 MB (0%) > > 1 volume: > V imap State: up Plexes: 1 Size: 68 GB > > 1 plex: > P imap.p0 R5 State: degraded Subdisks: 5 Size: 68 GB > > 5 subdisks: > S imap.p0.s4 State: down D: disk5 Size: 17 GB > S imap.p0.s3 State: up D: disk4 Size: 17 GB > S imap.p0.s2 State: up D: disk3 Size: 17 GB > S imap.p0.s1 State: up D: disk2 Size: 17 GB > S imap.p0.s0 State: up D: disk1 Size: 17 GB > > and here we are. Start on volume or plex give errno 16, start on subdisk > gives can't start: cannot start 'imap.p0.s4' - not yet supported. > > Can't find any descriptions of the proper way to do disk replacement, so > if this is wrong, I'd love to get updated. And how do we get the current > situation upa nd running? > > Regards, > Göran > > > > ................................................... the future isMobile > > Goran Lowkrantz > System Architect, isMobile, Aurorum 2, S-977 75 Luleå, Sweden > Phone: +46(0)920-75559 > Mobile: +46(0)70-587 87 82 Fax: +46(0)70-615 87 82 > > http://www.ismobile.com ............................................... > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- ................................................... the future isMobile Goran Lowkrantz System Architect, isMobile, Aurorum 2, S-977 75 Luleå, Sweden Phone: +46(0)920-75559 Mobile: +46(0)70-587 87 82 Fax: +46(0)70-615 87 82 http://www.ismobile.com ...............................................