From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 09:37:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55A74106566C for ; Wed, 5 Jan 2011 09:37:01 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 0AFDA8FC08 for ; Wed, 5 Jan 2011 09:37:00 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1PaPnf-000N1w-2W; Wed, 05 Jan 2011 11:36:59 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Clifton Royston In-reply-to: Your message of Tue, 4 Jan 2011 10:01:49 -1000 . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 05 Jan 2011 11:36:59 +0200 From: Daniel Braniss Message-ID: Cc: freebsd-stable@freebsd.org Subject: Re: gstripe/gpart problems. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 09:37:01 -0000 > On Tue, Jan 04, 2011 at 04:21:31PM +0200, Daniel Braniss wrote: > > Hi, > > I have 2 ada disks striped: > > > > # gstripe list > > Geom name: s1 > > State: UP > > Status: Total=2, Online=2 > > Type: AUTOMATIC > > Stripesize: 65536 > > ID: 2442772675 > > Providers: > > 1. Name: stripe/s1 > > Mediasize: 1000215674880 (932G) > > Sectorsize: 512 > > Stripesize: 65536 > > Stripeoffset: 0 > > Mode: r0w0e0 > > Consumers: > > 1. Name: ada0 > > Mediasize: 500107862016 (466G) > > Sectorsize: 512 > > Mode: r0w0e0 > > Number: 0 > > 2. Name: ada1 > > Mediasize: 500107862016 (466G) > > Sectorsize: 512 > > Mode: r0w0e0 > > Number: 1 > > > > boot complains: > > > > GEOM_STRIPE: Device s1 created (id=2442772675). > > GEOM_STRIPE: Disk ada0 attached to s1. > > GEOM: ada0: corrupt or invalid GPT detected. > > GEOM: ada0: GPT rejected -- may not be recoverable. > > GEOM_STRIPE: Disk ada1 attached to s1. > > GEOM_STRIPE: Device s1 activated. > > > > # gpart show > > => 34 1953546173 stripe/s1 GPT (932G) > > 34 128 1 freebsd-boot (64K) > > 162 1953546045 - free - (932G) > > # gpart show > > => 34 1953546173 stripe/s1 GPT (932G) > > 34 128 1 freebsd-boot (64K) > > 162 1953546045 - free - (932G) > > > > # gpart add -t freebsd-ufs -s 20g stripe/s1 > > GEOM: ada0: corrupt or invalid GPT detected. > > GEOM: ada0: GPT rejected -- may not be recoverable. > > stripe/s1p2 added > > # gpart show > > => 34 1953546173 stripe/s1 GPT (932G) > > 34 128 1 freebsd-boot (64K) > > 162 41943040 2 freebsd-ufs (20G) > > 41943202 1911603005 - free - (912G) > > > > if I go the MBR road, all seems ok, but as soon as I try to write > > the boot block (boot0cfg -B /dev/stripe/s1) again the kernel > > starts to complain about corrupted GEOM too. > > So are you trying to partition the drives and then stripe the > partitions within the drives, or are you trying to partition the > stripe? > > It seems here as though you might be trying to first partition the > drives (not clear on that) then stripe the whole drives - which will > mean the partition info is wrong for the resulting striped drive set - > and then repartition the striped drive set, and neither is ending up > valid. > > If what you are intending is to partition after striping the raw > drives, then you are doing the right steps, but when the geom layer > tries to look at the info on the individual drives as at boot, it will > find it invalid. If it the gpart layer is actually refusing to write > partition info to the drives which is wrong for the drives taken > individually, that would account for your problems. > > One valid order to do things in would be partition the drives with > gpart, creating identical sets of partitions on both drives, then > stripe the partitions created within them (syntax not exact): > > gpart add -t freebsd-ufs0 -s 10g ada0 > gpart add -t freebsd-ufs1 -s 10g ada1 > gstripe label freebsd-ufs freebsd-ufs0 freebsd-ufs1 > > That would give you a 20GB stripe, with valid partition info on each > drive. > > If this will be your boot drive, depending on how much needs to be read > from the drive before the geom_stripe kernel module gets loaded, I > would think there could also be a problem booting from the drive. This > is not like gmirroring two drives or partitions, where the info read > from either disk early in boot will be identical, and identical (except > for the last block of the partition) to what the OS sees later after > the mirror is formed. > > I assume you're bearing in mind that if you lose either drive to a > hardware fault you lose the whole thing, and consider the risk worth > the potential speed/size gain. > -- Clifton Hi Clifton, I was getting very frustrated yesterday, hence the cripted message, your response requieres some background :-) the box is a Sun Fire X2200, which has bays for 2 disks, (we have several of these) before the latest upgrade, the 2 disks were 'raided' via 'nVidia MediaShield' and appeared as ar0, when I upgraded to 8.2, it disappeared, since I had in the kernel config file ATA_CAM. So I starded fiddling with gstripe, which 'recoverd' the data. Next, since the kernel boot kept complaining abouf GEOM errors, (and not wanting to mislead the operators) I cleaned up the data, and started from scratch. the machine boots diskless, but I like to keep a root bootable partition just in case. the process was in every case the same, first the stripe, then gpart the stripe. btw, I know that if I loose a disk I lose everything. Also that I wont be able to boot from it (I can always boot via the net, and mount the root localy, or some other combination - USB, etc) But you gave me some ideas and will start experimenting soon. thanks, danny