From owner-freebsd-questions@FreeBSD.ORG Tue Aug 12 23:51:25 2003 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 039CD37B401; Tue, 12 Aug 2003 23:51:25 -0700 (PDT) Received: from eternal.slumber.org (eternal.slumber.org [68.98.216.195]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D9C543F3F; Tue, 12 Aug 2003 23:51:24 -0700 (PDT) (envelope-from flagg@eternal.slumber.org) Received: from eternal.slumber.org (localhost [127.0.0.1]) h7D6qASH008988; Tue, 12 Aug 2003 23:52:10 -0700 Received: (from flagg@localhost) by eternal.slumber.org (8.12.9/8.12.9/Debian-5) id h7D6qAkX008986; Tue, 12 Aug 2003 23:52:10 -0700 Date: Tue, 12 Aug 2003 23:52:09 -0700 From: Shawn Ostapuk To: "Greg 'groggy' Lehey" Message-ID: <20030813065209.GB3331@slumber.org> References: <20030812164049.GA5490@slumber.org> <20030813054510.GO23460@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030813054510.GO23460@wantadilla.lemis.com> User-Agent: Mutt/1.5.4i cc: freebsd-questions@FreeBSD.org Subject: Re: problem adding subdisk to vinum X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Aug 2003 06:51:25 -0000 (Sorry for being so long, thought it would be less confusing and save additional questions to show complete vinum lists) > > su-2.03# vinum lv -r pr0n > > V pr0n State: up Plexes: 1 Size: 1172 GB > > P vinum0.p0 C State: corrupt Subdisks: 11 Size: 1172 GB > > S vinum0.p0.s0 State: up PO: 0 B Size: 152 GB > > ... > > Hmm, that plex shouldn't have been corrupt. But plex states are > pretty meaningless anyway. Actually I may have posted the wrong line previously (i've been trying a million things). This is what it is normally (before) (the format or the numbers has changed from 1 to 01, i was trying everything :) vsu-2.03# vinum lv -r V pr0n State: up Plexes: 1 Size: 1023 GB P vinum0.p0 C State: up Subdisks: 10 Size: 1023 GB S vinum0.p0.s00 State: up PO: 0 B Size: 152 GB S vinum0.p0.s01 State: up PO: 152 GB Size: 28 GB S vinum0.p0.s02 State: up PO: 181 GB Size: 76 GB S vinum0.p0.s03 State: up PO: 257 GB Size: 76 GB S vinum0.p0.s04 State: up PO: 333 GB Size: 76 GB S vinum0.p0.s05 State: up PO: 410 GB Size: 76 GB S vinum0.p0.s06 State: up PO: 486 GB Size: 76 GB S vinum0.p0.s07 State: up PO: 562 GB Size: 74 GB S vinum0.p0.s08 State: up PO: 637 GB Size: 233 GB S vinum0.p0.s09 State: up PO: 871 GB Size: 152 GB > > after: > > S vinum0.p0.s10 State: empty PO: 1023 GB Size: 149 GB > > > > the old filesystem is not accessable at this point. invalid superblock > > and other errors with fsck and growfs. > > That's the kind of message I'd want to see. But given that the > subdisk isn't available, that's not surprising. After adding the new disk, it (confirmed) always looks like su-2.03# vinum lv -r V pr0n State: up Plexes: 1 Size: 1172 GB P vinum0.p0 C State: corrupt Subdisks: 11 Size: 1172 GB S vinum0.p0.s00 State: up PO: 0 B Size: 152 GB S vinum0.p0.s01 State: up PO: 152 GB Size: 28 GB S vinum0.p0.s02 State: up PO: 181 GB Size: 76 GB S vinum0.p0.s03 State: up PO: 257 GB Size: 76 GB S vinum0.p0.s04 State: up PO: 333 GB Size: 76 GB S vinum0.p0.s05 State: up PO: 410 GB Size: 76 GB S vinum0.p0.s06 State: up PO: 486 GB Size: 76 GB S vinum0.p0.s07 State: up PO: 562 GB Size: 74 GB S vinum0.p0.s08 State: up PO: 637 GB Size: 233 GB S vinum0.p0.s09 State: up PO: 871 GB Size: 152 GB S vinum0.p0.s10 State: empty PO: 1023 GB Size: 149 GB At this point i can either 1) reboot, and do vinum start vinum0.p0.s10 to get it to list as up (it says device busy if i try before reboot), or 2) setstate up vinum0.p0.s10, or 3) i even tried "init vinum0.p0.s10" which took forever but set the state as up but still no success At this point lv -r shows everything as up. same as above but with total size as 1172 GB. vinum list _looks_ perfect. su-2.03# vinum lv -r V pr0n State: up Plexes: 1 Size: 1172 GB P vinum0.p0 C State: up Subdisks: 11 Size: 1172 GB S vinum0.p0.s00 State: up PO: 0 B Size: 152 GB S vinum0.p0.s01 State: up PO: 152 GB Size: 28 GB S vinum0.p0.s02 State: up PO: 181 GB Size: 76 GB S vinum0.p0.s03 State: up PO: 257 GB Size: 76 GB S vinum0.p0.s04 State: up PO: 333 GB Size: 76 GB S vinum0.p0.s05 State: up PO: 410 GB Size: 76 GB S vinum0.p0.s06 State: up PO: 486 GB Size: 76 GB S vinum0.p0.s07 State: up PO: 562 GB Size: 74 GB S vinum0.p0.s08 State: up PO: 637 GB Size: 233 GB S vinum0.p0.s08 State: up PO: 637 GB Size: 233 GB S vinum0.p0.s09 State: up PO: 871 GB Size: 152 GB S vinum0.p0.s10 State: up PO: 1023 GB Size: 149 GB Either method results in an umountable and unaccessable filesystem. su-2.03# mount /dev/vinum/pr0n /data mount: /dev/vinum/pr0n on /data: incorrect super block su-2.03# fsck /dev/vinum/pr0n ** /dev/vinum/pr0n CANNOT READ: BLK 16 CONTINUE? [yn] ^C If i continue it says cannot read blk 17,18, etc. If i do fsck -b32, it does the same thing but for the 32>= values. i thought i saved the growfs error, but i guess i didnt. it was basically something like "growfs: rdfs: something error 16". normally i test by trying to mount or fsck the old filesystem. > > after chanigng the state to up "setstate up vinum0.p0.s10", the > > behaviour would also be the same. > > Really? Then I definitely want to see it. > > > (i am still unsure as to the empty state it says, as the man page > > says that if adding a subdisk to a plex to a volume that has no > > other plex's it would bring the state to up immediately. (i am also > > not positive at this point the state was always set to empty, i > > believe at least a few times it did come up with State: up) > > It's best to check this. It shouldn't be up immediately. I havent been able to get it "up" on default after adding the new disk, so that looks okay then. > > also, i have more than double verified the drive in the right place > > and is functioning. after having trouble i even re-fdisk'd the new > > drive (ad7e) and newfs's it resulting in zero errors or > > problems. the bios detects it quickly with no delays as well. > > That's not the issue. The real question is the states. Getting > states right is a black art, and it's possible that that's your real > problem. It didn't seem like a disk issue, but with the unusual problem i was trying to make aboslutely sure. > OK. Start from your last status (volume up, plex corrupt, last > subdisk empty). Set plex and last subdisk to up. Run fsck and tell > me what you get. If that works, try growfs. If that workds, do a > saveconfig. I've probably added the new disk over 100 times now trying every possible option. I can get vinum to list everything like its perfect, but once the new drive is listed, I am always completely unable to access the filesystem. I've tried init'd the subdisk, i've made smaller partitions on the new drive (tried adding a 25 gig paritition) in case i was reaching some limits, i've backported to the 4.7 release of vinum (45 lines difference i think was it) since i had upgraded to 4.8 since the last time i successfully added a drive, tried the old kernel as well, all without any luck. (just to give you an idea of how thorough i've been trying every possible option). Basically I've had vinum list _look_ good consistantly, but never ever can fsck or mount the filesystem. Half the subdisks currently on there were added this same way but never gave me any problems -- the behaviour definately does not seem to be acting the same way as before. doing a resetconfig, and vinum create -f origvinum.conf always returns access to the filesystem with no problems. If this is interesting enough you wanna take a look i'd be happy to arrange access, or provide any additional information. Thanks again. shawn