Date: Wed, 19 Jan 2005 22:57:21 -0800 From: "Ted Mittelstaedt" <tedm@toybox.placo.com> To: "Drew Tomlinson" <drew@mykitchentable.net>, "FreeBSD Questions" <freebsd-questions@freebsd.org> Subject: RE: One Last Plea For Vinum Assistance Message-ID: <LOBBIFDAGNMAMLGJJCKNAEBEFAAA.tedm@toybox.placo.com> In-Reply-To: <41ED86F4.1080301@mykitchentable.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Drew, Please read the following: http://www.vinumvm.org/vinum/how-to-debug.html And follow the instructions exactly. And I mean exactly. Also keep the following in mind, Greg will try to help but note carefully the sentence on this webpage: "Since I wrote it, FreeBSD has changed its I/O structure, breaking many things in Vinum. At the time of writing, a new version, provisionally called gvinum, is being written" I myself have had one serious crash on a vinum RAID volume as a result of a SCSI cable problem that blew away the volume. (2 drives were corrupted, instead of just one, making it impossible for the volume manager to repair by itself) I sent all the info to Greg but ultimately he wasn't able to offer any suggestions on recovering the array so I just wiped it and started over. Note that Greg DID NOT recommend wiping the array. In fact he didn't recommend anything. The lack of any recommendation appears to be his way of telling you "your volume is screwed, wipe it and start over" Like most UNIX commands, if Greg has nothing to offer, he says nothing at all, he won't tell you he has nothing to offer. So, the lack of a response to your original post you can probably take as an answer, to be honest. This did teach me a lesson that I kind of knew already but didn't think too much about. That is, a software array is no substitute for a hardware array. In other words, vinum is a great thing if what your wanting to do is use a bunch of cheap disks and cheap controller cards to either get a giant partition, or to stripe them together and get faster access. But it's not so good if the intent is to get some crash recovery. I don't use and have never used vinum for /etc, /, /usr, /var or any other system partitions. I only use it for partitions that I want to mount AFTER the system is booted. If I were in your shoes I'd nuke your system and start all over again and rethink how I had it laid out. I would use a single disk for the system then take the rest of the disks and put them together under vinum. Then I'd mount that on /ftp and I'd softlink whatever thing is gopping up space under /usr, for example /usr/local/www, to a directory under /ftp Vinum isn't going to give you any crash recovery for /usr so there is really no point in making /usr a vinum volume. Beyond that I really don't understand why you are putting /usr as a vinum volume, espically as you yourself said "Fortunately this volume is up and running or I would really be in a mess" I mean, your basically saying your hitting yourself in the face and you feel fortunate you haven't broken your nose yet. Anyway, one other thing I will bring up: How exactly did you update your system? Did you nuke and repave it? Or did you follow the instructions here EXACTLY: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html If you didn't do one or the other of these things then nobody is going to help you. Ted > -----Original Message----- > From: owner-freebsd-questions@freebsd.org > [mailto:owner-freebsd-questions@freebsd.org]On Behalf Of Drew Tomlinson > Sent: Tuesday, January 18, 2005 2:00 PM > To: FreeBSD Questions > Subject: One Last Plea For Vinum Assistance > > > I sent the message below a couple of times but did not receive > any response. > I assume that it's because either I have a really difficult > problem or am > asking something really stupid. :) But anyway, I want to install > additional > memory in this machine and am sure I will run across the same problems > after > shutting down. So if anyone has any suggestions on how I > might solve this > issue, I'd really appreciate the input. > > Thanks, > > Drew > > --- Original Message --- > Since an upgrade from 4.9 to 4.10, I've had problems with > vinum. The basic > problem is that upon reboot, two of my vinum drives show up as > "referenced" and > thus create the associated chaos. I've tried many things and fiddled > around > quite a bit so I can't say exactly what I've done. I can > include all of > the > entries in the history file since Oct. 31 if that's a help but > it would > be a > long list. > > So prior to digging that deep, I will describe where I stand > currently and > where I want to finish. Currently, I have one vinum volume > that I use for > /usr. Fortunately this volume is up and running or I would > really be in a > mess. Here's the 'vinum list' output in this state: > > blacklamb# vinum > vinum -> list > 2 drives: > D disk1 State: up Device /dev/da0s1h Avail: > 0/8383 MB (0%) > D disk2 State: up Device /dev/da1s1h Avail: > 0/8383 MB (0%) > > 1 volumes: > V usr State: up Plexes: 1 Size: > 16 GB > > 1 plexes: > P usr.p0 S State: up Subdisks: 2 Size: > 16 GB > > 2 subdisks: > S usr.p0.s0 State: up PO: 0 B Size: > 8383 MB > S usr.p0.s1 State: up PO: 256 kB Size: > 8383 MB > > I want to add another volume and mount it on /ftp. After creating the > volume, > vinum sees it and it appears OK as indicated in this output: > > vinum -> list > 5 drives: > D disk1 State: up Device /dev/da0s1h Avail: > 0/8383 MB (0%) > D disk2 State: up Device /dev/da1s1h Avail: > 0/8383 MB (0%) > D ftp1 State: up Device /dev/ad0s1h Avail: > 0/76319 MB (0%) > D ftp2 State: up Device /dev/ad1s1h Avail: > 0/76319 MB (0%) > D ftp3 State: up Device /dev/da3s1h Avail: > 0/114473 MB (0%) > > 2 volumes: > V usr State: up Plexes: 1 Size: > 16 GB > V ftp State: up Plexes: 1 Size: > 260 GB > > 2 plexes: > P usr.p0 S State: up Subdisks: 2 Size: > 16 GB > P ftp.p0 C State: up Subdisks: 3 Size: > 260 GB > > 5 subdisks: > S usr.p0.s0 State: up PO: 0 B Size: > 8383 MB > S usr.p0.s1 State: up PO: 256 kB Size: > 8383 MB > S ftp.p0.s0 State: up PO: 0 B Size: > 74 GB > S ftp.p0.s1 State: up PO: 74 GB Size: > 74 GB > S ftp.p0.s2 State: up PO: 149 GB Size: > 111 GB > > Next I 'quit' the vinum command line and fsck my newly created > volume. It > finishes successfully and I mount it: > > blacklamb# df > Filesystem 1K-blocks Used Avail Capacity Mounted on > /dev/da0s1a 302350 102088 176074 37% / > /dev/vinum/usr 16639674 5433762 9874739 35% /usr > procfs 4 4 0 100% /proc > /dev/vinum/ftp 265119539 119729707 132133856 48% /ftp > > I recheck with the 'vinum list' command and the output is the same as > above. > Now I cross my fingers and reboot to see if the volume comes > up properly on > startup. However ftp3 (/dev/da3s1h) comes up 'referenced' and causes > problems. > > Mounting root from ufs:/dev/da0s1a > dumpon: crash dumps to /dev/da1s1b (13, 131081) > vinum: loaded > vinum: reading configuration from /dev/ad1s1h > vinum: ftp.p0.s2 is crashed > vinum: ftp.p0 is corrupt > vinum: updating configuration from /dev/ad0s1h > vinum: updating configuration from /dev/da1s1h > vinum: updating configuration from /dev/da0s1h > vinum: /dev is mounted read-only, not rebuilding /dev/vinum > Warning: defective objects > > D ftp3 State: referenced Device Avail: 0/0 MB > P ftp.p0 C State: corrupt Subdisks: 3 Size: > 260 GB > S ftp.p0.s2 State: crashed PO: 149 GB Size: > 111 GB > swapon: adding /dev/da1s1b as swap device > Automatic boot in progress... > /dev/da0s1a: FILESYSTEM CLEAN; SKIPPING CHECKS > /dev/da0s1a: clean, 100130 free (938 frags, 12399 blocks, 0.6% > fragmentation) > /dev/vinum/ftp: CANNOT READ: BLK 546979872 > /dev/vinum/ftp: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. > /dev/vinum/ftp: CANNOT WRITE: BLK 2800 > /dev/vinum/ftp: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. > /dev/vinum/usr: FILESYSTEM CLEAN; SKIPPING CHECKS > /dev/vinum/usr: clean, 11205903 free (152463 frags, 1381680 > blocks, 0.9% > fragmentation) > THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: > /dev/vinum/ftp (/ftp) > File system preen failed, trying fsck -y . . . > ** /dev/da0s1a > ** Last Mounted on / > ** Root file system > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > ** Phase 3 - Check Connectivity > ** Phase 4 - Check Reference Counts > ** Phase 5 - Check Cyl groups > 2053 files, 51045 used, 100130 free (938 frags, 12399 blocks, 0.6% > fragmentation) > ** /dev/vinum/usr > ** Last Mounted on /usr > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > ** Phase 3 - Check Connectivity > ** Phase 4 - Check Reference Counts > ** Phase 5 - Check Cyl groups > 401072 files, 5433771 used, 11205903 free (152463 frags, > 1381680 blocks, > 0.9% fragmentation) > ** /dev/vinum/ftp > > CANNOT READ: BLK 546979872 > UNEXPECTED SOFT UPDATE INCONSISTENCY > > CONTINUE? yes > > THE FOLLOWING DISK SECTORS COULD NOT BE READ: 546979872, 546979873, > 546979874, 546979875, > /dev/vinum/ftp: CANNOT FIGURE OUT FILE SYSTEM PARTITION > > UNEXPECTED SOFT UPDATE INCONSISTENCY > WARNING: R/W mount of /ftp denied. Filesystem is not clean - run fsck > mount: /dev/vinum/ftp: Operation not permitted > Mounting /etc/fstab filesystems failed, startup aborted > Enter full pathname of shell or RETURN for /bin/sh: > > I enter the shell and run 'vinum list': > > vinum -> list > 4 drives: > D ftp1 State: up Device /dev/ad0s1h Avail: > 0/76319 MB (0%) > D ftp2 State: up Device /dev/ad1s1h Avail: > 0/76319 MB (0%) > D disk1 State: up Device /dev/da0s1h Avail: > 0/8383 MB (0%) > D disk2 State: up Device /dev/da1s1h Avail: > 0/8383 MB (0%) > D ftp3 State: referenced Device Avail: 0/0 MB > > 2 volumes: > V usr State: up Plexes: 1 Size: > 16 GB > V ftp State: up Plexes: 1 Size: > 260 GB > > 2 plexes: > P usr.p0 S State: up Subdisks: 2 Size: > 16 GB > P ftp.p0 C State: corrupt Subdisks: 3 Size: > 260 GB > > 5 subdisks: > S usr.p0.s0 State: up PO: 0 B Size: > 8383 MB > S usr.p0.s1 State: up PO: 256 kB Size: > 8383 MB > S ftp.p0.s0 State: up PO: 0 B Size: > 74 GB > S ftp.p0.s1 State: up PO: 74 GB Size: > 74 GB > S ftp.p0.s2 State: crashed PO: 149 GB Size: > 111 GB > > Now I delete all references to the ftp volume by running these > commands: > > vinum -> rm -f ftp.p0.s2 > vinum: removing ftp.p0.s2 > Correcting length of ftp.p0: was 547043829, is 312602446 > vinum: ftp.p0 is up > vinum -> rm -f ftp.p0.s1 > vinum: removing ftp.p0.s1 > Correcting length of ftp.p0: was 312602446, is 156301223 > vinum -> rm -f ftp.p0.s0 > vinum: removing ftp.p0.s0 > vinum -> rm -f ftp.p0 vinum: removing ftp.p0 > vinum: ftp is down > vinum -> rm -f ftp vinum: removing ftp > vinum -> rm -f ftp3 > vinum -> rm -f ftp2 > vinum -> rm -f ftp1 > > Yet the 'reference to ftp3 remains. > > vinum -> list > 1 drives: > D disk1 State: up Device /dev/da0s1h Avail: > 0/8383 MB (0%) > D disk2 State: up Device /dev/da1s1h Avail: > 0/8383 MB (0%) > D ftp3 State: referenced Device Avail: 0/0 MB > > 1 volumes: > V usr State: up Plexes: 1 Size: > 16 GB > > 1 plexes: > P usr.p0 S State: up Subdisks: 2 Size: > 16 GB > > 2 subdisks: > S usr.p0.s0 State: up PO: 0 B Size: > 8383 MB > S usr.p0.s1 State: up PO: 256 kB Size: > 8383 MB > > Why does the list output show "1 drives:" when there are three listed > and two > working? Anyway, I've been here before and I quit vinum. > Then I issue > these > commands: > > # df > Filesystem 512-blocks Used Avail Capacity Mounted on > /dev/da0s1a 604700 204180 352144 37% / > /dev/vinum/usr 33279348 10867544 19749458 35% /usr > procfs 8 8 0 100% /proc > # umount /usr > # vinum stop > vinum: unloaded > vinum unloaded > # vinum start > vinum: loaded > vinum: reading configuration from /dev/da1s1h > vinum: updating configuration from /dev/da0s1h > > Then I issue a 'vinum list' and all seems well. > > # vinum > vinum -> list > 2 drives: > D disk1 State: up Device /dev/da0s1h Avail: > 0/8383 MB (0%) > D disk2 State: up Device /dev/da1s1h Avail: > 0/8383 MB (0%) > > 1 volumes: > V usr State: up Plexes: 1 Size: > 16 GB > > 1 plexes: > P usr.p0 S State: up Subdisks: 2 Size: > 16 GB > > 2 subdisks: > S usr.p0.s0 State: up PO: 0 B Size: > 8383 MB > S usr.p0.s1 State: up PO: 256 kB Size: > 8383 MB > vinum -> > > Since everything appears correct, this seems like a good place > to do 'vinum > saveconfig'. Next I edit /etc/fstab so only /ftp is not mounted and > reboot. > The system reboots fine and the 'vinum list' is as it is > immediately above. > > > Running in full production mode, I create the ftp volume again: > > vinum -> create -f vinum_ftp.conf > Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp1 is up > Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp2 is up > Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp3 is up > 5 drives: > D disk1 State: up Device /dev/da0s1h Avail: > 0/8383 MB (0%) > D disk2 State: up Device /dev/da1s1h Avail: > 0/8383 MB (0%) > D ftp1 State: up Device /dev/ad0s1h Avail: > 0/76319 MB (0%) > D ftp2 State: up Device /dev/ad1s1h Avail: > 0/76319 MB (0%) > D ftp3 State: up Device /dev/da3s1h Avail: > 0/114473 MB (0%) > > 2 volumes: > V usr State: up Plexes: 1 Size: > 16 GB > V ftp State: up Plexes: 1 Size: > 260 GB > > 2 plexes: > P usr.p0 S State: up Subdisks: 2 Size: > 16 GB > P ftp.p0 C State: up Subdisks: 3 Size: > 260 GB > > 5 subdisks: > S usr.p0.s0 State: up PO: 0 B Size: > 8383 MB > S usr.p0.s1 State: up PO: 256 kB Size: > 8383 MB > S ftp.p0.s0 State: up PO: 0 B Size: > 74 GB > S ftp.p0.s1 State: up PO: 74 GB Size: > 74 GB > S ftp.p0.s2 State: up PO: 149 GB Size: > 111 GB > Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s0 is up > Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s1 is up > Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s2 is up > Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0 is up > Dec 24 16:37:45 blacklamb /kernel: vinum: ftp is up > > After running fsck, I can mount and access the volume. The system > continues to > run fine until the next reboot where the same problems start all over > again. I > also have the same problem with another volume I wish to run called > "backup". > How can I fix my vinum volumes so they survive reboots? > > Thanks for your help and Happy Holidays! > > Drew > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?LOBBIFDAGNMAMLGJJCKNAEBEFAAA.tedm>