From owner-freebsd-questions@FreeBSD.ORG Sat May 8 10:37:49 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF9F216A4CF for ; Sat, 8 May 2004 10:37:49 -0700 (PDT) Received: from spock.dilkie.com (spock.dilkie.com [206.51.1.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4036043D2D for ; Sat, 8 May 2004 10:37:46 -0700 (PDT) (envelope-from lee@dilkie.com) Received: from borg (borg.dilkie.com [206.51.1.193]) (authenticated bits=0) by spock.dilkie.com (8.12.11/8.12.11) with ESMTP id i48HbhcQ002043 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO) for ; Sat, 8 May 2004 13:37:44 -0400 (EDT) (envelope-from lee@dilkie.com) From: "Lee Dilkie" To: Date: Sat, 8 May 2004 13:37:42 -0400 Message-ID: <011001c43523$2abbba50$c10133ce@dilkie.com> X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.6604 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 Importance: Normal X-Scanned-By: MIMEDefang 2.42 Subject: vinum striped volume has corrupt plex, help needed X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2004 17:37:50 -0000 Hi there, I've been running a 5 disk vinum array (sripted, no redundancy) for a few months now. It's composed of 5 scsi drives of 4G each. I bought a new 120G ide drive, with the intention of copying over all the files from the vinum array and retiring the array (the scsi drives are really loud). All was fine until the file copy part. Shortly after starting, i started to get scsi errors and the scsi system reset the drives and re-spun them up in an attempt to provide data (this i could hear). Eventually vinum reported a read error. My machine kinda locked up because there were swap partitions on the scsi drives and things just went south when the OS couldn't swap properly. I rebooted and fsck'd my other partitions just fine but vinum reported that the plex was corrupt and one of the subdisks was stale (see "vinum list" output below). I also include the output from the command to read and parse the vium table on each drive ( as describe at http://www.vinumvm.org/vinum/how-to-debug.html ). it sure looks to me like all the disks have the same vinum info. When i tried to vinum start striped.p0.s1, most of the time i would get an error "Input/output error (5)" but a couple of times the command hung (as it is right now). Also, I reconfured my scsi (2940uw) to the lowest transfer speed, disabled wide negotation (these are wide fast drives), disabled disconnect and disabled synchronous transfers. Basicly, i slowed them down as slow as they can go. I am able to successfully read each drive (tested the first 1G of each using "dd if=/dev/da*s1e of=/dev/null bs=1m count=1000"). There were no write operations to the vinum volume when things crashed. I'm hoping i can get vinum up and running again so i can copy off this data. Question to the group. Would a vinum create using the original configuration (i have the file) recover this situation so i could mount and read the disk? Is there something else to do that will help? TIA, -lee info follows: # uname -a FreeBSD spock.dilkie.com 4.10-PRERELEASE FreeBSD 4.10-PRERELEASE #1: Sun May 2 15:11:04 EDT 2004 root@spock.dilkie.com:/usr/obj/usr/src/sys/SPOCK i386 # vinum list 5 drives: D a State: up Device /dev/da0s1e Avail: 1/4000 MB (0%) D b State: up Device /dev/da1s1e Avail: 1/4000 MB (0%) D c State: up Device /dev/da2s1e Avail: 1/4000 MB (0%) D d State: up Device /dev/da3s1e Avail: 1/4000 MB (0%) D e State: up Device /dev/da4s1e Avail: 1/4000 MB (0%) 1 volumes: V striped State: up Plexes: 1 Size: 19 GB 1 plexes: P striped.p0 S State: corrupt Subdisks: 5 Size: 19 GB 5 subdisks: S striped.p0.s0 State: up PO: 0 B Size: 3998 MB S striped.p0.s1 State: R 0% PO: 373 kB Size: 3998 MB *** Revive process for striped.p0.s1 has died *** (NOTE, this originally reported "stale") S striped.p0.s2 State: up PO: 746 kB Size: 3998 MB S striped.p0.s3 State: up PO: 1119 kB Size: 3998 MB S striped.p0.s4 State: up PO: 1492 kB Size: 3998 MB # results of reading the first bit of each drive... # for i in /dev/da0s1e /dev/da1s1e /dev/da2s1e /dev/da3s1e /dev/da4s1e; do (dd if=$i skip=8 count=6|tr -d '\000-\011\200-\377'; echo) >> log done IN VINOspock.dilkie.comcLg@@0WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.comdJg@p @} WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.comeBMg@A@ WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.comaLg@5@ZWEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.combLg@\@ WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.comcLg@@0WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.comdJg@p @} WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s IN VINOspock.dilkie.comeBMg@A@ WEVBJ volume striped state up plex name striped.p0 state corrupt org striped 746s vol striped sd name striped.p0.s0 drive a len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 0s sd name striped.p0.s1 drive b len 8189588s driveoffset 265s state stale plex striped.p0 plexoffset 746s sd name striped.p0.s2 drive c len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 1492s sd name striped.p0.s3 drive d len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2238s sd name striped.p0.s4 drive e len 8189588s driveoffset 265s state up plex striped.p0 plexoffset 2984s