From owner-freebsd-questions@FreeBSD.ORG Wed Jun 22 12:23:52 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E07816A41C for ; Wed, 22 Jun 2005 12:23:52 +0000 (GMT) (envelope-from jkerian@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.207]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2558843D49 for ; Wed, 22 Jun 2005 12:23:52 +0000 (GMT) (envelope-from jkerian@gmail.com) Received: by wproxy.gmail.com with SMTP id 69so248935wri for ; Wed, 22 Jun 2005 05:23:51 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=ZPi2P/pgBmEfrKzbutYu2Hvb8h8BAKste526mQjJJXaZeVn8rNHuy2n3rMCaHwUxZojkE7qdg+lJ2QRZENfSUkqXs3KfqL7yma5FqfHgqGzL+Cc0N7vD65mOaJ+tgVmTvpoOPp1v1kKjIBc6olK0fbFxkKYt8A9+3bRJww1r+v0= Received: by 10.54.57.56 with SMTP id f56mr404322wra; Wed, 22 Jun 2005 05:23:51 -0700 (PDT) Received: by 10.54.57.63 with HTTP; Wed, 22 Jun 2005 05:23:51 -0700 (PDT) Message-ID: <282b2dd9050622052314b8d6de@mail.gmail.com> Date: Wed, 22 Jun 2005 07:23:51 -0500 From: Joseph Kerian To: freebsd-questions@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: vinum question: ide chain crashed, one or two drives dead? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Joseph Kerian List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2005 12:23:52 -0000 I was away for a long weekend when I recieved an annoyed phone call regarding a site that I manage. While it was not terribly difficult to bring the website back online, I lost either a controller card or an IDE cable, and this produced a string of errors that eventually led to a kernel panic. I would very much like to be able to recover some of this data, and am curious if I am able to. Running FreeBSD nene 5.3-RELEASE-p5 with a custom kernel to include IPFW options. The following was my "var/log/messages" for the relevent times: Jun 20 04:30:44 nene kernel: ad2: WARNING - removed from configuration Jun 20 04:30:44 nene kernel: ad3: WARNING - removed from configuration Jun 20 04:30:44 nene kernel: ata1-master: FAILURE - unknown CMD (0xb0) time= d out Jun 20 04:30:44 nene kernel: vinum: ideraid.p0.s0 is crashed by force Jun 20 04:30:44 nene kernel: vinum: ideraid.p0 is degraded Jun 20 04:30:44 nene kernel: fatal:ideraid.p0.s0 read error, block 231758025 for 16384 bytes Jun 20 04:30:44 nene kernel: ideraid.p0.s0: user buffer block 1158788032 for 16384 bytes Jun 20 04:30:44 nene kernel: dua: fatal drive I/O error, block 231758025 for 16384 bytes Jun 20 04:30:44 nene kernel: vinum: drive dua is down Jun 20 04:30:44 nene kernel: vinum: ideraid.p0.s0 is stale by force Jun 20 04:30:44 nene kernel: fatal :ideraid.p0.s0 write error, block 231758025 for 16384 bytes Jun 20 04:30:44 nene kernel: ideraid.p0.s0: user buffer block 1158788032 for 16384 bytes Jun 20 04:30:44 nene kernel: dua: fatal drive I/O error, block 231758025 for 16384 bytes Jun 20 04:30:44 nene smartd[604]: Device: /dev/ad2, failed to read SMART Attribute Data Jun 20 04:30:44 nene kernel: vinum: Can't write config to /dev/ad3a, error = 6 Jun 20 04:30:44 nene kernel: vinum: drive eva34 is down Jun 20 04:30:44 nene kernel: vinum: ideraid.p0.s1 is crashed Jun 20 04:30:44 nene kernel: vinum: ideraid.p0 is corrupt Jun 21 03:01:08 nene kernel: fatal:ideraid.p0.s1 read error, block 301684457 for 16384 bytes Jun 21 03:01:08 nene kernel: ideraid.p0.s1: user buffer block 1508419040 for 16384 bytes Jun 21 03:01:08 nene kernel: eva34: fatal drive I/O error, block 301684457 for 16384 bytes Jun 21 03:01:08 nene kernel: fatal:ideraid.p0.s1 read error, block 301684457 for 16384 bytes Jun 21 03:01:08 nene kernel: ideraid.p0.s1: user buffer block 1508419040 for 16384 bytes Jun 21 03:01:08 nene kernel: eva34: fatal drive I/O error, block 301684457 for 16384 bytes Jun 21 03:01:09 nene kernel: 4 bytes Jun 21 03:01:09 nene kernel: fatal:ideraid.p0.s1 read error, block 301684457 for 16384 bytes Jun 21 03:01:09 nene kernel: ideraid.p0.s1: user buffer block 1508419040 for 16384 bytes Jun 21 03:01:09 nene kernel: eva34: fatal drive I/O error, block 301684457 for 16384 bytes (approximately 3000 repeats of those 3 lines later) Jun 21 03:01:37 nene kernel: vinum: ideraid.p0.s1 is stale by force Jun 21 03:01:37 nene kernel: vinum: ideraid.p0.s0 is crashed by force Jun 21 03:01:37 nene kernel: fatal:ideraid.p0.s0 read error, block 177262153 for 16384 bytes Jun 21 03:01:37 nene kernel: ideraid.p0.s0: user buffer block 886309184 for 16384 bytes Jun 21 03:01:37 nene kernel: dua: fatal drive I/O error, block 177262153 for 16384 bytes Jun 21 03:01:37 nene kernel: fatal:ideraid.p0.s0 read error, block 177563337 for 16384 bytes Jun 21 03:01:37 nene kernel: ideraid.p0.s0: user buffer block 887814592 for 16384 bytes Jun 21 03:01:37 nene kernel: dua: fatal drive I/O error, block 177563337 for 16384 bytes Jun 21 03:01:37 nene kernel: fatal:ideraid.p0.s0 read error, block 178466889 for 16384 bytes Jun 21 03:01:37 nene kernel: ideraid.p0.s0: user buffer block 892330816 for 16384 bytes Jun 21 03:01:37 nene kernel: dua: fatal drive I/O error, block 178466889 for 16384 bytes Jun 21 03:01:37 nene kernel: vinum: ideraid.p0.s0 is stale by force vinum list: (after using "start") 6 drives: D gva250 State: up /dev/ad7a A: 0/238475 MB (0%) D eva200 State: up /dev/ad6a A: 0/190782 MB (0%) D gva200 State: up /dev/ad5a A: 0/190782 MB (0%) D eva250 State: up /dev/ad4a A: 0/238475 MB (0%) D eva34 State: up /dev/ad3a A: 0/190782 MB (0%) D dua State: up /dev/ad2a A: 0/190782 MB (0%) 2 volumes: V ideraid State: up Plexes: 1 Size: 931 GB V mirror2 State: up Plexes: 2 Size: 46 GB 3 plexes: P ideraid.p0 R5 State: corrupt Subdisks: 6 Size: 931 G= B P mirror2.p0 C State: up Subdisks: 1 Size: 46 GB P mirror2.p1 C State: up Subdisks: 1 Size: 46 GB 8 subdisks: S ideraid.p0.s0 State: stale D: dua Size: 186 GB S ideraid.p0.s1 State: crashed D: eva34 Size: 186 GB S ideraid.p0.s2 State: up D: eva250 Size: 186 GB S ideraid.p0.s3 State: up D: gva200 Size: 186 GB S ideraid.p0.s4 State: up D: eva200 Size: 186 GB S ideraid.p0.s5 State: up D: gva250 Size: 186 GB S mirror2.p0.s0 State: up D: gva250 Size: 46 GB S mirror2.p1.s0 State: up D: eva250 Size: 46 GB vinum_history contains nothing of interest, a long history of "l" (list), start, stop, and a few helps. I'm not entirely sure why it doesn't contain the commands of initially creating the array. But the array appears in that vinum list as it was configured, although it would be nice if the ideraid array functioned. Any help in this would be greatly appreciated. --Joe Kerian