From owner-freebsd-stable@FreeBSD.ORG Thu Oct 21 22:16:47 2004 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5600616A4CF for ; Thu, 21 Oct 2004 22:16:47 +0000 (GMT) Received: from mta11.adelphia.net (mta11.adelphia.net [68.168.78.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id C8A2243D45 for ; Thu, 21 Oct 2004 22:16:46 +0000 (GMT) (envelope-from security@jim-liesl.org) Received: from smtp.jim-liesl.org ([68.71.52.28]) by mta11.adelphia.net (InterMail vM.6.01.03.02 201-2131-111-104-20040324) with ESMTP id <20041021221645.WVD2188.mta11.adelphia.net@smtp.jim-liesl.org> for ; Thu, 21 Oct 2004 18:16:45 -0400 Received: from [127.0.0.1] (localhost.jim-liesl.org [127.0.0.1]) by smtp.jim-liesl.org (Postfix) with ESMTP id 33C49152B4 for ; Thu, 21 Oct 2004 16:16:42 -0600 (MDT) Message-ID: <41783571.5070204@jim-liesl.org> Date: Thu, 21 Oct 2004 16:17:22 -0600 From: secmgr User-Agent: Mozilla Thunderbird 0.8 (Windows/20040913) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: newfs -v on vinum raid5 panics system (4.10-p3 RELEASE) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Oct 2004 22:16:47 -0000 I've got 7x9.1 gig scsi drives off an adaptec AIC7880 class controller. (da0-6). I want to raid5 these using vinum. I'm a newbie with vinum. If I use a bsd partition the same size as the slice (c=h), and then let the subdisks use the entire partition (len 0), the raid set is corrupted every time after initializing. This usually leads to a kernel panic during newfs. If I leave some amount free, (ie the subdisk only uses 8000mb of a 8675mb drive) no problem. If i'm RTFM'ing correctly, it looks like it should reserve 132kb per drive. I guess I assumed that would be automagically reserved, or if it's supposed to be, it's getting walked on. first I fdisk each drive to create the slice (using the fdisk in sysinstall and letting it autocreate the slice) The data for partition 1 is: sysid 165,(FreeBSD/NetBSD/386BSD) start 63, size 17767827 (8675 Meg), flag 80 (active) beg: cyl 0/ head 1/ sector 1; end: cyl 1023/ head 254/ sector 63 The data for partition 2 is: ... Next, I disklabel each drive (they're all identical) # /dev/da0s1c: # size offset fstype [fsize bsize bps/cpg] c: 17767827 0 unused 0 0 # (Cyl. 0 - 1105*) h: 17767827 0 vinum # (Cyl. 0 - 1105*) and finally, I use the following to create the volume drive d0 device /dev/da0s1h drive d1 device /dev/da1s1h drive d2 device /dev/da2s1h drive d3 device /dev/da3s1h drive d4 device /dev/da4s1h drive d5 device /dev/da5s1h drive d6 device /dev/da6s1h volume bob plex bp plex name bp org raid5 500k sd len 0 drive d0 sd len 0 drive d1 sd len 0 drive d2 sd len 0 drive d3 sd len 0 drive d4 sd len 0 drive d5 sd len 0 drive d6 I init the plex (about 90 minutes go by) and all subdisks show initialized. I type 'vinum start', and then 'vinum list' greybrd# vinum list 14 drives: 1 volumes: V bob State: up Plexes: 1 Size: 50 GB 1 plexes: P bp R5 State: up Subdisks: 7 Size: 50 GB 7 subdisks: S bp.s0 State: up PO: 0 B Size: 8675 MB S bp.s1 State: up PO: 500 kB Size: 8675 MB S bp.s2 State: up PO: 1000 kB Size: 8675 MB S bp.s3 State: up PO: 1500 kB Size: 8675 MB S bp.s4 State: up PO: 2000 kB Size: 8675 MB S bp.s5 State: up PO: 2500 kB Size: 8675 MB S bp.s6 State: up PO: 3000 kB Size: 8675 MB all seems well (except that the drives don't show). Just for grins, I do a 'vinum checkparity bp', and now all the subdisks show as crashed. I assume this is what happened when I tried the newfs also. greybrd# vinum list 14 drives: 1 volumes: V bob State: up Plexes: 1 Size: 50 GB 1 plexes: P bp R5 State: up Subdisks: 7 Size: 50 GB 7 subdisks: S bp.s0 State: crashed PO: 0 B Size: 8675 MB S bp.s1 State: crashed PO: 500 kB Size: 8675 MB S bp.s2 State: crashed PO: 1000 kB Size: 8675 MB S bp.s3 State: crashed PO: 1500 kB Size: 8675 MB S bp.s4 State: crashed PO: 2000 kB Size: 8675 MB S bp.s5 State: crashed PO: 2500 kB Size: 8675 MB S bp.s6 State: crashed PO: 3000 kB Size: 8675 MB If I stop and start vinum, the plex is hosed. greybrd# vinum start *** Warning: configuration updates are disabled. *** Warning: defective objects V bob State: down Plexes: 1 Size: 0 B P bp R5 State: init Subdisks: -28 Size: 0 B *** Warning: configuration updates are disabled. *** (syslog output) Oct 20 04:54:17 greybrd /kernel: vinum: bp.s0 is up Oct 20 04:54:17 greybrd /kernel: vinum: bp.s1 is up (repeated for all subdisks) Oct 20 04:54:17 greybrd /kernel: vinum: bp is up (the plex) Oct 20 04:54:17 greybrd /kernel: vinum: bob is up (the volume) Oct 20 05:15:41 greybrd /kernel: vinum: bp.s0 is crashed by force Oct 20 05:15:41 greybrd /kernel: vinum: bp.s1 is crashed by force (msg repeated for all the subdisks) Oct 20 05:21:29 greybrd /kernel: vinum: unloaded Oct 20 05:21:34 greybrd /kernel: vinum: loaded Oct 20 05:21:34 greybrd /kernel: vinum: reading configuration from /dev/da6s1h Oct 20 05:21:34 greybrd /kernel: vinum: bp.s0: invalid keyword: 17767000s Oct 20 05:21:34 greybrd /kernel: Disabling configuration updates Oct 20 05:21:34 greybrd /kernel: vinum: No space for on d0 Oct 20 05:21:35 greybrd /kernel: vinum: bp.s1: invalid keyword: 17767000s Oct 20 05:21:35 greybrd /kernel: vinum: No space for on d1 (msg repeated for the remaining subdisks) This is what I dumped out of the disk header IN VINOgreybrd.FQDNd0I7vA;^NAvA^Q&;^^volume bob state down plex name bp state init org raid5 1000s vol bob sd name bp.s0 drive d0 plex len 17767000s driveoffset 265s state initialized plexoffset 0s sd name bp.s1 drive d1 plex len 17767000s driveoffset 265s state initialized plexoffset 1000s sd name bp.s2 drive d2 plex len 17767000s driveoffset 265s state initialized plexoffset 2000s sd name bp.s3 drive d3 plex len 17767000s driveoffset 265s state initialized plexoffset 3000s sd name bp.s4 drive d4 plex len 17767000s driveoffset 265s state initialized plexoffset 4000s sd name bp.s5 drive d5 plex len 17767000s driveoffset 265s state initialized plexoffset 5000s sd name bp.s6 drive d6 plex len 17767000s driveoffset 265s state initialized plexoffset 6000s