From owner-freebsd-current@FreeBSD.ORG Sat Jan 10 20:25:32 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A175D16A4CE; Sat, 10 Jan 2004 20:25:32 -0800 (PST) Received: from smaug.vex.net (smaug.vex.net [66.246.136.211]) by mx1.FreeBSD.org (Postfix) with ESMTP id 28DE343D31; Sat, 10 Jan 2004 20:25:29 -0800 (PST) (envelope-from x@xxvii.net) Received: from bee.vii.net (69-90-55-67.fastdsl.ca [69.90.55.67]) by smaug.vex.net (Postfix) with ESMTP id CE506485E1; Sat, 10 Jan 2004 23:25:29 -0500 (EST) Received: by bee.vii.net (Postfix, from userid 1000) id 2F2EFBD11; Sat, 10 Jan 2004 23:25:18 -0500 (EST) From: Tim Middleton Organization: xxvii.net To: freebsd-current@freebsd.org Date: Sat, 10 Jan 2004 23:25:16 -0500 User-Agent: KMail/1.5.94 References: <200401061608.i06G8Gq14706@catwoman.cs.moravian.edu> <200401100021.02691.x@Vex.Net> <20040110110001.GV7617@wantadilla.lemis.com> In-Reply-To: <20040110110001.GV7617@wantadilla.lemis.com> X-Whee: Yes, Please. MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200401102325.16547.x@Vex.Net> cc: Greg 'groggy' Lehey cc: Stephen Corbesero Subject: Re: is vinum in current working for anyone X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2004 04:25:32 -0000 On Saturday 10 January 2004 06:00, Greg 'groggy' Lehey wrote: > > it's an interesting coincidence. And needless to say... i've removed > > start_vinum="YES" from rc.conf just in case. Greg, further to the email i sent you earlier, where i had resetconfig and re-created the raid5 config, I noticed a bit later the 3rd subdisk had gone away again. So this time I resetconfig, and did a clean init... By merely doing vinum start and stop a number of times you will see below I can lose the 3rd subdisk. Below is a transcript of all commands issued. They are all simply "vinum start", "vinum stop", and "vinum l". There's some time gaps as i was distracted a few times. However, nothing else touched the vinum disk in between times, and the disk was never even mounted. You'll see in the first "vinum l" i was just completing the "init" (97% on the slowest subdisk). You'll see that after the second last vinum start that vinum still has a still knows about /dev/ad2s1d, though raid5.p0.s2 has a status of crashed. Then the last stop/start then loses /dev/ad2s1d completely. This post is a bit long; but decided to reply on the list in case anyone else testing this can duplicate/confirm, or shed more light on this. But first a snippet from my /var/log/messages from the same timespan (starting with the vinum resetconfig) which shows not good things... (malloc errors).... ============================================================== Jan 10 22:17:07 bee kernel: vinum: CONFIGURATION OBLITERATED Jan 10 22:17:16 bee kernel: vinum: exiting with malloc table inconsistency at 0xc6c14800 from vinumio.c:868 Jan 10 22:17:16 bee kernel: vinum: unloaded Jan 10 22:17:18 bee kernel: vinum: loaded Jan 10 22:17:18 bee kernel: vinum: reading configuration from Jan 10 22:17:18 bee kernel: vinum_scandisk: is unallocated Jan 10 22:17:18 bee kernel: vinum: Can't read device , error 5 Jan 10 22:17:18 bee kernel: vinum: couldn't read configuration Jan 10 22:17:33 bee kernel: vinum: drive a is up Jan 10 22:17:33 bee kernel: vinum: drive b is up Jan 10 22:17:33 bee kernel: vinum: drive c is up Jan 10 22:17:51 bee kernel: vinum: raid5.p0.s0 is initializing by force Jan 10 22:17:51 bee kernel: vinum: raid5.p0 is initializing Jan 10 22:17:51 bee kernel: vinum: raid5.p0.s2 is initializing by force Jan 10 22:17:51 bee kernel: vinum: raid5.p0.s1 is initializing by force Jan 10 22:26:56 bee kernel: vinum: raid5.p0.s2 is initialized by force Jan 10 22:26:56 bee kernel: vinum: raid5.p0.s2 is initialized Jan 10 22:28:16 bee kernel: vinum: raid5.p0.s0 is initialized by force Jan 10 22:28:16 bee kernel: vinum: raid5.p0.s0 is initialized Jan 10 22:29:09 bee kernel: vinum: raid5.p0.s1 is initialized by force Jan 10 22:29:09 bee kernel: vinum: raid5.p0.s0 is up Jan 10 22:29:09 bee kernel: vinum: raid5.p0.s1 is up Jan 10 22:29:09 bee kernel: vinum: raid5.p0.s2 is up Jan 10 22:29:09 bee kernel: vinum: raid5.p0 is up Jan 10 22:29:09 bee kernel: vinum: raid5 is up Jan 10 22:29:09 bee kernel: vinum: raid5.p0.s1 is up Jan 10 23:02:32 bee kernel: vinum: exiting with malloc table inconsistency at 0xc6aff800 from vinumio.c:868 Jan 10 23:02:32 bee kernel: vinum: unloaded Jan 10 23:02:33 bee kernel: vinum: loaded Jan 10 23:02:38 bee kernel: vinum: reading configuration from /dev/ad2s1d Jan 10 23:02:38 bee kernel: vinum: updating configuration from /dev/ad1s3d Jan 10 23:02:38 bee kernel: vinum: updating configuration from /dev/ad0s2d Jan 10 23:02:41 bee kernel: vinum: already read config from c Jan 10 23:02:41 bee kernel: vinum: already read config from b Jan 10 23:02:41 bee kernel: vinum: already read config from a Jan 10 23:02:41 bee kernel: vinum: no additional drives found Jan 10 23:02:47 bee kernel: vinum: exiting with malloc table inconsistency at 0xc7e75800 from vinumio.c:868 Jan 10 23:02:47 bee kernel: vinum: unloaded Jan 10 23:02:50 bee kernel: vinum: loaded Jan 10 23:02:51 bee kernel: vinum: reading configuration from /dev/ad0s2d Jan 10 23:02:51 bee kernel: vinum: raid5.p0.s2 is crashed Jan 10 23:02:51 bee kernel: vinum: raid5.p0 is degraded Jan 10 23:02:51 bee kernel: vinum: updating configuration from /dev/ad1s3d Jan 10 23:02:51 bee kernel: vinum: updating configuration from /dev/ad1s3d Jan 10 23:02:51 bee kernel: vinum: incompatible sector sizes. raid5.p0.s2 has 0 bytes, raid5.p0 has 512 bytes. Ignored. Jan 10 23:02:54 bee kernel: vinum: exiting with malloc table inconsistency at 0xc6c12800 from vinumio.c:868 Jan 10 23:02:54 bee kernel: vinum: unloaded Jan 10 23:02:56 bee kernel: vinum: loaded Jan 10 23:02:59 bee kernel: vinum: reading configuration from /dev/ad0s2d Jan 10 23:02:59 bee kernel: vinum: updating configuration from /dev/ad1s3d Jan 10 23:02:59 bee kernel: vinum: updating configuration from /dev/ad2s1d Jan 10 23:03:21 bee kernel: vinum: exiting with malloc table inconsistency at 0xc70c2800 from vinumio.c:868 Jan 10 23:03:21 bee kernel: vinum: unloaded Jan 10 23:03:26 bee kernel: vinum: loaded Jan 10 23:03:26 bee kernel: vinum: reading configuration from /dev/ad0s2d Jan 10 23:03:26 bee kernel: vinum: updating configuration from /dev/ad1s3d Jan 10 23:03:26 bee kernel: vinum: updating configuration from /dev/ad1s3d Jan 10 23:03:26 bee kernel: vinum: incompatible sector sizes. raid5.p0.s2 has 0 bytes, raid5.p0 has 512 bytes. Ignored. ============================================================== 22:28:45 root@bee:~/work # vinum l 3 drives: D a State: up /dev/ad0s2d A: 523/20473 MB (2%) D b State: up /dev/ad1s3d A: 523/20473 MB (2%) D c State: up /dev/ad2s1d A: 523/20473 MB (2%) 1 volumes: V raid5 State: down Plexes: 1 Size: 38 GB 1 plexes: P raid5.p0 R5 State: initializing Subdisks: 3 Size: 38 GB 3 subdisks: S raid5.p0.s0 State: initialized D: a Size: 19 GB S raid5.p0.s1 State: I 97% D: b Size: 19 GB S raid5.p0.s2 State: initialized D: c Size: 19 GB 22:28:49 root@bee:~/work # vinum l 3 drives: D a State: up /dev/ad0s2d A: 523/20473 MB (2%) D b State: up /dev/ad1s3d A: 523/20473 MB (2%) D c State: up /dev/ad2s1d A: 523/20473 MB (2%) 1 volumes: V raid5 State: up Plexes: 1 Size: 38 GB 1 plexes: P raid5.p0 R5 State: up Subdisks: 3 Size: 38 GB 3 subdisks: S raid5.p0.s0 State: up D: a Size: 19 GB S raid5.p0.s1 State: up D: b Size: 19 GB S raid5.p0.s2 State: up D: c Size: 19 GB 23:02:26 root@bee:~/work # vinum stop vinum unloaded 23:02:31 root@bee:~/work # vinum l 0 drives: 0 volumes: 0 plexes: 0 subdisks: 23:02:33 root@bee:~/work # vinum start 23:02:38 root@bee:~/work # vinum l 3 drives: D c State: up /dev/ad2s1d A: 523/20473 MB (2%) D b State: up /dev/ad1s3d A: 523/20473 MB (2%) D a State: up /dev/ad0s2d A: 523/20473 MB (2%) 1 volumes: V raid5 State: up Plexes: 1 Size: 38 GB 1 plexes: P raid5.p0 R5 State: up Subdisks: 3 Size: 38 GB 3 subdisks: S raid5.p0.s0 State: up D: a Size: 19 GB S raid5.p0.s1 State: up D: b Size: 19 GB S raid5.p0.s2 State: up D: c Size: 19 GB 23:02:39 root@bee:~/work # vinum start ** no additional drives found: No such file or directory 23:02:41 root@bee:~/work # vinum stop vinum unloaded 23:02:47 root@bee:~/work # vinum start 23:02:51 root@bee:~/work # vinum stop vinum unloaded 23:02:54 root@bee:~/work # vinum l 0 drives: 0 volumes: 0 plexes: 0 subdisks: 23:02:56 root@bee:~/work # vinum start 23:02:59 root@bee:~/work # vinum l 3 drives: D c State: up /dev/ad2s1d A: 523/20473 MB (2%) D b State: up /dev/ad1s3d A: 523/20473 MB (2%) D a State: up /dev/ad0s2d A: 523/20473 MB (2%) 1 volumes: V raid5 State: up Plexes: 1 Size: 38 GB 1 plexes: P raid5.p0 R5 State: degraded Subdisks: 3 Size: 38 GB 3 subdisks: S raid5.p0.s0 State: up D: a Size: 19 GB S raid5.p0.s1 State: up D: b Size: 19 GB S raid5.p0.s2 State: crashed D: c Size: 19 GB 23:03:01 root@bee:~/work # vinum stop vinum unloaded 23:03:21 root@bee:~/work # vinum start 23:03:26 root@bee:~/work # vinum l 3 drives: D b State: up /dev/ad1s3d A: 523/20473 MB (2%) D a State: up /dev/ad0s2d A: 523/20473 MB (2%) D c State: referenced unknown A: 0/0 MB 1 volumes: V raid5 State: up Plexes: 1 Size: 38 GB 1 plexes: P raid5.p0 R5 State: degraded Subdisks: 3 Size: 38 GB 3 subdisks: S raid5.p0.s0 State: up D: a Size: 19 GB S raid5.p0.s1 State: up D: b Size: 19 GB S raid5.p0.s2 State: crashed D: c Size: 19 GB 23:03:28 root@bee:~/work # -- Tim Middleton | Cain Gang Ltd | Christianity didn't [...] for 20 centuries x@veX.net | www.Vex.Net | [...] shit Hallmark before a live studio...