From owner-freebsd-geom@FreeBSD.ORG Fri Sep 3 12:11:34 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 085B716A4CE for ; Fri, 3 Sep 2004 12:11:34 +0000 (GMT) Received: from v05184.home.net.pl (v05184.home.net.pl [212.85.117.104]) by mx1.FreeBSD.org (Postfix) with SMTP id DEC0E43D1F for ; Fri, 3 Sep 2004 12:11:32 +0000 (GMT) (envelope-from kris@home.pl) Received: from bm.unizeto.pl (HELO kris) (kris@home@213.222.192.3) by matrix11.home.net.pl with SMTP; 3 Sep 2004 12:11:30 -0000 Message-ID: <00d701c491af$253ebf30$fe78a8c0@kris> From: =?iso-8859-2?Q?Krzysztof_Ciep=B3ucha?= To: Date: Fri, 3 Sep 2004 14:11:30 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 Subject: problems with GEOM_MIRROR X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Sep 2004 12:11:34 -0000 Hi, I am testing mirroring using GEOM_MIRROR and have found several a little bit strange behaviours. But first my configuration (in short): The server is using adaptec scsi controller with two scsi disks: ibm235# camcontrol devlist at scbus1 target 0 lun 0 (pass1,da0) at scbus1 target 1 lun 0 (pass2,da1) The kernel is compiled from fresh cvsup-ed 5.3BETA2 sources with statically added GEOM_LABEL and GEOM_MIRROR. On top of this disks I have put a labels using GEOM_LABEL: label/disk0 on da0 and label/disk1 on da1. Then I have made a mirror (GEOM_MIRROR) using those labels, with hardcoded providers: mirror/mirror0 using label/disk0 and label/disk1. On top of the mirror I have created a standard bsdlabel (with boot loader), set up filesystems and moved the whole system from the third disk using dump and restore. ibm235# bsdlabel /dev/mirror/mirror0 # /dev/mirror/mirror0: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 524288 0 4.2BSD 2048 16384 32776 b: 5191376 524288 swap c: 71096638 0 unused 0 0 # "raw" part, don't edit d: 524288 5715664 4.2BSD 2048 16384 32776 e: 524288 6239952 4.2BSD 2048 16384 32776 f: 64332398 6764240 4.2BSD 2048 16384 28552 So finally I have running system booting and running from mirror/mirror1: ibm235# cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/mirror/mirror0b none swap sw 0 0 /dev/mirror/mirror0a / ufs rw 1 1 /dev/mirror/mirror0e /tmp ufs rw 2 2 /dev/mirror/mirror0f /usr ufs rw 2 2 /dev/mirror/mirror0d /var ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 The mirror is fully synchronized (at least I think it is): ibm235# geom mirror list Geom name: mirror0 Components: 2 Balance: split Slice: 4096 Flags: NONE SyncID: 3 ID: 993051015 Providers: 1. Name: mirror/mirror0 Mediasize: 36401478656 (34G) Sectorsize: 512 Mode: r5w5e1 Consumers: 1. Name: label/disk0 Mediasize: 36401479168 (34G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY, HARDCODED SyncID: 3 ID: 2288148496 2. Name: label/disk1 Mediasize: 36401479168 (34G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY, HARDCODED SyncID: 3 ID: 3740254788 Geom name: mirror0.sync Finally the list of problems: 1. After server restart the mirror is being rebuilding form the beginning, even if before restart it was fully synchronized. Here is a fragment from dmesg: Sep 3 07:30:47 ibm235 kernel: SMP: AP CPU #1 Launched! Sep 3 07:30:47 ibm235 kernel: GEOM_LABEL: Label for provider da0 is label/disk1. Sep 3 07:30:47 ibm235 kernel: GEOM_LABEL: Label for provider da1 is label/disk0. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0 created (id=993051015). Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 detected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk0 detected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk0 activated. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider mirror/mirror0 launched. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1. Sep 3 07:30:47 ibm235 kernel: Mounting root from ufs:/dev/mirror/mirror0a Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 disconnected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1 stopped. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 detected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1. Sep 3 08:17:42 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1 finished. Sep 3 08:17:42 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 activated. 2. sysctl -b kern.geom.conftxt shows BSD labels like label/disk0a ... disk0f which shouldn't exist, should they? ibm235# sysctl -b kern.geom.conftxt 0 DISK da1 36401479680 512 hd 255 sc 63 1 LABEL label/disk1 36401479168 512 i 0 o 0 2 MIRROR mirror/mirror0 36401478656 512 3 BSD mirror/mirror0f 32938187776 512 i 5 o 3463290880 ty 7 3 BSD mirror/mirror0e 268435456 512 i 4 o 3194855424 ty 7 3 BSD mirror/mirror0d 268435456 512 i 3 o 2926419968 ty 7 3 BSD mirror/mirror0c 36401478656 512 i 2 o 0 ty 0 3 BSD mirror/mirror0b 2657984512 512 i 1 o 268435456 ty 1 3 BSD mirror/mirror0a 268435456 512 i 0 o 0 ty 7 0 DISK da0 36401479680 512 hd 255 sc 63 1 LABEL label/disk0 36401479168 512 i 0 o 0 2 BSD label/disk0f 32938187776 512 i 5 o 3463290880 ty 7 2 BSD label/disk0e 268435456 512 i 4 o 3194855424 ty 7 2 BSD label/disk0d 268435456 512 i 3 o 2926419968 ty 7 2 BSD label/disk0c 36401478656 512 i 2 o 0 ty 0 2 BSD label/disk0b 2657984512 512 i 1 o 268435456 ty 1 2 BSD label/disk0a 268435456 512 i 0 o 0 ty 7 2 MIRROR mirror/mirror0 36401478656 512 3 BSD mirror/mirror0f 32938187776 512 i 5 o 3463290880 ty 7 3 BSD mirror/mirror0e 268435456 512 i 4 o 3194855424 ty 7 3 BSD mirror/mirror0d 268435456 512 i 3 o 2926419968 ty 7 3 BSD mirror/mirror0c 36401478656 512 i 2 o 0 ty 0 3 BSD mirror/mirror0b 2657984512 512 i 1 o 268435456 ty 1 3 BSD mirror/mirror0a 268435456 512 i 0 o 0 ty 7 Does anybody have working configuration like this? Is there a bug in GEOM implementation, or it is a problem with my configuration. -- kris-at-home-dot-pl