Date: Fri, 3 Sep 2004 14:11:30 +0200 From: =?iso-8859-2?Q?Krzysztof_Ciep=B3ucha?= <kris@home.pl> To: <freebsd-geom@freebsd.org> Subject: problems with GEOM_MIRROR Message-ID: <00d701c491af$253ebf30$fe78a8c0@kris>
next in thread | raw e-mail | index | archive | help
Hi, I am testing mirroring using GEOM_MIRROR and have found several a little bit strange behaviours. But first my configuration (in short): The server is using adaptec scsi controller with two scsi disks: ibm235# camcontrol devlist <IBM-ESXS ST336605LC !# B245> at scbus1 target 0 lun 0 (pass1,da0) <IBM-ESXS ST336605LC !# B245> at scbus1 target 1 lun 0 (pass2,da1) The kernel is compiled from fresh cvsup-ed 5.3BETA2 sources with statically added GEOM_LABEL and GEOM_MIRROR. On top of this disks I have put a labels using GEOM_LABEL: label/disk0 on da0 and label/disk1 on da1. Then I have made a mirror (GEOM_MIRROR) using those labels, with hardcoded providers: mirror/mirror0 using label/disk0 and label/disk1. On top of the mirror I have created a standard bsdlabel (with boot loader), set up filesystems and moved the whole system from the third disk using dump and restore. ibm235# bsdlabel /dev/mirror/mirror0 # /dev/mirror/mirror0: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 524288 0 4.2BSD 2048 16384 32776 b: 5191376 524288 swap c: 71096638 0 unused 0 0 # "raw" part, don't edit d: 524288 5715664 4.2BSD 2048 16384 32776 e: 524288 6239952 4.2BSD 2048 16384 32776 f: 64332398 6764240 4.2BSD 2048 16384 28552 So finally I have running system booting and running from mirror/mirror1: ibm235# cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/mirror/mirror0b none swap sw 0 0 /dev/mirror/mirror0a / ufs rw 1 1 /dev/mirror/mirror0e /tmp ufs rw 2 2 /dev/mirror/mirror0f /usr ufs rw 2 2 /dev/mirror/mirror0d /var ufs rw 2 2 /dev/acd0 /cdrom cd9660 ro,noauto 0 0 The mirror is fully synchronized (at least I think it is): ibm235# geom mirror list Geom name: mirror0 Components: 2 Balance: split Slice: 4096 Flags: NONE SyncID: 3 ID: 993051015 Providers: 1. Name: mirror/mirror0 Mediasize: 36401478656 (34G) Sectorsize: 512 Mode: r5w5e1 Consumers: 1. Name: label/disk0 Mediasize: 36401479168 (34G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY, HARDCODED SyncID: 3 ID: 2288148496 2. Name: label/disk1 Mediasize: 36401479168 (34G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY, HARDCODED SyncID: 3 ID: 3740254788 Geom name: mirror0.sync Finally the list of problems: 1. After server restart the mirror is being rebuilding form the beginning, even if before restart it was fully synchronized. Here is a fragment from dmesg: <skipped> Sep 3 07:30:47 ibm235 kernel: SMP: AP CPU #1 Launched! Sep 3 07:30:47 ibm235 kernel: GEOM_LABEL: Label for provider da0 is label/disk1. Sep 3 07:30:47 ibm235 kernel: GEOM_LABEL: Label for provider da1 is label/disk0. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0 created (id=993051015). Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 detected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk0 detected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk0 activated. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider mirror/mirror0 launched. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1. Sep 3 07:30:47 ibm235 kernel: Mounting root from ufs:/dev/mirror/mirror0a Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 disconnected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1 stopped. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 detected. Sep 3 07:30:47 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1. Sep 3 08:17:42 ibm235 kernel: GEOM_MIRROR: Device mirror0: rebuilding provider label/disk1 finished. Sep 3 08:17:42 ibm235 kernel: GEOM_MIRROR: Device mirror0: provider label/disk1 activated. 2. sysctl -b kern.geom.conftxt shows BSD labels like label/disk0a ... disk0f which shouldn't exist, should they? ibm235# sysctl -b kern.geom.conftxt 0 DISK da1 36401479680 512 hd 255 sc 63 1 LABEL label/disk1 36401479168 512 i 0 o 0 2 MIRROR mirror/mirror0 36401478656 512 3 BSD mirror/mirror0f 32938187776 512 i 5 o 3463290880 ty 7 3 BSD mirror/mirror0e 268435456 512 i 4 o 3194855424 ty 7 3 BSD mirror/mirror0d 268435456 512 i 3 o 2926419968 ty 7 3 BSD mirror/mirror0c 36401478656 512 i 2 o 0 ty 0 3 BSD mirror/mirror0b 2657984512 512 i 1 o 268435456 ty 1 3 BSD mirror/mirror0a 268435456 512 i 0 o 0 ty 7 0 DISK da0 36401479680 512 hd 255 sc 63 1 LABEL label/disk0 36401479168 512 i 0 o 0 2 BSD label/disk0f 32938187776 512 i 5 o 3463290880 ty 7 2 BSD label/disk0e 268435456 512 i 4 o 3194855424 ty 7 2 BSD label/disk0d 268435456 512 i 3 o 2926419968 ty 7 2 BSD label/disk0c 36401478656 512 i 2 o 0 ty 0 2 BSD label/disk0b 2657984512 512 i 1 o 268435456 ty 1 2 BSD label/disk0a 268435456 512 i 0 o 0 ty 7 2 MIRROR mirror/mirror0 36401478656 512 3 BSD mirror/mirror0f 32938187776 512 i 5 o 3463290880 ty 7 3 BSD mirror/mirror0e 268435456 512 i 4 o 3194855424 ty 7 3 BSD mirror/mirror0d 268435456 512 i 3 o 2926419968 ty 7 3 BSD mirror/mirror0c 36401478656 512 i 2 o 0 ty 0 3 BSD mirror/mirror0b 2657984512 512 i 1 o 268435456 ty 1 3 BSD mirror/mirror0a 268435456 512 i 0 o 0 ty 7 Does anybody have working configuration like this? Is there a bug in GEOM implementation, or it is a problem with my configuration. -- kris-at-home-dot-pl
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?00d701c491af$253ebf30$fe78a8c0>