From owner-freebsd-geom@FreeBSD.ORG Tue Sep 21 04:32:00 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 78AB116A4CE for ; Tue, 21 Sep 2004 04:32:00 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 05B1B43D1D for ; Tue, 21 Sep 2004 04:32:00 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8L4VvRK054869 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 21 Sep 2004 00:31:59 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8L4VpiY009755 for ; Tue, 21 Sep 2004 00:31:52 -0400 (EDT) From: Paul Mather To: freebsd-geom@freebsd.org Content-Type: text/plain Message-Id: <1095741109.9448.84.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Tue, 21 Sep 2004 00:31:51 -0400 Content-Transfer-Encoding: 7bit Subject: Fresh install onto geom_stripe or geom_mirror? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2004 04:32:00 -0000 Does anyone have any advice as to the best way to do a fresh install onto an all-mirrored or all-striped geom setup? I'm about to obtain a Dell Optiplex (GX280?) on which I plan to run some simulation experiments. The system currently has Windows XP installed (so I believe, anyway), and I plan on wiping that and installing RELENG_5 (or perhaps HEAD). The PC has two 80 GB drives fitted that I'd like either to mirror or stripe together as one "drive" and then install onto that. What is the best way to do that? I run FreeBSD-CURRENT at home with a root-on-gvinum mirrored setup. Currently, geom_vinum lacks good read load-balancing, which would be advantageous to have when running my simulations. I will be reading and writing (mostly reading) a multi-gigabyte Berkeley DB during the simulation runs, so having both drives contribute to reads instead of just one is pretty much a necessity. (According to a recent freebsd-current digest, the load balancing issue may now have been addressed in geom_vinum; I'm building a new kernel to see if this is so. But, even if true, I don't really need the complexity of the gvinum LVM and would like to try something more more basic like geom_mirror or geom_stripe.) Usually, when I install a system I do a minimal install; cvsup recent sources; and then build a customised kernel and world. With "gmirror" it is possible to create a mirror with only one initial provider. So, I could perform a minimal install onto, say, ad0s1 and then create and populate a mirror on ad2s1. After booting the mirror, ad0s1 could be added to the mirror to get a true two-drive mirror. (Is it better to use adXsY or just adX when creating mirrors? All the examples use the raw devices, not slices.) However, gstripe requires two initial providers according to the man page, ruling out the above technique. :-\ Is there some alternative way of creating a stripe covering two drives entirely on an already-installed two-drive system? Perhaps an alternative approach would be to have one slice containing a conventional install of /, swap, /var, and /usr (perhaps mirrored via geom_mirror), and then a second slice for, say, /data, that will be made into a geom_stripe for the simulation data files after install. Hints and suggestions welcomed. :-) Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Tue Sep 21 07:08:40 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 885EC16A4CE for ; Tue, 21 Sep 2004 07:08:40 +0000 (GMT) Received: from fmmailgate03.web.de (fmmailgate03.web.de [217.72.192.234]) by mx1.FreeBSD.org (Postfix) with ESMTP id E46C343D46 for ; Tue, 21 Sep 2004 07:08:39 +0000 (GMT) (envelope-from ilsa.gold@web.de) Received: by fmmailgate03.web.de (8.12.6/8.12.6/webde Linux 0.7) with SMTP id i8L77e04014015 for freebsd-geom@freebsd.org; Tue, 21 Sep 2004 09:08:38 +0200 Received: from 62.180.31.25 by freemailng0106.web.de with HTTP; Tue, 21 Sep 2004 09:08:37 +0200 Date: Tue, 21 Sep 2004 09:08:37 +0200 Message-Id: <1293070566@web.de> MIME-Version: 1.0 From: =?iso-8859-1?Q? "Stefan=20Krau=DF" ?= To: freebsd-geom@freebsd.org Precedence: fm-user Organization: http://freemail.web.de/ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Subject: Another problem with not re-attachable gbde slice X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2004 07:08:40 -0000 Hello list, I wrote a few weeks ago about a problem with reattaching a gbde encrypted slice under FreeBSD 5.2.1. The slice worked for about 3 month without any problem (could be attached, detached and so on). After writing lots of data to this attached slice (about 9-10 Gig) the slice couldn't be attached any more. There appeared no /dev/ad0s3a.bde-device file after an "gbde attach /dev/ad0s3a -l /etc/gbde/ad0s3a". I upgraded to RELENG_5 last sunday hoping that this maybe solve my problem, but it didn't. So I tried the follwing command to change my key (just to check if the access to my key is borken) and got an error: # gbde setkey /dev/ad0s3a -l /etc/gbde/ad0s3a Enter passphrase: gbde: Lock not found Did I something wrong or is the gbde-slice finally unusable (I read a few maillings in this list that there were some bugs regarding the reattaching of gbde-slices)? Best regards Stefan _______________________________________________________ WEB.DE Video-Mail - Sagen Sie mehr mit bewegten Bildern Informationen unter: http://freemail.web.de/?mc=021199 From owner-freebsd-geom@FreeBSD.ORG Tue Sep 21 08:31:08 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7CB4516A4CE for ; Tue, 21 Sep 2004 08:31:08 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id B2C5543D5A for ; Tue, 21 Sep 2004 08:31:07 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id i8L8V6aN067694; Tue, 21 Sep 2004 10:31:06 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: "=?iso-8859-1?Q? " Stefan=20Krau=DF " ?=" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 21 Sep 2004 09:08:37 +0200." <1293070566@web.de> Date: Tue, 21 Sep 2004 10:31:06 +0200 Message-ID: <67693.1095755466@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: freebsd-geom@freebsd.org Subject: Re: Another problem with not re-attachable gbde slice X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2004 08:31:08 -0000 In message <1293070566@web.de>, =?iso-8859-1?Q? "Stefan=20Krau=DF" ?= writes: >Did I something wrong or is the gbde-slice finally unusable (I >read a few maillings in this list that there were some bugs regarding >the reattaching of gbde-slices)? A bug has been fixed with respect to GBDE devices which have less than the full set of four keys konfigured. In other words, if you specified a "max_keys" less than four when you ran "gbde init" you may have hit this bug. It is fixed in 5.3-BETA5 and -current. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Tue Sep 21 12:45:43 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 188CF16A4CE for ; Tue, 21 Sep 2004 12:45:43 +0000 (GMT) Received: from afields.ca (afields.ca [216.194.67.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id CF31343D31 for ; Tue, 21 Sep 2004 12:45:42 +0000 (GMT) (envelope-from afields@afields.ca) Received: from afields.ca (localhost.afields.ca [127.0.0.1]) by afields.ca (8.12.11/8.12.11) with ESMTP id i8LCjgDO064781; Tue, 21 Sep 2004 08:45:42 -0400 (EDT) (envelope-from afields@afields.ca) Received: (from afields@localhost) by afields.ca (8.12.11/8.12.11/Submit) id i8LCjg9V064780; Tue, 21 Sep 2004 08:45:42 -0400 (EDT) (envelope-from afields) Date: Tue, 21 Sep 2004 08:45:42 -0400 From: Allan Fields To: =?us-ascii?B?PT9pc28tODg1OS0xP1E/?= Stefan=20Krau=DF ?= Message-ID: <20040921124542.GE47410@afields.ca> References: <1293070566@web.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1293070566@web.de> User-Agent: Mutt/1.4i cc: freebsd-geom@freebsd.org Subject: Re: Another problem with not re-attachable gbde slice X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2004 12:45:43 -0000 On Tue, Sep 21, 2004 at 09:08:37AM +0200, =?iso-8859-1?Q? Stefan=20Krau=DF ?= wrote: > Hello list, > > I wrote a few weeks ago about a problem with reattaching a gbde > encrypted slice under FreeBSD 5.2.1. The slice worked for about 3 > month without any problem (could be attached, detached and so on). > After writing lots of data to this attached slice (about 9-10 Gig) > the slice couldn't be attached any more. There appeared no > /dev/ad0s3a.bde-device file after an "gbde attach /dev/ad0s3a -l > /etc/gbde/ad0s3a". I observed similar and was able to reproduce this behaviour as you/others have encountered with a test volume: usually it took a while to fill up the filesystem enough. > I upgraded to RELENG_5 last sunday hoping that this maybe solve > my problem, but it didn't. So I tried the follwing command to change > my key (just to check if the access to my key is borken) and got > an error: > > # gbde setkey /dev/ad0s3a -l /etc/gbde/ad0s3a > Enter passphrase: > gbde: Lock not found That is because it would attempt to find a now non-existent and likely moved-in-location lock sector. > Did I something wrong or is the gbde-slice finally unusable (I > read a few maillings in this list that there were some bugs regarding > the reattaching of gbde-slices)? Afraid not. I take it your existing device couldn't be reattached with the newest patch applied: This isn't unexpected given the most likely cause of your problem was that the lock sector had been overwritten, if you only had one key, then that option has been exhausted. > Best regards > Stefan Thanks for your and others' reports.. -- Allan Fields, AFRSL - http://afields.ca 2D4F 6806 D307 0889 6125 C31D F745 0D72 39B4 5541 From owner-freebsd-geom@FreeBSD.ORG Tue Sep 21 16:42:27 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6234C16A4CE for ; Tue, 21 Sep 2004 16:42:27 +0000 (GMT) Received: from kestrel.alerce.com (kestrel.alerce.com [209.182.219.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1CBFC43D58 for ; Tue, 21 Sep 2004 16:42:27 +0000 (GMT) (envelope-from hartzell@kestrel.alerce.com) Received: from satchel.alerce.com (0-d-60-f8-9f-4a.dhcp.lbl.gov [131.243.195.161]) (authenticated bits=128) by kestrel.alerce.com (8.12.10/8.12.10) with ESMTP id i8LGgPUQ058912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 21 Sep 2004 09:42:26 -0700 (PDT) (envelope-from hartzell@kestrel.alerce.com) Received: from satchel.alerce.com (localhost [127.0.0.1]) by satchel.alerce.com (8.13.1/8.13.1) with ESMTP id i8LGgplt003683 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 21 Sep 2004 09:42:51 -0700 (PDT) (envelope-from hartzell@satchel.alerce.com) Received: (from hartzell@localhost) by satchel.alerce.com (8.13.1/8.13.1/Submit) id i8LGgnrR003679; Tue, 21 Sep 2004 09:42:49 -0700 (PDT) (envelope-from hartzell) From: George Hartzell MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16720.23049.587056.935674@satchel.alerce.com> Date: Tue, 21 Sep 2004 09:42:49 -0700 To: Paul Mather In-Reply-To: <1095741109.9448.84.camel@zappa.Chelsea-Ct.Org> References: <1095741109.9448.84.camel@zappa.Chelsea-Ct.Org> X-Mailer: VM 7.17 under 21.4 (patch 15) "Security Through Obscurity" XEmacs Lucid X-Virus-Scanned: ClamAV version 'clamd / ClamAV version 0.65', clamav-milter version '0.60p' cc: freebsd-geom@freebsd.org Subject: Re: Fresh install onto geom_stripe or geom_mirror? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: hartzell@kestrel.alerce.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2004 16:42:27 -0000 Paul Mather writes: > [...] The PC has two 80 GB drives fitted that I'd > like either to mirror or stripe together as one "drive" and then install > onto that. > > What is the best way to do that? > [...] I don't know whether my approach was "best" or not, but it seemed to work. I have a dual-bus scsi based system that can boot off of any drive I want. It has a matching pair of nice drives in the 1U case, and I wanted them to be identical copies. I ended up hanging an oddball external scsi drive off of the system, doing a minimal install onto that, booting the minimal install, running /stand/sysinstall and using it to partition and add boot blocks to both drives (w/ identical partition tables). Then I exited sysinstall and ran gmirror by hand to create a mirror (gm0s1) that consisted of da0s1 and da1s1. I bsdlabeled/partitioned it by hand, and then did a "dump ... | restore ..." for each partition from the external drive, the reset my bios to boot off of da0. I managed to expose a couple of problems which Pawel fixed, I think that some of the fixes have made it into the beta's and some are on their way. In particular, if you're going to swap onto the mirror, you need to give the system a chance to leave the swap partition in a nice state, otherwise the mirror appears to be dirty and get's resynched. Pawel touched up /etc/rc.d/{swap1,addswap}, and I had to retrain myself to use "shutdown -r now" instead of just "reboot" so that they'd have a chance to run. The second problem was more insidious, after updating to a newer BETA (3, I think) the system started insisting that one of the mirrors was dirty, no matter how I shutdown. Pawel figured out what was going on, and I tested a patch for it last night, which seems to work fine. Pawel's considering the aesthetics of the fix and will commit some version of it soon. g. From owner-freebsd-geom@FreeBSD.ORG Tue Sep 21 17:54:00 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D706116A4CE for ; Tue, 21 Sep 2004 17:54:00 +0000 (GMT) Received: from zibbi.icomtek.csir.co.za (zibbi.icomtek.csir.co.za [146.64.24.58]) by mx1.FreeBSD.org (Postfix) with ESMTP id B309B43D31 for ; Tue, 21 Sep 2004 17:53:49 +0000 (GMT) (envelope-from jhay@icomtek.csir.co.za) Received: from zibbi.icomtek.csir.co.za (localhost [127.0.0.1]) i8LHrhQD055124; Tue, 21 Sep 2004 19:53:43 +0200 (SAST) (envelope-from jhay@zibbi.icomtek.csir.co.za) Received: (from jhay@localhost)i8LHrhth055123; Tue, 21 Sep 2004 19:53:43 +0200 (SAST) (envelope-from jhay) Date: Tue, 21 Sep 2004 19:53:43 +0200 From: John Hay To: Paul Mather Message-ID: <20040921175343.GC52502@zibbi.icomtek.csir.co.za> References: <1095741109.9448.84.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1095741109.9448.84.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.4.1i cc: freebsd-geom@freebsd.org Subject: Re: Fresh install onto geom_stripe or geom_mirror? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2004 17:54:01 -0000 On Tue, Sep 21, 2004 at 12:31:51AM -0400, Paul Mather wrote: > Does anyone have any advice as to the best way to do a fresh install > onto an all-mirrored or all-striped geom setup? There seems to be a lot of possible combinations and I tried a few of them, each time thinking that I did something wrong and all the time it was just the smp race that was tripping me up. :-) It is your choice if you want to use "raw" disks or work in a slice. We now have two machines here with a geom mirror setup, the one using a slice and the one not. My preference would be to use a slice, but when I got it to work, I had the setup without slices and I was too lazy to change it. :-) I did a minimum install on the first disk. Then I prepared the second disk. For the raw disk I used glabel, gmirror and bsdlabel. I then mounted the new partitions and copied everything to the new partitions using tar. I then fixed up the new fstab to use the geom partitions and add geom_mirror_load="YES" and geom_label_load="YES" to the new loader.conf. Then I rebooted with the new disk and used gmirror insert to add the initial disk. I'm trying out swap on the mirror, so I modified the line "stop_cmd=':'" in /etc/rc.d/swap1 to "stop_cmd='swapoff -a'". The output of "sysctl -b kern.geom.conftxt" on this machine looks like this: 0 DISK ad6 80026361856 512 hd 16 sc 63 1 LABEL label/disk1 80026361344 512 i 0 o 0 2 MIRROR mirror/gm0 80026360832 512 3 BSD mirror/gm0e 77878877184 512 i 4 o 2147483648 ty 7 3 BSD mirror/gm0d 536870912 512 i 3 o 1610612736 ty 7 3 BSD mirror/gm0c 80026360832 512 i 2 o 0 ty 0 3 BSD mirror/gm0b 1073741824 512 i 1 o 536870912 ty 1 3 BSD mirror/gm0a 536870912 512 i 0 o 0 ty 7 0 DISK ad4 80026361856 512 hd 16 sc 63 1 LABEL label/disk0 80026361344 512 i 0 o 0 2 MIRROR mirror/gm0 80026360832 512 3 BSD mirror/gm0e 77878877184 512 i 4 o 2147483648 ty 7 3 BSD mirror/gm0d 536870912 512 i 3 o 1610612736 ty 7 3 BSD mirror/gm0c 80026360832 512 i 2 o 0 ty 0 3 BSD mirror/gm0b 1073741824 512 i 1 o 536870912 ty 1 3 BSD mirror/gm0a 536870912 512 i 0 o 0 ty 7 John -- John Hay -- John.Hay@icomtek.csir.co.za / jhay@FreeBSD.org From owner-freebsd-geom@FreeBSD.ORG Thu Sep 23 21:46:05 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9E2E016A4CE for ; Thu, 23 Sep 2004 21:46:05 +0000 (GMT) Received: from freebee.digiware.nl (dsl439.iae.nl [212.61.63.187]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10FC543D1F for ; Thu, 23 Sep 2004 21:46:04 +0000 (GMT) (envelope-from wjw@withagen.nl) Received: from [212.61.27.71] (dual [212.61.27.71]) by freebee.digiware.nl (8.12.10/8.12.10) with ESMTP id i8NLk19S098910 for ; Thu, 23 Sep 2004 23:46:02 +0200 (CEST) (envelope-from wjw@withagen.nl) Message-ID: <4153441A.9010303@withagen.nl> Date: Thu, 23 Sep 2004 23:46:02 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2004 21:46:05 -0000 Hi, I'm trying to build a geom mirror on 2 partitions on 2 different disks. Below you can see that the partitioning should be correct. But everytime I try to create the mirror, it seems to forget the 'h' on the second disk and take the whole disk a second part of the mirror. This strikes me a a sort of odd. Questions: - should this work? - should it work this way? - What does error=17 mean in the logging? Running -CURRENT as of 22 sept. --WjW asus# newfs /dev/ad2s1h /dev/ad2s1h: 13586.2MB (27824512 sectors) block size 16384, fragment size 2048 using 74 cylinder groups of 183.77MB, 11761 blks, 23552 inodes. super-block backups (for fsck -b #) at: 160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328, 3763680, 4140032, 4516384, 4892736, 5269088, 5645440, 6021792, 6398144, 6774496, 7150848, 7527200, 7903552, 8279904, 8656256, 9032608, 9408960, 9785312, 10161664, 10538016, 10914368, 11290720, 11667072, 12043424, 12419776, 12796128, 13172480, 13548832, 13925184, 14301536, 14677888, 15054240, 15430592, 15806944, 16183296, 16559648, 16936000, 17312352, 17688704, 18065056, 18441408, 18817760, 19194112, 19570464, 19946816, 20323168, 20699520, 21075872, 21452224, 21828576, 22204928, 22581280, 22957632, 23333984, 23710336, 24086688, 24463040, 24839392, 25215744, 25592096, 25968448, 26344800, 26721152, 27097504, 27473856 asus# gmirror label -v -s 2048 -b split home2 ad0s1h ad2s1h Metadata value stored on ad0s1h. Metadata value stored on ad2s1h. Done. asus# gmirror list Geom name: home2 State: COMPLETE Components: 2 Balance: split Slice: 2048 Flags: NONE SyncID: 1 ID: 2464573592 Providers: 1. Name: mirror/home2 Mediasize: 14246150656 (13G) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: ad0s1h Mediasize: 14246151168 (13G) Sectorsize: 512 Mode: r0w0e0 State: ACTIVE Priority: 0 Flags: NONE SyncID: 1 ID: 1917681422 2. Name: ad2s1 Mediasize: 40015954944 (37G) Sectorsize: 512 Mode: r0w0e0 State: ACTIVE Priority: 1 Flags: NONE SyncID: 1 ID: 1293810591 Geom name: home2.sync asus# bsdlabel -r ad0s1 # /dev/ad0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 2048 16384 28552 b: 2097152 2097152 swap c: 78156162 0 unused 0 0 # "raw" part, don't edit d: 2097152 4194304 4.2BSD 2048 16384 28552 e: 2097152 6291456 4.2BSD 2048 16384 28552 f: 20971520 8388608 4.2BSD 2048 16384 28552 g: 20971520 29360128 4.2BSD 2048 16384 28552 h: 27824514 50331648 4.2BSD 2048 16384 28552 asus# bsdlabel -r ad2s1 # /dev/ad2s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 2048 16384 28552 b: 2097152 2097152 swap c: 78156162 0 unused 0 0 # "raw" part, don't edit d: 2097152 4194304 4.2BSD 2048 16384 28552 e: 2097152 6291456 4.2BSD 2048 16384 28552 f: 20971520 8388608 4.2BSD 2048 16384 28552 g: 20971520 29360128 4.2BSD 2048 16384 28552 h: 27824514 50331648 4.2BSD 2048 16384 28552 /var/log/messages: Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Device home2 created (id=2464573592). Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Device home2: provider ad0s1h detected . Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Device home2: provider ad2s1 detected. Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Device home2: provider ad2s1 activated . Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1a, start 0 length 1073741824 e nd 1073741823 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1b, start 1073741824 length 107 3741824 end 2147483647 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1c, start 0 length 40015954944 end 40015954943 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1d, start 2147483648 length 107 3741824 end 3221225471 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1e, start 3221225472 length 107 3741824 end 4294967295 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1f, start 4294967296 length 107 37418240 end 15032385535 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1g, start 15032385536 length 10 737418240 end 25769803775 Sep 23 23:38:03 asus kernel: GEOM: Configure ad2s1h, start 25769803776 length 14 246151168 end 40015954943 Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Device home2: provider ad0s1h activate d. Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Device home2: provider mirror/home2 la unched. Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Cannot add disk ad2s1c to home2 (error =17). Sep 23 23:38:03 asus kernel: GEOM_MIRROR: Cannot add disk ad2s1h to home2 (error =17). Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2a, start 0 length 10737 41824 end 1073741823 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2b, start 1073741824 len gth 1073741824 end 2147483647 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2c, start 0 length 40015 954944 end 40015954943 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2d, start 2147483648 len gth 1073741824 end 3221225471 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2e, start 3221225472 len gth 1073741824 end 4294967295 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2f, start 4294967296 len gth 10737418240 end 15032385535 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2g, start 15032385536 le ngth 10737418240 end 25769803775 Sep 23 23:38:03 asus kernel: GEOM: Configure mirror/home2h, start 25769803776 le ngth 14246151168 end 40015954943 From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 02:43:52 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1608416A4CE for ; Fri, 24 Sep 2004 02:43:52 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF75143D39 for ; Fri, 24 Sep 2004 02:43:51 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8O2hm7K006841 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 23 Sep 2004 22:43:50 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8O2hgld025417 for ; Thu, 23 Sep 2004 22:43:42 -0400 (EDT) From: Paul Mather To: freebsd-geom@freebsd.org Content-Type: text/plain Message-Id: <1095993821.5665.124.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Thu, 23 Sep 2004 22:43:42 -0400 Content-Transfer-Encoding: 7bit Subject: gstripe stripe size units? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 02:43:52 -0000 What are the units for the stripe size argument to gstripe's "-s" option? E.g., does "gstripe label -s 4096 ..." mean use a stripe size of 4096 bytes; 4096 512-byte blocks; or what? The man page isn't specific. What is a good suggested value for stripe size? Does anyone have any empirical results to share? Does it make sense to split the file system block size over the providers of the stripe when calculating the stripe size to use? E.g., for a 16 KB block size over two providers, make the stripe size 8 KB. Or, is geom_stripe I/O done in much larger transfers? (Usually, small transfers are poor for performance, arguing in favour of a larger, rather than smaller, stripe size.) Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 07:49:27 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 78E8216A4CE for ; Fri, 24 Sep 2004 07:49:27 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F71D43D2F for ; Fri, 24 Sep 2004 07:49:27 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id E24FAACAFE; Fri, 24 Sep 2004 09:49:25 +0200 (CEST) Date: Fri, 24 Sep 2004 09:49:25 +0200 From: Pawel Jakub Dawidek To: Willem Jan Withagen Message-ID: <20040924074925.GD9550@darkness.comp.waw.pl> References: <4153441A.9010303@withagen.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ffoCPvUAPMgSXi6H" Content-Disposition: inline In-Reply-To: <4153441A.9010303@withagen.nl> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-geom@freebsd.org Subject: Re: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 07:49:27 -0000 --ffoCPvUAPMgSXi6H Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 23, 2004 at 11:46:02PM +0200, Willem Jan Withagen wrote: +> Hi, +>=20 +> I'm trying to build a geom mirror on 2 partitions on 2 different disks. +> Below you can see that the partitioning should be correct. +> But everytime I try to create the mirror, it seems to forget the 'h' on= =20 +> the second disk and take the whole disk a second part of the mirror. +> This strikes me a a sort of odd. +>=20 +> Questions: +> - should this work? +> - should it work this way? Yes. It is because gstripe store metadata in the last provider's sector and if two providers ends in the same place - you have a problem. In that case, you should create you stripe by adding '-h' option, which will add provider's name to metadata, so only this provider will be connected to your stripe. +> - What does error=3D17 mean in the logging? errno(2): EEXIST - File exists. It means that gstripe detects two (or more) providers which claim to be your stripe components. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --ffoCPvUAPMgSXi6H Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBU9GFForvXbEpPzQRAkS2AKDr0HSYg521UWWN5ATq7Ponlc20hwCgsh5Y 4RNbuSS+sssVYJVslRhLk/o= =YJ04 -----END PGP SIGNATURE----- --ffoCPvUAPMgSXi6H-- From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 07:55:55 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B73B216A4CE for ; Fri, 24 Sep 2004 07:55:55 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5C73643D1D for ; Fri, 24 Sep 2004 07:55:55 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id C47E9AC861; Fri, 24 Sep 2004 09:55:53 +0200 (CEST) Date: Fri, 24 Sep 2004 09:55:53 +0200 From: Pawel Jakub Dawidek To: Paul Mather Message-ID: <20040924075553.GE9550@darkness.comp.waw.pl> References: <1095993821.5665.124.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i3lJ51RuaGWuFYNw" Content-Disposition: inline In-Reply-To: <1095993821.5665.124.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-geom@freebsd.org Subject: Re: gstripe stripe size units? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 07:55:55 -0000 --i3lJ51RuaGWuFYNw Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 23, 2004 at 10:43:42PM -0400, Paul Mather wrote: +> What are the units for the stripe size argument to gstripe's "-s" +> option? E.g., does "gstripe label -s 4096 ..." mean use a stripe size +> of 4096 bytes; 4096 512-byte blocks; or what? The man page isn't +> specific. Bytes. +> What is a good suggested value for stripe size? Does anyone have any +> empirical results to share? Does it make sense to split the file system +> block size over the providers of the stripe when calculating the stripe +> size to use? E.g., for a 16 KB block size over two providers, make the +> stripe size 8 KB. Or, is geom_stripe I/O done in much larger transfers? +> (Usually, small transfers are poor for performance, arguing in favour of +> a larger, rather than smaller, stripe size.) The best you can do is to just try it. There is a tool for this, which I wrote for tests like this: src/tools/tools/raidtest/. If gstripe is running in "fast" mode (kern.geom.stripe.fast=3D1), size of stripe could be small, because then, it still sends as large I/O requests as possible and reorganize the data in memory, but this method consumes a lot of memory if you want it to be efficient. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --i3lJ51RuaGWuFYNw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBU9MJForvXbEpPzQRAoc8AKDsQvoWiUSJEYhNM/m/LY3Xz3bO1gCg7e+z P261sOYQADBHzz3cfA1Ybig= =OqiU -----END PGP SIGNATURE----- --i3lJ51RuaGWuFYNw-- From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 15:15:27 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9AA5C16A4DF for ; Fri, 24 Sep 2004 15:15:27 +0000 (GMT) Received: from imap.univie.ac.at (mailbox-lmtp.univie.ac.at [131.130.1.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id E987443D5C for ; Fri, 24 Sep 2004 15:15:26 +0000 (GMT) (envelope-from le@FreeBSD.org) Received: from pcle2.cc.univie.ac.at (pcle2.cc.univie.ac.at [131.130.2.177]) by imap.univie.ac.at (8.12.10/8.12.10) with ESMTP id i8OFFJ3w090664 for ; Fri, 24 Sep 2004 17:15:22 +0200 Date: Fri, 24 Sep 2004 17:15:37 +0200 (CEST) From: Lukas Ertl To: freebsd-geom@FreeBSD.org Message-ID: <20040924171245.Q549@pcle2.cc.univie.ac.at> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-DCC-ZID-Univie-Metrics: mx9.univie.ac.at 4249; Body=1 Fuz1=1 Fuz2=1 Subject: Outstanding I/O after 10 seconds X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 15:15:27 -0000 Hi there, I've now seen several times the following messages: WARNING: Final close of geom_dev(da9s1a) still has outstanding I/O after 10 seconds. Completing close anyway, panic may happen later. This happens when I run benchmarks on the specified device (like rawio). The strange thing is that when I look at what triggers this message, then cp->nstart is one less than cp->nend, like there was an additional g_io_deliver() on a bio. Any ideas why cp->nend would be larger than cp->nstart? cheers, le -- Lukas Ertl http://homepage.univie.ac.at/l.ertl/ le@FreeBSD.org http://people.freebsd.org/~le/ From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 16:06:25 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D8B8F16A4CF for ; Fri, 24 Sep 2004 16:06:25 +0000 (GMT) Received: from clueful.shagged.org (clueful.shagged.org [212.13.201.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A80E43D1F for ; Fri, 24 Sep 2004 16:06:25 +0000 (GMT) (envelope-from chris@clueful.shagged.org) Received: from chris by clueful.shagged.org with local (Exim 4.40 (FreeBSD)) id 1CAsaS-000KO5-U0 for freebsd-geom@FreeBSD.org; Fri, 24 Sep 2004 17:06:20 +0100 Date: Fri, 24 Sep 2004 17:06:20 +0100 From: Chris Elsworth To: freebsd-geom@FreeBSD.org Message-ID: <20040924160620.GA77997@shagged.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.6i Sender: Chris Elsworth X-Shagged-MailScanner-Information: See www.mailscanner.info for information X-Shagged-MailScanner: Found to be clean X-MailScanner-From: chris@clueful.shagged.org Subject: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 16:06:26 -0000 Hello, I'm having a play with gmirror, but I'm consistently running up against this problem: WARNING: Expected rawoffset 0, found 63 My chain of events is this: Start with a fresh disk, one slice covering the entire disk, no partitions yet: app1# disklabel -r da1s1 # /dev/da1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 71119692 0 unused 0 0 # "raw" part, don't edit Set up a gmirror on it (this disk only at the moment, no other providers): app1# gmirror label -hv gms1 da1s1 Metadata value stored on da1s1. Done. In response to this command on console I see the following: GEOM_MIRROR: Device gms1 created (id=2385087336). GEOM_MIRROR: Device gms1: provider da1s1 detected. GEOM_MIRROR: Device gms1: provider da1s1 activated. GEOM_MIRROR: Device gms1: provider mirror/gms1 launched. WARNING: Expected rawoffset 0, found 63 app1# disklabel -r /dev/mirror/gms1 # /dev/mirror/gms1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 71119692 63 unused 0 0 # "raw" part, don't edit partition c: partition extends past end of unit disklabel: partition c doesn't start at 0! disklabel: partition c doesn't cover the whole unit! disklabel: An incorrect partition c may cause problems for standard system utilities da1s1 still shows: app1# disklabel -r da1s1 # /dev/da1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 71119692 0 unused 0 0 # "raw" part, don't edit I feel I'm missing something blatantly obvious here. Is that offset of 63 something to do with metadata? Have I not left any space for metadata that I should have done? I was hoping that I'd be able to end up applying this gmirror command to an existing disk with live data on it, and then attach a free disk to achieve a complete mirror from an existing system? Are these warnings about partition c actually ignorable, or even normal? This system is 5.3-BETA5, built moments ago, and supped just before that. Appreciate any help. -- Chris From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 16:55:49 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CDE8116A4CE for ; Fri, 24 Sep 2004 16:55:49 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5F18A43D39 for ; Fri, 24 Sep 2004 16:55:49 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8OGtk4E015215 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 24 Sep 2004 12:55:48 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8OGteTt021903; Fri, 24 Sep 2004 12:55:40 -0400 (EDT) From: Paul Mather To: Chris Elsworth In-Reply-To: <20040924160620.GA77997@shagged.org> References: <20040924160620.GA77997@shagged.org> Content-Type: text/plain Message-Id: <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Fri, 24 Sep 2004 12:55:40 -0400 Content-Transfer-Encoding: 7bit cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 16:55:49 -0000 On Fri, 2004-09-24 at 12:06, Chris Elsworth wrote: > I feel I'm missing something blatantly obvious here. Is that offset of > 63 something to do with metadata? Have I not left any space for > metadata that I should have done? I was hoping that I'd be able to end > up applying this gmirror command to an existing disk with live data on > it, and then attach a free disk to achieve a complete mirror from an > existing system? Intuitively, I'd say it was more logical to create the mirror first on the slice and *then* label/partition and newfs that. In other words, attach the free disk and create the mirror on that (copying across data from the existing system). Then, when you have the mirror running, add the old drive to it for a two-disk mirror. I recently set up a geom_mirror on a 5.3-BETA5 system, so I thought I'd chip in with my experience. In my case, I was doing a fresh install onto a two-disk system. I did a regular minimal install onto disk 1. Then, I created slices on disk 2 (I wanted one slice for a geom_mirror, another slice for a geom_stripe). Next, I used glabel to label the slice on disk 2 I'd be using for my mirror. (I want to be able to have the disks move ATA ids and not worry about breaking the mirror.) I then created a geom_mirror using gmirror using the labelled provider as the sole provider (and using "-h" to hard-code the providers into the metadata). Now I had my mirror. Then, it was a matter of bsdlabelling the mirror. In my case, I used the same size partitions as my original install on disk 1, but there's no reason why you couldn't resize them. Then, I newfs'd each partition on the mirror and dump/restored the content over from disk 1. Finally, I edited the fstab on the mirror to use /dev/mirror/... for the filesystem mount points, and made sure I was loading geom_label and geom_mirror in its loader.conf. I rebooted using the mirror for a root file system. It was then a simple matter to erase (using dd) the content of the slice that had my initial installation on disk 1 (now inactive) and then label that slice using glabel. Finally, I inserted that labelled slice on disk 1 as a provider to the mirror and automagically geom_mirror began synchronising, copying data from drive 2 onto it. I only ran into three real problems (the first of which was my fault): 1) Initially, I forgot to put a boot block onto drive 2 (the one containing the mirror), and so ran into problems when I erased the drive 1 slice and added it to the mirror, because I'd unwittingly erased my only boot block. (I got around this by booting from the 5.3-BETA5 install CD.:) I'm a bit confused about the "correct way" to render a mirror bootable. (Do you "bsdlabel ... -B ..." the mirror or the underlying providers? I had the same problem with RAIDframe on my NetBSD/alpha. It seems intuitively for a RAID 1 you should be able to installboot onto /dev/raidXc and have the boot blocks transparently written to all the component drives, but you can't. You have to installboot onto the underlying component devices individually.) In the end, I used "fdisk -B" on both underlying drives, and this seemed to work. 2) Sysinstall no longer worked after I created the mirror. This made it inconvenience to do some post-installation tasks that sysinstall makes easy. So, if doing a fresh install, you might want to do those before making your installation into a mirrored one. 3) I was bitten by the lack of a "swapoff -a" in /etc/rc.d/swap1 on 5.3-BETA5. So, I would get a drive synchronising after every reboot. Hopefully the fix in -CURRENT (that I applied by hand) will be MT5 before 5.3-RELEASE. Having done both, I really prefer the easier setup of an all-geom_mirrored system from scratch than a root-on-gvinum setup (at least when using the _Complete FreeBSD_ technique of fiddling with the label and offsets). Geom_mirror seems much easier and cleaner to do; easier, I suspect, largely because you can start initially with just one provider in the mirror and then very easily add another when the original install drive becomes free. You also have much less of a learning curve than with Vinum. (I'm not knocking Vinum here. I use it and like it, but for those people who just want a basic RAID 1 setup on two drives, geom_mirror is the better [simpler] choice, IMHO.) Kudos to Pawel for such a useful GEOM class! Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 17:18:17 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0BA7D16A4CE for ; Fri, 24 Sep 2004 17:18:17 +0000 (GMT) Received: from clueful.shagged.org (clueful.shagged.org [212.13.201.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF21E43D45 for ; Fri, 24 Sep 2004 17:18:16 +0000 (GMT) (envelope-from chris@clueful.shagged.org) Received: from chris by clueful.shagged.org with local (Exim 4.40 (FreeBSD)) id 1CAthy-000LBX-B6; Fri, 24 Sep 2004 18:18:10 +0100 Date: Fri, 24 Sep 2004 18:18:10 +0100 From: Chris Elsworth To: Paul Mather Message-ID: <20040924171810.GA80912@shagged.org> References: <20040924160620.GA77997@shagged.org> <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.5.6i Sender: Chris Elsworth X-Shagged-MailScanner-Information: See www.mailscanner.info for information X-Shagged-MailScanner: Found to be clean X-MailScanner-From: chris@clueful.shagged.org cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 17:18:17 -0000 On Fri, Sep 24, 2004 at 12:55:40PM -0400, Paul Mather wrote: > On Fri, 2004-09-24 at 12:06, Chris Elsworth wrote: > > > I feel I'm missing something blatantly obvious here. Is that offset of > > 63 something to do with metadata? Have I not left any space for > > metadata that I should have done? I was hoping that I'd be able to end > > up applying this gmirror command to an existing disk with live data on > > it, and then attach a free disk to achieve a complete mirror from an > > existing system? > > Intuitively, I'd say it was more logical to create the mirror first on > the slice and *then* label/partition and newfs that. In other words, > attach the free disk and create the mirror on that (copying across data > from the existing system). Then, when you have the mirror running, add > the old drive to it for a two-disk mirror. Well, I think that's basically what I'm doing? I'm practising on the free disk first (good job, really). Starting with a completely fresh disk with no partition table or anything (dd if=/dev/zero of=/dev/da1 count=100 should ensure that?) I'm using sysinstall to create a slice; the Fdisk option in sysinstall, and all I'm doing there is using the entire disk and making it bootable. I save that out, then quit sysinstall and what I'm left with is: app1# disklabel -r da1s1 # /dev/da1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 71119692 0 unused 0 0 # "raw" part, don't edit Is that what you'd start off with? So then by mirroring the slice, I'd do: gmirror label -hv gms1 da1s1 And then I can start partitioning in /dev/mirror/gms1a etc, after using disklabel to create it. But the gmirror command is where the problems start; see pastes of disklabel on the gms1 device in first post. > In my case, I was doing a fresh install onto a two-disk system. I did a > regular minimal install onto disk 1. Then, I created slices on disk 2 > (I wanted one slice for a geom_mirror, another slice for a > geom_stripe). Next, I used glabel to label the slice on disk 2 I'd be > using for my mirror. (I want to be able to have the disks move ATA ids > and not worry about breaking the mirror.) I then created a geom_mirror > using gmirror using the labelled provider as the sole provider (and > using "-h" to hard-code the providers into the metadata). Now I had my > mirror. > > Then, it was a matter of bsdlabelling the mirror. At this point, or indeed, even now, if you disklabel -r on your mirror device, does it not give you warnings about partition c? If not, that's where we start to differ :) > Kudos to Pawel for such a useful GEOM class! Definitely, I've always been a fan of Solstice Disk-Suite on Solaris and this looks like being a very good FreeBSD competitor :) -- Chris From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 18:36:37 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 39C4916A4CE for ; Fri, 24 Sep 2004 18:36:37 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 992E643D5D for ; Fri, 24 Sep 2004 18:36:36 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8OIaXT0015505 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 24 Sep 2004 14:36:35 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8OIaRKI011489; Fri, 24 Sep 2004 14:36:28 -0400 (EDT) From: Paul Mather To: Chris Elsworth In-Reply-To: <20040924171810.GA80912@shagged.org> References: <20040924160620.GA77997@shagged.org> <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> <20040924171810.GA80912@shagged.org> Content-Type: text/plain Message-Id: <1096050986.9306.72.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Fri, 24 Sep 2004 14:36:27 -0400 Content-Transfer-Encoding: 7bit cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 18:36:37 -0000 On Fri, 2004-09-24 at 13:18, Chris Elsworth wrote: > At this point, or indeed, even now, if you disklabel -r on your mirror > device, does it not give you warnings about partition c? If not, > that's where we start to differ :) First of all, you should probably get into the habit of using bsdlabel under 5.x, not disklabel. (I remember at one point that disklabel used to complain about that rawoffset difference.) Perhaps they are exactly the same nowadays, but at one point they weren't. Here's what I get when I run disklabel on my mirror: >>>>> zappa# disklabel -r /dev/mirror/raid1 # /dev/mirror/raid1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 524288 0 4.2BSD 2048 16384 32776 b: 2039392 524288 swap c: 20964760 0 unused 0 0 # "raw" part, don't edit d: 524288 2563680 4.2BSD 2048 16384 32776 e: 17876792 3087968 4.2BSD 2048 16384 28552 <<<<< The only time I get a complaint about partition c is when I run it on the underlying components (providers). Then, it complains about it not covering the whole device, which makes sense because it is picking up a "ghost" label of the mirror, which is slightly smaller than the underlying providers because it uses a few sectors from them for its own metadata. E.g.: >>>>> zappa# disklabel -r /dev/ad1s1 # /dev/ad1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 524288 0 4.2BSD 2048 16384 32776 b: 2039392 524288 swap c: 20964760 0 unused 0 0 # "raw" part, don't edit d: 524288 2563680 4.2BSD 2048 16384 32776 e: 17876792 3087968 4.2BSD 2048 16384 28552 disklabel: partition c doesn't cover the whole unit! disklabel: An incorrect partition c may cause problems for standard system utilities zappa# disklabel -r /dev/label/drive1s1 # /dev/label/drive1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 524288 0 4.2BSD 2048 16384 32776 b: 2039392 524288 swap c: 20964760 0 unused 0 0 # "raw" part, don't edit d: 524288 2563680 4.2BSD 2048 16384 32776 e: 17876792 3087968 4.2BSD 2048 16384 28552 disklabel: partition c doesn't cover the whole unit! disklabel: An incorrect partition c may cause problems for standard system utilities zappa# gmirror list Geom name: raid1 State: COMPLETE Components: 2 Balance: split Slice: 4096 Flags: NONE SyncID: 1 ID: 2974744038 Providers: 1. Name: mirror/raid1 Mediasize: 10733957120 (10.0) Sectorsize: 512 Mode: r4w4e1 Consumers: 1. Name: label/drive0s1 Mediasize: 10733957632 (10.0) Sectorsize: 512 Mode: r4w4e2 State: ACTIVE Priority: 0 Flags: DIRTY, HARDCODED SyncID: 1 ID: 3924739636 2. Name: label/drive1s1 Mediasize: 10733957632 (10.0) Sectorsize: 512 Mode: r4w4e2 State: ACTIVE Priority: 0 Flags: DIRTY, HARDCODED SyncID: 1 ID: 1083938085 Geom name: raid1.sync zappa# <<<<< (I get the same results when using bsdlabel instead of disklabel. Note how the mediasize of the providers of the mirror is slightly bigger [by one 512-byte sector] than that of the mirror itself.) Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 19:19:45 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E6E316A4CE for ; Fri, 24 Sep 2004 19:19:45 +0000 (GMT) Received: from clueful.shagged.org (clueful.shagged.org [212.13.201.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1E04643D53 for ; Fri, 24 Sep 2004 19:19:45 +0000 (GMT) (envelope-from chris@clueful.shagged.org) Received: from chris by clueful.shagged.org with local (Exim 4.40 (FreeBSD)) id 1CAvbW-000MLa-U8; Fri, 24 Sep 2004 20:19:38 +0100 Date: Fri, 24 Sep 2004 20:19:38 +0100 From: Chris Elsworth To: Paul Mather Message-ID: <20040924191938.GA84500@shagged.org> References: <20040924160620.GA77997@shagged.org> <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> <20040924171810.GA80912@shagged.org> <1096050986.9306.72.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1096050986.9306.72.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.5.6i Sender: Chris Elsworth X-Shagged-MailScanner-Information: See www.mailscanner.info for information X-Shagged-MailScanner: Found to be clean X-MailScanner-From: chris@clueful.shagged.org cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 19:19:45 -0000 On Fri, Sep 24, 2004 at 02:36:27PM -0400, Paul Mather wrote: > On Fri, 2004-09-24 at 13:18, Chris Elsworth wrote: > > > At this point, or indeed, even now, if you disklabel -r on your mirror > > device, does it not give you warnings about partition c? If not, > > that's where we start to differ :) > > First of all, you should probably get into the habit of using bsdlabel > under 5.x, not disklabel. (I remember at one point that disklabel used > to complain about that rawoffset difference.) Perhaps they are exactly > the same nowadays, but at one point they weren't. They seem to be exactly the same at the moment (same inode) although that might not be true forever, so noted, thanks :) > Here's what I get when I run disklabel on my mirror: > > >>>>> > zappa# disklabel -r /dev/mirror/raid1 > # /dev/mirror/raid1: > 8 partitions: > # size offset fstype [fsize bsize bps/cpg] > a: 524288 0 4.2BSD 2048 16384 32776 > b: 2039392 524288 swap > c: 20964760 0 unused 0 0 # "raw" part, don't edit > d: 524288 2563680 4.2BSD 2048 16384 32776 > e: 17876792 3087968 4.2BSD 2048 16384 28552 > <<<<< > > The only time I get a complaint about partition c is when I run it on > the underlying components (providers). Then, it complains about it not > covering the whole device, which makes sense because it is picking up a > "ghost" label of the mirror, which is slightly smaller than the > underlying providers because it uses a few sectors from them for its own > metadata. E.g.: Aha! After reading that a couple of times something clicked - the mirror had the disklabel from the disk *before* the mirror was applied. So I created the mirror, then cleaned the mirror with a dd of zero bytes to the first few sectors, then bsdlabel -w /dev/mirror/gm, and I have the following: app1# bsdlabel -r /dev/da1s1 # /dev/da1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 71119675 16 unused 0 0 c: 71119691 0 unused 0 0 # "raw" part, don't edit bsdlabel: partition c doesn't cover the whole unit! bsdlabel: An incorrect partition c may cause problems for standard system utilities app1# bsdlabel -r /dev/mirror/gm # /dev/mirror/gm: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 71119675 16 unused 0 0 c: 71119691 0 unused 0 0 # "raw" part, don't edit So now another two things crop up - where has that a partition come from (perhaps that's just what bsdlabel -w does by default, no big deal) but more importantly, why do they both have an offset of 16 bytes.. It didn't seem to mind me changing it to zero, though. So now I'll see if I can get it booting :) Thanks for the brain stirring, it did the trick :) -- Chris From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 20:05:58 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3810B16A4CE for ; Fri, 24 Sep 2004 20:05:58 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9419643D4C for ; Fri, 24 Sep 2004 20:05:57 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8OK5tbv015953 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 24 Sep 2004 16:05:56 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8OK5ncB029382; Fri, 24 Sep 2004 16:05:49 -0400 (EDT) From: Paul Mather To: Chris Elsworth In-Reply-To: <20040924191938.GA84500@shagged.org> References: <20040924160620.GA77997@shagged.org> <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> <20040924171810.GA80912@shagged.org> <1096050986.9306.72.camel@zappa.Chelsea-Ct.Org> <20040924191938.GA84500@shagged.org> Content-Type: text/plain Message-Id: <1096056348.9306.87.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Fri, 24 Sep 2004 16:05:48 -0400 Content-Transfer-Encoding: 7bit cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 20:05:58 -0000 On Fri, 2004-09-24 at 15:19, Chris Elsworth wrote: > app1# bsdlabel -r /dev/mirror/gm > # /dev/mirror/gm: > 8 partitions: > # size offset fstype [fsize bsize bps/cpg] > a: 71119675 16 unused 0 0 > c: 71119691 0 unused 0 0 # "raw" part, don't edit > > > So now another two things crop up - where has that a partition come from > (perhaps that's just what bsdlabel -w does by default, no big deal) > but more importantly, why do they both have an offset of 16 bytes.. > It didn't seem to mind me changing it to zero, though. So now I'll see > if I can get it booting :) I believe the above is bsdlabel's idea of an "auto" label. I don't know why the 16 sector offset for the "a" partition, because you don't get that when you label a slice via sysinstall and choose the "auto defaults." (Maybe there's a case for making bsdlabel's "auto" label behave the same as sysinstall.) I edited mine from 16 to 0, too, when using (roughly) the same label as my sysinstalled minimal system on the mirror device. I haven't noticed any ill effects (touch wood:). Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 21:47:08 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 21B6116A4CE; Fri, 24 Sep 2004 21:47:08 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7280543D41; Fri, 24 Sep 2004 21:47:07 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id i8OLl65Y054597; Fri, 24 Sep 2004 23:47:06 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Lukas Ertl From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 24 Sep 2004 17:15:37 +0200." <20040924171245.Q549@pcle2.cc.univie.ac.at> Date: Fri, 24 Sep 2004 23:47:06 +0200 Message-ID: <54596.1096062426@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: freebsd-geom@freebsd.org Subject: Re: Outstanding I/O after 10 seconds X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 21:47:08 -0000 In message <20040924171245.Q549@pcle2.cc.univie.ac.at>, Lukas Ertl writes: >Hi there, > >I've now seen several times the following messages: > >WARNING: Final close of geom_dev(da9s1a) still has outstanding I/O after >10 seconds. Completing close anyway, panic may happen later. > >This happens when I run benchmarks on the specified device (like rawio). >The strange thing is that when I look at what triggers this message, then >cp->nstart is one less than cp->nend, like there was an additional >g_io_deliver() on a bio. > >Any ideas why cp->nend would be larger than cp->nstart? A bug. A very serious bug. Likely a double biodone(). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 21:50:27 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1C2A116A4CE for ; Fri, 24 Sep 2004 21:50:27 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 719CE43D3F for ; Fri, 24 Sep 2004 21:50:25 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id i8OLoNk7054733; Fri, 24 Sep 2004 23:50:24 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Paul Mather From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 24 Sep 2004 16:05:48 EDT." <1096056348.9306.87.camel@zappa.Chelsea-Ct.Org> Date: Fri, 24 Sep 2004 23:50:23 +0200 Message-ID: <54732.1096062623@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: freebsd-geom@freebsd.org cc: Chris Elsworth Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 21:50:27 -0000 In message <1096056348.9306.87.camel@zappa.Chelsea-Ct.Org>, Paul Mather writes: >On Fri, 2004-09-24 at 15:19, Chris Elsworth wrote: > >> app1# bsdlabel -r /dev/mirror/gm >> # /dev/mirror/gm: >> 8 partitions: >> # size offset fstype [fsize bsize bps/cpg] >> a: 71119675 16 unused 0 0 >> c: 71119691 0 unused 0 0 # "raw" part, don't edit >> >> >> So now another two things crop up - where has that a partition come from >> (perhaps that's just what bsdlabel -w does by default, no big deal) >> but more importantly, why do they both have an offset of 16 bytes.. >> It didn't seem to mind me changing it to zero, though. So now I'll see >> if I can get it booting :) > >I believe the above is bsdlabel's idea of an "auto" label. I don't know >why the 16 sector offset for the "a" partition, because you don't get >that when you label a slice via sysinstall and choose the "auto >defaults." (Maybe there's a case for making bsdlabel's "auto" label >behave the same as sysinstall.) The 16 offset is to protect the disklabel and boot code. You have no idea how much I hate the person who made the hack to leave the metadata inside the trafic partitions. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 21:52:03 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 817BD16A4CE; Fri, 24 Sep 2004 21:52:03 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10B8C43D6B; Fri, 24 Sep 2004 21:52:03 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8OLq0Sk016360 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 24 Sep 2004 17:52:02 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8OLpsR7023030; Fri, 24 Sep 2004 17:51:55 -0400 (EDT) From: Paul Mather To: Pawel Jakub Dawidek In-Reply-To: <20040924075553.GE9550@darkness.comp.waw.pl> References: <1095993821.5665.124.camel@zappa.Chelsea-Ct.Org> <20040924075553.GE9550@darkness.comp.waw.pl> Content-Type: text/plain Message-Id: <1096062713.9306.119.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Fri, 24 Sep 2004 17:51:54 -0400 Content-Transfer-Encoding: 7bit cc: freebsd-geom@freebsd.org Subject: Re: gstripe stripe size units? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 21:52:03 -0000 On Fri, 2004-09-24 at 03:55, Pawel Jakub Dawidek wrote: > The best you can do is to just try it. There is a tool for this, which > I wrote for tests like this: src/tools/tools/raidtest/. Thanks for the pointer. I used that program over various stripe sizes (doubling each time) on a two-drive geom_stripe. In my tests, it seemed that for stripes 4096--32768 the performance was roughly even (+1 req/sec at each doubling). For 65536--524288 there was a reasonable increase (+10 req/sec at each doubling). After that, it levelled off somewhat until 4 MB to 16 MB it seemed to settle at 131 and 132 req/sec. > If gstripe is running in "fast" mode (kern.geom.stripe.fast=1), size of > stripe could be small, because then, it still sends as large I/O requests > as possible and reorganize the data in memory, but this method consumes > a lot of memory if you want it to be efficient. One thing that puzzles me is that no matter how large I made the stripe size, I never got a kern.geom.stripe.fast_failed > 0. Here is what I have after the last run of raidtest with a stripe size of 16 MB: >>>>> Read 50000 requests from raidtest.data. Number of READ requests: 24991. Number of WRITE requests: 25009. Number of bytes to transmit: 3288266752. Number of processes: 10. Bytes per second: 8745390 Requests per second: 132 <<<<< Here are the values afterwards for kern.geom.stripe sysctls: >>>>> kern.geom.stripe.debug: 0 kern.geom.stripe.fast: 1 kern.geom.stripe.maxmem: 6553600 kern.geom.stripe.fast_failed: 0 <<<<< I'm puzzled because a stripe of 16 MB will not fit in 6553600 bytes, so surely fast_failed should be > 0 at stripes of 8 MB or greater in my tests for the above value of kern.geom.stripe.maxmem? Also, I don't know what the distribution of request sizes is in raidtest.data. The raidtest program operates over the raw device, and so may not necessarily behave in terms of issuing requests as, say, a UFS filesystem might. Any thoughts for tuning stripe size for a UFS filesystem so as not to have a bad effect on the VM system? Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Fri Sep 24 22:10:45 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A197416A4CE for ; Fri, 24 Sep 2004 22:10:45 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id 56B8F43D45 for ; Fri, 24 Sep 2004 22:10:45 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8OMAgen016433 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 24 Sep 2004 18:10:44 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8OMAaXG006332; Fri, 24 Sep 2004 18:10:37 -0400 (EDT) From: Paul Mather To: Poul-Henning Kamp In-Reply-To: <54732.1096062623@critter.freebsd.dk> References: <54732.1096062623@critter.freebsd.dk> Content-Type: text/plain Message-Id: <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Fri, 24 Sep 2004 18:10:36 -0400 Content-Transfer-Encoding: 7bit cc: freebsd-geom@freebsd.org cc: Chris Elsworth Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2004 22:10:45 -0000 On Fri, 2004-09-24 at 17:50, Poul-Henning Kamp wrote: > In message <1096056348.9306.87.camel@zappa.Chelsea-Ct.Org>, Paul Mather writes: > >I believe the above is bsdlabel's idea of an "auto" label. I don't know > >why the 16 sector offset for the "a" partition, because you don't get > >that when you label a slice via sysinstall and choose the "auto > >defaults." (Maybe there's a case for making bsdlabel's "auto" label > >behave the same as sysinstall.) > > The 16 offset is to protect the disklabel and boot code. You have no > idea how much I hate the person who made the hack to leave the metadata > inside the trafic partitions. So I guess the case to be made is really to have sysinstall behave the same as bsdlabel, not vice versa. I'm presuming sysinstall-style partition-a-starts-at-offset-0 labels are still safe, though, right? Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 00:28:29 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8E59416A4CE for ; Sat, 25 Sep 2004 00:28:29 +0000 (GMT) Received: from clueful.shagged.org (clueful.shagged.org [212.13.201.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 311CE43D2F for ; Sat, 25 Sep 2004 00:28:29 +0000 (GMT) (envelope-from chris@clueful.shagged.org) Received: from chris by clueful.shagged.org with local (Exim 4.40 (FreeBSD)) id 1CB0QD-000PkM-6j; Sat, 25 Sep 2004 01:28:17 +0100 Date: Sat, 25 Sep 2004 01:28:17 +0100 From: Chris Elsworth To: Paul Mather Message-ID: <20040925002817.GA98852@shagged.org> References: <54732.1096062623@critter.freebsd.dk> <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.5.6i Sender: Chris Elsworth X-Shagged-MailScanner-Information: See www.mailscanner.info for information X-Shagged-MailScanner: Found to be clean X-MailScanner-From: chris@clueful.shagged.org cc: Poul-Henning Kamp cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 00:28:29 -0000 On Fri, Sep 24, 2004 at 06:10:36PM -0400, Paul Mather wrote: > > > > The 16 offset is to protect the disklabel and boot code. You have no > > idea how much I hate the person who made the hack to leave the metadata > > inside the trafic partitions. > > So I guess the case to be made is really to have sysinstall behave the > same as bsdlabel, not vice versa. Ah, so we should really be leaving that 16 offset there? I'm having trouble getting the kernel to mount a mirrored partition as root now - I have geom_mirror_load="YES" in loader.conf, but if I boot -as I can only see da0* and da1* available, no gmirror drives. If I just select a daX drive then kldstat, geom_mirror is present in the list. Additionally, gmirror list shows nothing until I kldunload it, then kldload it again, at which point I see GEOM_MIRROR messages about my configured mirror. Did I miss something out of loader.conf ? I can't find anything to suggest I need more in there. -- Chris From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 07:01:25 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EC37216A4CE for ; Sat, 25 Sep 2004 07:01:25 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B86C43D49 for ; Sat, 25 Sep 2004 07:01:25 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id i8P71MjE063061; Sat, 25 Sep 2004 09:01:23 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Paul Mather From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 24 Sep 2004 18:10:36 EDT." <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> Date: Sat, 25 Sep 2004 09:01:22 +0200 Message-ID: <63060.1096095682@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: freebsd-geom@freebsd.org cc: Chris Elsworth Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 07:01:26 -0000 In message <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org>, Paul Mather writes : >On Fri, 2004-09-24 at 17:50, Poul-Henning Kamp wrote: >> In message <1096056348.9306.87.camel@zappa.Chelsea-Ct.Org>, Paul Mather writes: > >> >I believe the above is bsdlabel's idea of an "auto" label. I don't know >> >why the 16 sector offset for the "a" partition, because you don't get >> >that when you label a slice via sysinstall and choose the "auto >> >defaults." (Maybe there's a case for making bsdlabel's "auto" label >> >behave the same as sysinstall.) >> >> The 16 offset is to protect the disklabel and boot code. You have no >> idea how much I hate the person who made the hack to leave the metadata >> inside the trafic partitions. > >So I guess the case to be made is really to have sysinstall behave the >same as bsdlabel, not vice versa. yes. >I'm presuming sysinstall-style partition-a-starts-at-offset-0 labels are >still safe, though, right? As "safe" as they always were, which means "not very". They have a tendency to blow up on people if they try to change them. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 13:00:22 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F45216A4CE; Sat, 25 Sep 2004 13:00:22 +0000 (GMT) Received: from freebee.digiware.nl (dsl439.iae.nl [212.61.63.187]) by mx1.FreeBSD.org (Postfix) with ESMTP id B669D43D49; Sat, 25 Sep 2004 13:00:20 +0000 (GMT) (envelope-from wjw@withagen.nl) Received: from [212.61.27.71] (dual [212.61.27.71]) by freebee.digiware.nl (8.12.10/8.12.10) with ESMTP id i8PD0I9S047670; Sat, 25 Sep 2004 15:00:19 +0200 (CEST) (envelope-from wjw@withagen.nl) Message-ID: <41556BE2.5060803@withagen.nl> Date: Sat, 25 Sep 2004 15:00:18 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4153441A.9010303@withagen.nl> <20040924074925.GD9550@darkness.comp.waw.pl> In-Reply-To: <20040924074925.GD9550@darkness.comp.waw.pl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-geom@freebsd.org Subject: Re: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 13:00:22 -0000 Pawel Jakub Dawidek wrote: >On Thu, Sep 23, 2004 at 11:46:02PM +0200, Willem Jan Withagen wrote: >+> Hi, >+> >+> I'm trying to build a geom mirror on 2 partitions on 2 different disks. >+> Below you can see that the partitioning should be correct. >+> But everytime I try to create the mirror, it seems to forget the 'h' on >+> the second disk and take the whole disk a second part of the mirror. >+> This strikes me a a sort of odd. >+> >+> Questions: >+> - should this work? >+> - should it work this way? > >Yes. It is because gstripe store metadata in the last provider's sector >and if two providers ends in the same place - you have a problem. >In that case, you should create you stripe by adding '-h' option, which >will add provider's name to metadata, so only this provider will be >connected to your stripe. > > the -h was my friend..... I do not really understand why it was trying to use the whole slice for mirroring?? But that might be my ignorance. --WjW From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 13:19:10 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2178216A4CE for ; Sat, 25 Sep 2004 13:19:10 +0000 (GMT) Received: from clueful.shagged.org (clueful.shagged.org [212.13.201.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id D933143D2F for ; Sat, 25 Sep 2004 13:19:09 +0000 (GMT) (envelope-from chris@clueful.shagged.org) Received: from chris by clueful.shagged.org with local (Exim 4.40 (FreeBSD)) id 1CBCRy-0007w0-BM; Sat, 25 Sep 2004 14:18:54 +0100 Date: Sat, 25 Sep 2004 14:18:54 +0100 From: Chris Elsworth To: Paul Mather Message-ID: <20040925131854.GA30301@shagged.org> References: <54732.1096062623@critter.freebsd.dk> <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> <20040925002817.GA98852@shagged.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040925002817.GA98852@shagged.org> User-Agent: Mutt/1.5.6i Sender: Chris Elsworth X-Shagged-MailScanner-Information: See www.mailscanner.info for information X-Shagged-MailScanner: Found to be clean X-MailScanner-From: chris@clueful.shagged.org cc: Poul-Henning Kamp cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 13:19:10 -0000 On Sat, Sep 25, 2004 at 01:28:17AM +0100, Chris Elsworth wrote: > > I'm having trouble getting the kernel to mount a mirrored partition as > root now - I have geom_mirror_load="YES" in loader.conf, but if I > > boot -as > > I can only see da0* and da1* available, no gmirror drives. If I just > select a daX drive then kldstat, geom_mirror is present in the list. > Additionally, gmirror list shows nothing until I kldunload it, then > kldload it again, at which point I see GEOM_MIRROR messages about my > configured mirror. Did I miss something out of loader.conf ? I can't > find anything to suggest I need more in there. I've just tried the root_mount.patch from pjd but it hasn't made any difference, so I wonder if this is a new problem? I've tried setting kern.geom.mirror.debug=2 in loader.conf, and the only messages I get from GEOM_MIRROR are "Tasting fd0" "Tasting cd0" "Tasting da0" "Tasting da0s1a" .. "Tasting da1" "Tasting da1a" "Tasting da1c" It does this for each partition I have configured, but nothing else. It doesn't seem to recognise the mirror on da1. Anyone know what's going on here.. ? Thanks, -- Chris From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 13:34:17 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C8B7D16A4CE for ; Sat, 25 Sep 2004 13:34:17 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6E42643D1D for ; Sat, 25 Sep 2004 13:34:17 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id BF77EACAFB; Sat, 25 Sep 2004 15:34:15 +0200 (CEST) Date: Sat, 25 Sep 2004 15:34:15 +0200 From: Pawel Jakub Dawidek To: Chris Elsworth Message-ID: <20040925133415.GJ9550@darkness.comp.waw.pl> References: <54732.1096062623@critter.freebsd.dk> <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> <20040925002817.GA98852@shagged.org> <20040925131854.GA30301@shagged.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZG5hGh9V5E9QzVHS" Content-Disposition: inline In-Reply-To: <20040925131854.GA30301@shagged.org> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: Paul Mather cc: Poul-Henning Kamp cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 13:34:17 -0000 --ZG5hGh9V5E9QzVHS Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 25, 2004 at 02:18:54PM +0100, Chris Elsworth wrote: +> Anyone know what's going on here.. ? Yes, gmirror ignores providers with 0 sector size. It has been fixed in -CURRENT: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/geom/mirror/g_mirror.c.diff?= r1=3D1.26&r2=3D1.27 --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --ZG5hGh9V5E9QzVHS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBVXPXForvXbEpPzQRAi1rAKDU4iQ38cnAkvm1cAWMSgrk+URgJQCeJUmV 4fSmHw5SwHq1C69GGnju71g= =8kQ9 -----END PGP SIGNATURE----- --ZG5hGh9V5E9QzVHS-- From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 13:38:16 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 386DA16A4CE for ; Sat, 25 Sep 2004 13:38:16 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id C567D43D2D for ; Sat, 25 Sep 2004 13:38:15 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 78C7BACAFB; Sat, 25 Sep 2004 15:38:14 +0200 (CEST) Date: Sat, 25 Sep 2004 15:38:14 +0200 From: Pawel Jakub Dawidek To: Paul Mather Message-ID: <20040925133814.GK9550@darkness.comp.waw.pl> References: <20040924160620.GA77997@shagged.org> <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lHuqAdgBYNjQz/wy" Content-Disposition: inline In-Reply-To: <1096044939.9306.54.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-geom@freebsd.org cc: Chris Elsworth Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 13:38:16 -0000 --lHuqAdgBYNjQz/wy Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 24, 2004 at 12:55:40PM -0400, Paul Mather wrote: +> I'm a bit confused about the "correct way" to render a mirror bootable.= =20 +> (Do you "bsdlabel ... -B ..." the mirror or the underlying providers? If you operate on full disk providers (e.g. adX, daX), there should be no difference, as gmirror uses the last provider's sector. +> 2) Sysinstall no longer worked after I created the mirror. This made it +> inconvenience to do some post-installation tasks that sysinstall makes +> easy. So, if doing a fresh install, you might want to do those before +> making your installation into a mirrored one. It has been fixes in -CURRENT: http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libdisk/open_disk.c.diff?r1= =3D1.7&r2=3D1.8 --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --lHuqAdgBYNjQz/wy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBVXTGForvXbEpPzQRApdPAJ0QID9BgWJjmNfPSk4RbXvpJjQtWQCg5VK+ o01zygznelCPo9X4A/ZL1jU= =XTHn -----END PGP SIGNATURE----- --lHuqAdgBYNjQz/wy-- From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 13:41:14 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 11A0E16A4CE for ; Sat, 25 Sep 2004 13:41:14 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id A69FC43D1D for ; Sat, 25 Sep 2004 13:41:13 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 3AC76ACC65; Sat, 25 Sep 2004 15:41:12 +0200 (CEST) Date: Sat, 25 Sep 2004 15:41:12 +0200 From: Pawel Jakub Dawidek To: Willem Jan Withagen Message-ID: <20040925134112.GL9550@darkness.comp.waw.pl> References: <4153441A.9010303@withagen.nl> <20040924074925.GD9550@darkness.comp.waw.pl> <41556BE2.5060803@withagen.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="00sTfE/IIAT5d2r5" Content-Disposition: inline In-Reply-To: <41556BE2.5060803@withagen.nl> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-geom@freebsd.org Subject: Re: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 13:41:14 -0000 --00sTfE/IIAT5d2r5 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 25, 2004 at 03:00:18PM +0200, Willem Jan Withagen wrote: +> >Yes. It is because gstripe store metadata in the last provider's sector +> >and if two providers ends in the same place - you have a problem. +> >In that case, you should create you stripe by adding '-h' option, which +> >will add provider's name to metadata, so only this provider will be +> >connected to your stripe. +> >=20 +> > +> the -h was my friend..... +> I do not really understand why it was trying to use the whole slice for= =20 +> mirroring?? But that might be my ignorance. Because those two providers share the same last sector, so they share the same metadata and there is no information by default, which provider is the right one (you should add '-h' option). IMHO this is a better behaviour, because you can move your disk, copy it to another disk via dd(1), etc. and it will be still recognized by gmirror. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --00sTfE/IIAT5d2r5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBVXV4ForvXbEpPzQRAgArAKDP4jYRnHAkl/zZNhbsbwd2vTbN7QCeOSPI EUR9O9zC4WtXba4I6A8H/8c= =FcBY -----END PGP SIGNATURE----- --00sTfE/IIAT5d2r5-- From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 14:01:47 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6901216A51F; Sat, 25 Sep 2004 14:01:47 +0000 (GMT) Received: from freebee.digiware.nl (dsl439.iae.nl [212.61.63.187]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6434B43D55; Sat, 25 Sep 2004 14:01:46 +0000 (GMT) (envelope-from wjw@withagen.nl) Received: from [212.61.27.71] (dual [212.61.27.71]) by freebee.digiware.nl (8.12.10/8.12.10) with ESMTP id i8PE1i9S049702; Sat, 25 Sep 2004 16:01:45 +0200 (CEST) (envelope-from wjw@withagen.nl) Message-ID: <41557A48.6090403@withagen.nl> Date: Sat, 25 Sep 2004 16:01:44 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4153441A.9010303@withagen.nl> <20040924074925.GD9550@darkness.comp.waw.pl> <41556BE2.5060803@withagen.nl> <20040925134112.GL9550@darkness.comp.waw.pl> In-Reply-To: <20040925134112.GL9550@darkness.comp.waw.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-geom@FreeBSD.org Subject: Re: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 14:01:47 -0000 Pawel Jakub Dawidek wrote: >On Sat, Sep 25, 2004 at 03:00:18PM +0200, Willem Jan Withagen wrote: >+> >Yes. It is because gstripe store metadata in the last provider's sector >+> >and if two providers ends in the same place - you have a problem. >+> >In that case, you should create you stripe by adding '-h' option, which >+> >will add provider's name to metadata, so only this provider will be >+> >connected to your stripe. >+> > >+> > >+> the -h was my friend..... >+> I do not really understand why it was trying to use the whole slice for >+> mirroring?? But that might be my ignorance. > >Because those two providers share the same last sector, so they share >the same metadata and there is no information by default, which provider >is the right one (you should add '-h' option). >IMHO this is a better behaviour, because you can move your disk, copy it >to another disk via dd(1), etc. and it will be still recognized by gmirror. > > Thanx for the clarification, but more questions rise: What two providers are we talking about?? I specifically entered ad0s1h and ad2s1h..... On the first one it only takes the 'h'partition, but on the second on it prefers to take the whole disk. I fail to see where the exact difference is. I'm hearing you say: always you -h, there is little reason not to do so? --WjW From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 14:14:01 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F412E16A4CE for ; Sat, 25 Sep 2004 14:14:00 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9600B43D53 for ; Sat, 25 Sep 2004 14:14:00 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 54318ACAE6; Sat, 25 Sep 2004 16:13:59 +0200 (CEST) Date: Sat, 25 Sep 2004 16:13:59 +0200 From: Pawel Jakub Dawidek To: Willem Jan Withagen Message-ID: <20040925141359.GM9550@darkness.comp.waw.pl> References: <4153441A.9010303@withagen.nl> <20040924074925.GD9550@darkness.comp.waw.pl> <41556BE2.5060803@withagen.nl> <20040925134112.GL9550@darkness.comp.waw.pl> <41557A48.6090403@withagen.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MSd2ShuMixI0uVaZ" Content-Disposition: inline In-Reply-To: <41557A48.6090403@withagen.nl> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-geom@FreeBSD.org Subject: Re: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 14:14:01 -0000 --MSd2ShuMixI0uVaZ Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 25, 2004 at 04:01:44PM +0200, Willem Jan Withagen wrote: +> Thanx for the clarification, but more questions rise: +>=20 +> What two providers are we talking about?? +> I specifically entered ad0s1h and ad2s1h..... +> On the first one it only takes the 'h'partition, but on the second=20 +> on it prefers to take the whole disk. +> I fail to see where the exact difference is. It depends on GEOM which provider it gives me to taste first. In your scenario, ad2s1 and ad2s1h ends in the same place. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --MSd2ShuMixI0uVaZ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBVX0nForvXbEpPzQRAsx9AKCupTI2CgNer7WwWW/2EpXHc5k+zQCgvrYG D6IC7iExKqUKh+vxLuDJ8q4= =W0IB -----END PGP SIGNATURE----- --MSd2ShuMixI0uVaZ-- From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 14:52:38 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F09416A4CE for ; Sat, 25 Sep 2004 14:52:38 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2003B43D55 for ; Sat, 25 Sep 2004 14:52:38 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 3B09BACAFE; Sat, 25 Sep 2004 16:52:36 +0200 (CEST) Date: Sat, 25 Sep 2004 16:52:36 +0200 From: Pawel Jakub Dawidek To: Paul Mather Message-ID: <20040925145236.GO9550@darkness.comp.waw.pl> References: <1095993821.5665.124.camel@zappa.Chelsea-Ct.Org> <20040924075553.GE9550@darkness.comp.waw.pl> <1096062713.9306.119.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TRkqPRiqIDKgfg/F" Content-Disposition: inline In-Reply-To: <1096062713.9306.119.camel@zappa.Chelsea-Ct.Org> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-geom@freebsd.org Subject: Re: gstripe stripe size units? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 14:52:38 -0000 --TRkqPRiqIDKgfg/F Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 24, 2004 at 05:51:54PM -0400, Paul Mather wrote: +> > If gstripe is running in "fast" mode (kern.geom.stripe.fast=3D1), size= of +> > stripe could be small, because then, it still sends as large I/O reque= sts +> > as possible and reorganize the data in memory, but this method consumes +> > a lot of memory if you want it to be efficient. +>=20 +> One thing that puzzles me is that no matter how large I made the stripe +> size, I never got a kern.geom.stripe.fast_failed > 0. [...] It will be hard for raidtest tests, but it is really easy for UFS+softupdates and small stripe size. +> I'm puzzled because a stripe of 16 MB will not fit in 6553600 bytes, so +> surely fast_failed should be > 0 at stripes of 8 MB or greater in my +> tests for the above value of kern.geom.stripe.maxmem? This is a problem for small stripe sizes, not for big ones. Remember, that no metter how large stripe size is, maximum I/O request size is still 128kB, so for stripe size >=3D 64kB there is not even chance to fail - gstripe don't allocate memory if it is not needed, i.e. if we can split request on two pieces or even send it as one piece. +> Also, I don't know what the distribution of request sizes is in +> raidtest.data. The raidtest program operates over the raw device, and +> so may not necessarily behave in terms of issuing requests as, say, a +> UFS filesystem might. Of course. You should run some file systems tests as well. --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --TRkqPRiqIDKgfg/F Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFBVYY0ForvXbEpPzQRAqYcAJ9e5LXWgcpMI+P4usPwszlcfks7AwCeKehD PslUqk8dcrJ2ggUFGQNh6yE= =QiNi -----END PGP SIGNATURE----- --TRkqPRiqIDKgfg/F-- From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 16:06:34 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 07D2816A4CE; Sat, 25 Sep 2004 16:06:34 +0000 (GMT) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id A05DD43D5F; Sat, 25 Sep 2004 16:06:33 +0000 (GMT) (envelope-from paul@gromit.dlib.vt.edu) Received: from hawkwind.Chelsea-Ct.Org (pool-151-199-91-61.roa.east.verizon.net [151.199.91.61]) by gromit.dlib.vt.edu (8.13.1/8.13.1) with ESMTP id i8PG6ULp027819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sat, 25 Sep 2004 12:06:32 -0400 (EDT) (envelope-from paul@gromit.dlib.vt.edu) Received: from [192.168.1.25] (zappa [192.168.1.25])i8PG6OQE001993; Sat, 25 Sep 2004 12:06:24 -0400 (EDT) From: Paul Mather To: Pawel Jakub Dawidek In-Reply-To: <20040925134112.GL9550@darkness.comp.waw.pl> References: <4153441A.9010303@withagen.nl> <41556BE2.5060803@withagen.nl> <20040925134112.GL9550@darkness.comp.waw.pl> Content-Type: text/plain Message-Id: <1096128383.10839.20.camel@zappa.Chelsea-Ct.Org> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Sat, 25 Sep 2004 12:06:24 -0400 Content-Transfer-Encoding: 7bit cc: Willem Jan Withagen cc: freebsd-geom@freebsd.org Subject: Re: building a mirror in a partition X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 16:06:34 -0000 On Sat, 2004-09-25 at 09:41, Pawel Jakub Dawidek wrote: > On Sat, Sep 25, 2004 at 03:00:18PM +0200, Willem Jan Withagen wrote: > +> >Yes. It is because gstripe store metadata in the last provider's sector > +> >and if two providers ends in the same place - you have a problem. > +> >In that case, you should create you stripe by adding '-h' option, which > +> >will add provider's name to metadata, so only this provider will be > +> >connected to your stripe. > +> > > +> > > +> the -h was my friend..... > +> I do not really understand why it was trying to use the whole slice for > +> mirroring?? But that might be my ignorance. > > Because those two providers share the same last sector, so they share > the same metadata and there is no information by default, which provider > is the right one (you should add '-h' option). > IMHO this is a better behaviour, because you can move your disk, copy it > to another disk via dd(1), etc. and it will be still recognized by gmirror. Just to seek clarification, is the above strictly true? If I hard code, say, ad0 as a provider name and then move the disk so that it then becomes, say, ad4, won't geom_mirror get confused when trying to activate the mirror and there is no longer an ad0 provider in the system (or ad0 becomes something different entirely)? As a realistic scenario, I have a Dell system whose BIOS is not very good when it comes to boot flexibility: it will only let you boot the master device on the first ATA channel when booting ATA hard drives. So, if you have a two-drive mirror, one master and one slave on the same ATA channel (i.e., ad0 and ad1) and the master drive dies, then the only way to boot the system is physically to change the slave so it is now master. In other words, the ad1 drive then becomes ad0. If hard coding the raw providers, I think this would create problems for geom_mirror when rebooting such a switched setup, because there would no longer be an ad1. (That system currently uses a root-on-vinum setup, but because Vinum uses logical names for drives that are stored on the disk itself, it's not a problem to switch around ATA ids in that setup.) For that reason, I have adopted the policy of using geom_label to label my underlying providers and then add only the labelled devices to higher-level aggregations (such as mirrors and stripes). That way, the mirrors, stripes, etc. are using truly logical identifiers that are not "tied down" like hardware device names might be, and really are free to be moved about. (I think of the geom_label identifiers the same as the component identifiers used when creating RAID sets in RAIDframe on NetBSD, except geom_labels have a more general application.) Cheers, Paul. -- e-mail: paul@gromit.dlib.vt.edu "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 16:08:43 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A07216A4CE; Sat, 25 Sep 2004 16:08:43 +0000 (GMT) Received: from clueful.shagged.org (clueful.shagged.org [212.13.201.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D36243D2F; Sat, 25 Sep 2004 16:08:43 +0000 (GMT) (envelope-from chris@clueful.shagged.org) Received: from chris by clueful.shagged.org with local (Exim 4.40 (FreeBSD)) id 1CBF6C-0009pV-Pg; Sat, 25 Sep 2004 17:08:36 +0100 Date: Sat, 25 Sep 2004 17:08:36 +0100 From: Chris Elsworth To: Pawel Jakub Dawidek Message-ID: <20040925160836.GA37552@shagged.org> References: <54732.1096062623@critter.freebsd.dk> <1096063835.9306.130.camel@zappa.Chelsea-Ct.Org> <20040925002817.GA98852@shagged.org> <20040925131854.GA30301@shagged.org> <20040925133415.GJ9550@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040925133415.GJ9550@darkness.comp.waw.pl> User-Agent: Mutt/1.5.6i Sender: Chris Elsworth X-Shagged-MailScanner-Information: See www.mailscanner.info for information X-Shagged-MailScanner: Found to be clean X-MailScanner-From: chris@clueful.shagged.org cc: Paul Mather cc: Poul-Henning Kamp cc: freebsd-geom@freebsd.org Subject: Re: WARNING: Expected rawoffset 0, found 63 ? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 16:08:43 -0000 On Sat, Sep 25, 2004 at 03:34:15PM +0200, Pawel Jakub Dawidek wrote: > On Sat, Sep 25, 2004 at 02:18:54PM +0100, Chris Elsworth wrote: > +> Anyone know what's going on here.. ? > > Yes, gmirror ignores providers with 0 sector size. It has been fixed in > -CURRENT: > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/geom/mirror/g_mirror.c.diff?r1=1.26&r2=1.27 Excellent, that fixes my problem :) I can now boot of a gmirror device, hurrah :) I still have applied the root_mount delay patch. From what I can make out this is only necessary if I need to boot off a degraded mirror, so I should be able to remove that. I'm not really sure if this really has much bearing on anything, but my devices at the moment appear to be da1a, da1c, etc, when I'm configuring up the mirror. If nothing else, it appears that gstat doesn't recognise those so monitoring their activity is difficult. Would it be better to configure the mirror in a slice, so using something like "gmirror label -hv gm da1s1", or does it really make no odds at all? -- Chris From owner-freebsd-geom@FreeBSD.ORG Sat Sep 25 19:40:00 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E31C16A4CE for ; Sat, 25 Sep 2004 19:40:00 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B36B43D48 for ; Sat, 25 Sep 2004 19:39:59 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id i8PJduKQ001917 for ; Sat, 25 Sep 2004 21:39:56 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: geom@freebsd.org From: Poul-Henning Kamp Date: Sat, 25 Sep 2004 21:39:56 +0200 Message-ID: <1916.1096141196@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Subject: cheap Fibre-channel stuff on eBay X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Sep 2004 19:40:00 -0000 There was some talk about FC recently, and I just happened to notice that the kind of FC disk gadget I use is up on ebay again: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&category=22462&item=5722486918&rd=1 This is the disk-tray from a SUN blade1000, you'll need a powersupply (+5, +12, grab it from your computer), an 80mm fan for cooling and a cable to hook it to your FC controller. This is the cheapest way to get a couple of FC disks running I'm aware off. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.