Date: Tue, 7 Jul 2009 10:12:34 +0100 From: Anton Shterenlikht <mexas@bristol.ac.uk> To: ToyoRunner <toyorunner@gmail.com> Cc: freebsd-ia64@freebsd.org Subject: gmirror per partition on ia64 - handbook incorrect Message-ID: <20090707091234.GA39999@mech-cluster238.men.bris.ac.uk> In-Reply-To: <4A5296F2.80800@gmail.com> References: <20090706201733.GA17827@mech-cluster238.men.bris.ac.uk> <20090706205125.GA18002@mech-cluster238.men.bris.ac.uk> <20090706222229.GA23689@mech-cluster238.men.bris.ac.uk> <20090706224435.GA23791@mech-cluster238.men.bris.ac.uk> <4A5283BC.2000107@gmail.com> <20090706232552.GA23964@mech-cluster238.men.bris.ac.uk> <4A5296F2.80800@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jul 06, 2009 at 06:29:38PM -0600, ToyoRunner wrote: > Anton Shterenlikht wrote: > > On Mon, Jul 06, 2009 at 05:07:40PM -0600, ToyoRunner wrote: > > > >> BTW, did you have any luck with gmirror? > >> > > > > # gpart show > > => 34 143374671 da0 GPT (68G) > > 34 819200 1 efi (400M) > > 819234 1048576 2 freebsd-ufs (512M) > > 1867810 4194304 3 freebsd-swap (2.0G) > > 6062114 2097152 4 freebsd-ufs (1.0G) > > 8159266 2097152 5 freebsd-ufs (1.0G) > > 10256418 133118287 6 freebsd-ufs (63G) > > > > => 34 143374671 da1 GPT (68G) > > 34 819200 1 efi (400M) > > 819234 1048576 2 freebsd-ufs (512M) > > 1867810 4194304 3 freebsd-swap (2.0G) > > 6062114 2097152 4 freebsd-ufs (1.0G) > > 8159266 2097152 5 freebsd-ufs (1.0G) > > 10256418 133118287 6 freebsd-ufs (63G) > > > > # > > > > > > # gmirror status > > Name Status Components > > mirror/efi COMPLETE da0p1 > > da1p1 > > mirror/root COMPLETE da1p2 > > da0p2 > > mirror/swap COMPLETE da1p3 > > da0p3 > > mirror/usr DEGRADED da0p6 > > da1p6 (25%) > > mirror/tmp COMPLETE da0p5 > > da1p5 > > mirror/var COMPLETE da0p4 > > da1p4 > > # > > > > why is rebuilding such a slow process? there are less than 300MB of > > data on /usr as of fresh minimal install. However the partition is > > 63GB. But does the partition size matter? I'd imagine only data need > > to be rebuild. I've got to 25% after about 10 minutes. The data > > transfer rate for these disks supposed to be 300 MB/s according to > > dmesg: > > > > da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit) > > > > I'm surely missing something simple. > > > > > Actually, I believe the whole disks need to be indexed regardless. Who > knows which zero block is the zero block you need. > > Could you post the steps you took to get your mirror operating correctly > to the list? I would bow before you. See below, sorry if it's too much detail. Some steps are related to install on my particular box, i.e. not generic ia64 requirements. It seems quite complicated, requiring two reboots. Please see if you can shorten or simplify the procedure. I'd like to write a section on RAID-1 on ia64 for the handbook based on this, because existing advice is incorrect when it comes to ia64. So, I welcome any suggestions on these quidelines. This example is based on setting up gmirror on rx2600 with two directly attached scsi disks, da0 and da1. 1. preliminary: read man pages for gpart(8), gmirror(8), geom(4), geom(8), dump(8), restore(8). use # gpart show # gmirror status # gmirror list between the steps to see partitions and mirror state. 2. fresh FBSD install on da0, as default, but: a) /efi is 400MB b) set up at least one network interface c) create at least one user and add him to group wheel (b) and (c) are necessary because /etc/ttys by default does not open getty on console via MP, so ssh connection is required. 3. (optional, possibly for my box only): add hw.ata.atapi_dma=0 to /boot/device.hints 4. (optional) enable extra debug messages: # sysctl kern.geom.debugflags=17 5. partition a spare disk, da1, exactly as the boot disk: # gpart create -s gpt da1 # gpart add ... so that the results look like: # gpart show => 34 143374671 da0 GPT (68G) 34 819200 1 efi (400M) 819234 1048576 2 freebsd-ufs (512M) 1867810 4194304 3 freebsd-swap (2.0G) 6062114 2097152 4 freebsd-ufs (1.0G) 8159266 2097152 5 freebsd-ufs (1.0G) 10256418 133118287 6 freebsd-ufs (63G) => 34 143374671 da1 GPT (68G) 34 819200 1 efi (400M) 819234 1048576 2 freebsd-ufs (512M) 1867810 4194304 3 freebsd-swap (2.0G) 6062114 2097152 4 freebsd-ufs (1.0G) 8159266 2097152 5 freebsd-ufs (1.0G) 10256418 133118287 6 freebsd-ufs (63G) # 6. load gmirror kernel module # gmirror load 7. create mirror for EFI partition: a) unmount /efi because GEOM manipulations can be performed only on unmounted, not in use, partition: # umount /efi b) create efi mirror on the boot(!) disk: # gmirror label -vb round-robin efi da0p1 c) add spare disk's efi partition to the mirror: # gmirror insert efi da1p1 rebuilding should start, check progress with # gmirror status after rebuilding is complete you should see: # gmirror status efi Name Status Components mirror/efi COMPLETE da0p1 da1p1 # d) mount mirror efi: # mount -t msdosfs /dev/mirror/efi /efi # df Filesystem 1K-blocks Used Avail Capacity Mounted on ... /dev/mirror/efi 409504 163264 246240 40% /efi # 8. create mirror for root (/) partition. This involves extra steps since / cannot be unmounted. a) create mirror on the spare disk: # gmirror label -vb round-robin root da1p2 b) create ufs filesystem on the mirror: # newfs /dev/mirror/root c) mount root mirror temporarily, say under /mnt: # mount /dev/mirror/root /mnt d) now copy / onto /mnt (actually onto the root mirror, /dev/mirror/root). Use a combination of dump(8) and restore(8). No other copying tool will do it right: # cd /mnt # dump 0aLf - / | restore rf - (check the man pages for more details on the options) 9. update fstab on the mirror(!) Edit /mnt/etc/fstab and change da0p1 into mirror/efi and da0p2 into mirror/root. 10. update /boot/loader.conf a) put this line at the very beginning: geom_mirror_load="YES" f) In this line: vfs.root.mountfrom="ufs:/dev/da0p2" replace da0p2 with mirror/root, so that it is: vfs.root.mountfrom="ufs:/dev/mirror/root" 11. reboot into single user(!) mode on shutdown you should see root and efi mirrors destroyed. 12. on boot you should see gmirror loaded, /dev/mirror/root and /dev/mirror/efi started and /dev/mirror/root is used as a boot device, 13. now that you have booted from /dev/da1p2, da0p2 is not mounted, so it can be inserted into root mirror: # gmirror insert root da0p2 14. create mirrors for all other partitions of da0, which are now not mounted: # gmirror label -vb round-robin swap da0p3 # gmirror label -vb round-robin var da0p4 # gmirror label -vb round-robin tmp da0p5 # gmirror label -vb round-robin usr da0p6 15. edit /etc/fstab and change da0p3 into mirrior/swap da0p4 into mirrior/var da0p5 into mirrior/tmp da0p6 into mirrior/usr 16. reboot 17. on startup add spare disks to mirrors: # gmirror insert swap da1p3 # gmirror insert var da1p4 # gmirror insert tmp da1p5 # gmirror insert usr da1p6 18. done! -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 928 8233 Fax: +44 (0)117 929 4423
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090707091234.GA39999>