Date: Fri, 14 Aug 2009 11:00:14 GMT From: Anton Shterenlikht <mexas@bristol.ac.uk> To: freebsd-doc@FreeBSD.org Subject: RE: docs/136712: [handbook] [patch] draft new section on gmirror per partition Message-ID: <200908141100.n7EB0E7o017758@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR docs/136712; it has been noted by GNATS.
From: Anton Shterenlikht <mexas@bristol.ac.uk>
To: bug-followup@FreeBSD.org
Cc:
Subject: RE: docs/136712: [handbook] [patch] draft new section on gmirror
per partition
Date: Fri, 14 Aug 2009 11:58:53 +0100
below is the proposed patch
############
--- /usr/build/doc/en_US.ISO8859-1/books/handbook/geom/chapter.sgml 2009-05-06 15:12:13.000000000 +0100
+++ ./chapter.sgml 2009-08-14 11:31:05.000000000 +0100
@@ -220,10 +220,10 @@
<screen>&prompt.root; <userinput>echo 'geom_stripe_load="YES"' >> /boot/loader.conf</userinput></screen>
</sect1>
-
<sect1 id="GEOM-mirror">
- <title>RAID1 - Mirroring</title>
+ <title>RAID1 - Mirroring</title>
+
<indexterm>
<primary>GEOM</primary>
</indexterm>
@@ -375,6 +375,295 @@
</sect2>
<sect2>
+
+ <sect2info>
+ <authorgroup>
+ <author>
+ <firstname>Anton</firstname>
+ <surname>Shterenlikht</surname>
+ <contrib>With contributions from </contrib>
+ </author>
+ </authorgroup>
+ </sect2info>
+
+
+ <title>Mirroring individual partitions</title>
+
+ <note>
+ <para>in some partitioning schemes mirroring a whole disk
+ is not possible at all. The most notable
+ example is <acronym role="GUID Partition Table">GPT</acronym>,
+ used in ia64 architecture. Secondary <acronym>GPT</acronym>
+ header is stored in the last sector on disk. Since
+ <command>gmirror</command> writes its data in the last sector,
+ it destroys secondary <acronym>GPT</acronym>.
+ A very good introduction to
+ <ulink url="http://en.wikipedia.org/wiki/GUID_Partition_Table">GPT
+ (GUID Partition Table)</ulink> can be found on Wikipedia.</para>
+ </note>
+
+ <para>Mirroring individual partitions is a more
+ flexible way of achieving <acronym>RAID1</acronym> compared to
+ mirroring a whole disk. Below is a step by step guide.</para>
+
+ <orderedlist>
+ <listitem>
+ <para>Check partition of the boot disk using &man.gpart.8;:</para>
+<screen>&prompt.root; <userinput>gpart show</userinput>
+=> 34 143374671 da0 GPT (68G)
+ 34 819200 1 efi (400M)
+ 819234 1048576 2 freebsd-ufs (512M)
+ 1867810 4194304 3 freebsd-swap (2.0G)
+ 6062114 2097152 4 freebsd-ufs (1.0G)
+ 8159266 2097152 5 freebsd-ufs (1.0G)
+ 10256418 133118287 6 freebsd-ufs (63G)</screen>
+ <note>
+ <para>this example is taken from ia64 architecture,
+ which includes EFI boot partition. Depending on the
+ architecture of your system you might or might not have
+ this partition present.</para>
+ </note>
+ </listitem>
+
+ <listitem>
+ <para>Partition a spare disk, <devicename>da1</devicename>,
+ exactly as the boot disk. First create a partitioning
+ scheme on <devicename>da1</devicename>, identical to
+ that on <devicename>da0</devicename>, in this example
+ this is <acronym>GPT</acronym>:</para>
+<screen>&prompt.root; <userinput>gpart create -s gpt da1</userinput></screen>
+ <para>Add partitions exactly as on
+ <devicename>da0</devicename>:</para>
+<screen>&prompt.root; <userinput>gpart add -b 34 -s 819200 -t freebsd-efi da1</userinput>
+&prompt.root; <userinput>gpart add -b 819234 -s 1048576 -t freebsd-ufs da1</userinput>
+&prompt.root; <userinput>gpart add -b 1867810 -s 4194304 -t freebsd-swap da1</userinput>
+&prompt.root; <userinput>gpart add -b 6062114 -s 2097152 -t freebsd-ufs da1</userinput>
+&prompt.root; <userinput>gpart add -b 8159266 -s 2097152 -t freebsd-ufs da1</userinput>
+&prompt.root; <userinput>gpart add -b 10256418 -s 133118287 -t freebsd-ufs da1</userinput></screen>
+ <note>
+ <para><literal>-b</literal> sets the starting block of
+ a new partition, <literal>-s</literal> specifies partition
+ size, in blocks, and <literal>-t</literal> is partition
+ type. On ia64 systems the boot partition is of EFI type,
+ hence the first partition added with <command>gpart</command>
+ is <literal>freebsd-efi</literal>. Likewise,
+ <literal>freebsd-swap</literal> is used to add a
+ <literal>swap</literal> partition.
+ </para>
+ </note>
+ <tip>
+ <para>do <literal>gpart show</literal> before each
+ <literal>gpart add</literal>. This will tell you the
+ starting block.</para>
+ </tip>
+ <para>When you are done both disks should be partitioned
+ identically:</para>
+<screen>&prompt.root; <userinput>gpart show</userinput>
+=> 34 143374671 da0 GPT (68G)
+ 34 819200 1 efi (400M)
+ 819234 1048576 2 freebsd-ufs (512M)
+ 1867810 4194304 3 freebsd-swap (2.0G)
+ 6062114 2097152 4 freebsd-ufs (1.0G)
+ 8159266 2097152 5 freebsd-ufs (1.0G)
+ 10256418 133118287 6 freebsd-ufs (63G)
+
+=> 34 143374671 da1 GPT (68G)
+ 34 819200 1 efi (400M)
+ 819234 1048576 2 freebsd-ufs (512M)
+ 1867810 4194304 3 freebsd-swap (2.0G)
+ 6062114 2097152 4 freebsd-ufs (1.0G)
+ 8159266 2097152 5 freebsd-ufs (1.0G)
+ 10256418 133118287 6 freebsd-ufs (63G)</screen>
+ </listitem>
+
+ <listitem>
+ <para>load gmirror kernel module:</para>
+<screen>&prompt.root; <userinput>gmirror load</userinput></screen>
+ </listitem>
+
+ <listitem>
+ <para>Create mirror for EFI partition.<para>
+ <note>
+ <para>If you do not have an EFI partition, skip this step.</para>
+ </note>
+ <para>Unmount <filename>/efi</filename> because GEOM
+ manipulations can be performed only on unmounted,
+ not in use, partition:</para>
+<screen>&prompt.root; <userinput>umount /efi</userinput></screen>
+ <para>create EFI mirror on the <emphasis>boot</emphasis> disk,
+ <devicename>da0</devicename>, in our example:
+<screen>&prompt.root; <userinput>gmirror label -vb round-robin efi da0p1</userinput></screen>
+ <para>This would create <filename>/dev/mirror/efi</filename>
+ device.<para>
+ <para>Add EFI partition of the spare disk,
+ <devicename>da1</devicename> to the mirror:</para>
+<screen>&prompt.root; <userinput>gmirror insert efi da1p1</userinput></screen>
+ <para>The EFI partition in this example is only 400MB, so it
+ rebuilds quickly:</para>
+<screen>&prompt.root; <userinput>gmirror status</userinput>
+ Name Status Components
+mirror/efi COMPLETE da0p1
+ da1p1</screen>
+ <para>mount EFI mirror and check:</para>
+<screen>&prompt.root; <userinput>mount -t msdosfs /dev/mirror/efi /efi</userinput>
+&prompt.root; <userinput>df</userinput>
+Filesystem 512-blocks Used Avail Capacity Mounted on
+...
+/dev/mirror/efi 819008 141728 677280 17% /efi</screen>
+ </listitem>
+
+ <listitem>
+ <para>Create mirror for root (<filename>/</filename>) partition.
+ This involves extra steps since <filename>/</filename>
+ cannot be unmounted.</para>
+ <para>Create mirror on the spare disk,
+ <devicename>da1</devicename>:</para>
+<screen>&prompt.root; <userinput>gmirror label -vb round-robin root da1p2</userinput></screen>
+ <para>Create ufs filesystem on this mirror:</para>
+<screen>&prompt.root; <userinput>newfs /dev/mirror/root</userinput></screen>
+ <para>Mount root mirror temporarily, say under
+ <filename>/mnt</filename>:</para>
+<screen>&prompt.root; <userinput>mount /dev/mirror/root /mnt</userinput></screen>
+ <para>Copy <filename>/</filename> onto <filename>/mnt</filename>
+ (which is the root mirror,
+ <filename>/dev/mirror/root</filename>), using a
+ combination of &man.dump.8; and &man.restore.8;:</para>
+<screen>&prompt.root; <userinput>cd /mnt</userinput>
+&prompt.root; <userinput>dump 0aLf - / | restore rf -</userinput></screen>
+ <warning>
+ <para>&man.dump.8; is the only safe way to copy root
+ partition. Any other copying tool is not guaranteed
+ to do it right.</para>
+ </warning>
+ </listitem>
+
+ <listitem>
+ <para>Update <filename>fstab</filename> on the
+ <emphasis>mirror</emphasis>. Edit
+ <filename>/mnt/etc/fstab</filename> and change
+ <devicename>da0p1</devicename> into
+ <devicename>mirror/efi</devicename> and
+ <devicename>da0p2</devicename> into
+ <devicename>mirror/root</devicename>:</para>
+<screen>&prompt.root; <userinput>cat /mnt/etc/fstab</userinput>
+# Device Mountpoint FStype Options Dump Pass#
+/dev/da0p3 none swap sw 0 0
+/dev/mirror/root / ufs rw 1 1
+/dev/mirror/efi /efi msdosfs rw 0 0
+/dev/da0p5 /tmp ufs rw 2 2
+/dev/da0p6 /usr ufs rw 2 2
+/dev/da0p4 /var ufs rw 2 2
+/dev/acd0 /cdrom cd9660 ro,noauto 0 0</screen>
+ </listitem>
+
+ <listitem>
+ <para>Enable loading of the <filename>geom_mirror.ko</filename>
+ kernel module during system initialization. For this do
+ either:</para>
+<screen>&prompt.root; <userinput>echo 'geom_mirror_load="YES"' >> /boot/loader.conf</userinput></screen>
+ <para>or add
+ <programlisting>options GEOM_MIRROR</programlisting>
+ to your kernel configuration file.</para>
+ </listitem>
+
+ <listitem>
+ <para>Change the root device in
+ <filename>/boot/loader.conf</filename>. The root
+ device is specified with <literal>vfs.root.mountfrom</literal>
+ option. It should point to the root mirror. For example, change
+<programlisting>vfs.root.mountfrom="ufs:/dev/da0p2"</programlisting>
+ into
+<programlisting>vfs.root.mountfrom="ufs:/dev/mirror/root"</programlisting>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>Reboot into <emphasis>single user</emphasis> mode.</para>
+ <tip>
+ <para>On ia64 type <literal>boot -s</literal> on the
+ boot prompt.</para>
+ </tip>
+ </listitem>
+
+ <listitem>
+ <para>At boot you should see <literal>gmirror</literal> loaded,
+ and then these lines:</para>
+<screen>GEOM_MIRROR: Device mirror/efi launched (2/2).
+GEOM_MIRROR: Device mirror/root launched (1/1).
+Trying to mount root from ufs:/dev/mirror/root</screen>
+ </listitem>
+
+ <listitem>
+ <para>Now that <devicename>da0p2</devicename> is not
+ mounted, it can be inserted into root mirror:
+<screen>&prompt.root; <userinput>gmirror insert root da0p2</userinput></screen>
+ and after mirror rebuild is complete you should see:
+<screen>&prompt.root; <userinput>gmirror status</userinput>
+ Name Status Components
+ mirror/efi COMPLETE da0p1
+ da1p1
+mirror/root COMPLETE da1p2
+ da0p2</screen>
+ <para>
+ </listitem>
+
+ <listitem>
+ <para>Create mirrors for all other partitions of
+ <devicename>da0</devicename>, which are now not mounted:</para>
+<screen>&prompt.root; <userinput>gmirror label -vb round-robin swap da0p3</userinput>
+&prompt.root; <userinput>gmirror label -vb round-robin var da0p4</userinput>
+&prompt.root; <userinput>gmirror label -vb round-robin tmp da0p5</userinput>
+&prompt.root; <userinput>gmirror label -vb round-robin usr da0p6</userinput></screen>
+ </listitem>
+
+ <listitem>
+ <para>Edit <filename>/etc/fstab</filename> and change each remaining
+ <devicename>da0</devicename> partition into its mirror:</para>
+<screen>&prompt.root; <userinput>cat /etc/fstab</userinput>
+# Device Mountpoint FStype Options Dump Pass#
+/dev/mirror/swap none swap sw 0 0
+/dev/mirror/root / ufs rw 1 1
+/dev/mirror/efi /efi msdosfs rw 0 0
+/dev/mirror/tmp /tmp ufs rw 2 2
+/dev/mirror/usr /usr ufs rw 2 2
+/dev/mirror/var /var ufs rw 2 2
+/dev/acd0 /cdrom cd9660 ro,noauto 0 0</screen>
+ </listitem>
+
+ <listitem>
+ <para>Reboot.</para>
+ </listitem>
+
+ <listitem>
+ <para>Add remaining <devicename>da1</devicename> partitions
+ to mirrors:</para>
+<screen>&prompt.root; <userinput>gmirror insert swap da1p3</userinput>
+&prompt.root; <userinput>gmirror insert var da1p4</userinput>
+&prompt.root; <userinput>gmirror insert tmp da1p5</userinput>
+&prompt.root; <userinput>gmirror insert usr da1p6</userinput></screen>
+ </listitem>
+
+ <listitem>
+ <para>Done!:</para>
+<screen>&prompt.root; <userinput>gmirror status</userinput>
+ Name Status Components
+ mirror/efi COMPLETE da0p1
+ da1p1
+mirror/root COMPLETE da0p2
+ da1p2
+mirror/swap COMPLETE da0p3
+ da1p3
+ mirror/var COMPLETE da0p4
+ da1p4
+ mirror/tmp COMPLETE da0p5
+ da1p5
+ mirror/usr COMPLETE da0p6
+ da1p6</screen>
+ </listitem>
+ </orderedlist>
+ </sect2>
+
+ <sect2>
<title>Troubleshooting</title>
<sect3>
@@ -382,9 +671,9 @@
<para>If the system boots up to a prompt similar to:</para>
- <programlisting>ffs_mountroot: can't find rootvp
+ <screen>ffs_mountroot: can't find rootvp
Root mount failed: 6
-mountroot></programlisting>
+mountroot></screen>
<para>Reboot the machine using the power or reset button. At
the boot menu, select option six (6). This will drop the
@@ -404,6 +693,45 @@
<para>in the kernel configuration file, rebuild and reinstall.
That should remedy this issue.</para>
</sect3>
+
+ <sect3>
+ <title>System cannot find the boot device</title>
+
+ <para>In that case the system boot process would stop with a
+ prompt similar to:</para>
+
+<screen><![ CDATA [Loader variables:
+vfs.root.mountfrom=
+vfs.root.mountfrom.options=
+
+Manual root filesystem specification:
+ <fstype>:<device> Mount <device> using filesystem <fstype>
+ eg. ufs:/dev/da0s1a
+ eg. cd9660:/dev/acd0
+ This is equivalent to: mount -t cd9660 /dev/acd0 /
+
+ ? List valid disk boot devices
+ <empty line> Abort manual input
+
+mountroot>]]></screen>
+
+ <para>You would get this message most probably if you forget
+ to add the correct boot device in
+ <filename>/boot/loader.conf</filename>. The hints given by the
+ system are clear. If your root device is
+ <devicename>/dev/mirror/root</devicename>, and the filesystem is
+ <literal>ufs</literal>, enter:</para>
+
+<screen>mountroot> <userinput>ufs:/dev/mirror/root</userinput></screen>
+
+ <para>The boot process would then continue:</para>
+
+<screen>Trying to mount root from ufs:/dev/mirror/root</screen>
+
+ <para>When the system is up remember to edit your root device
+ in <filename>/boot/loader.conf</filename>.</para>
+
+ </sect3>
</sect2>
<sect2>
--
Anton Shterenlikht
Room 2.6, Queen's Building
Mech Eng Dept
Bristol University
University Walk, Bristol BS8 1TR, UK
Tel: +44 (0)117 928 8233
Fax: +44 (0)117 929 4423
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200908141100.n7EB0E7o017758>