Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 9 Jan 2011 23:58:18 -0500
From:      Carl Chave <carl@chave.us>
To:        FreeBSD Questions <freebsd-questions@freebsd.org>
Subject:   ZFS + GPT with root on memory stick and mirrored SATA drives
Message-ID:  <AANLkTi=veX-%2BS5sPXJXAHzcgTOBqsg-m8sf23LO%2B9b3_@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
Posting the below for input.  The bulk of this is from a guide that
Morgan Wesstr=F6m posted to this list.  Some of it is taken from the
root on ZFS wiki entries on freebsd.org.  Some from a pjd post here:
http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-conf=
iguration/

And then there's this that Svein Skogen posted to the list:

I usually (today) set up something similar. I sysinstall FreeBSD onto a
CF card with the "one-big-root" method, then create a zpool (on
spinning-metal-storage) where I create the usr, tmp, var fs'es, tar|tar
the originals over and fix the mountpoint info on the zfs'es. Then I add
swap on a zvol (since I don't know how to properly use a kernel dump, I
don't need swap to store it).

I'm setting up a new home server and I always agonize over
partitioning.  So the steps below install the base system with zfs
root on a usb stick and /tmp /usr /var and swap on mirrored sata
drives.
I've tested these steps and everything works but before I press on
with actually configuring and using the server, does anybody have any
input on whether I should or shouldn't do it this way?  ZFS best
practices suggests that having elements of the root filesystem on
different pools is a bad idea.  So that might be strike 1.

Memory Stick
------------
/
/bin
/boot
/dev
/etc
/lib
/libexec
/media
/mnt
/proc
/rescue
/root
/sbin
/sys --> /usr/src/sys

Hard disk zpool
---------------
/tmp
/usr
/var
swap on zvol

Separate zfs datasets
---------------------
/tmp
/usr
/usr/home
/usr/local
/usr/obj
/usr/ports
/usr/ports/distfiles
/usr/ports/packages
/usr/src
/var
/var/log
/var/audit
/var/tmp

Install Procedure (Mostly by Morgan Wesstr=F6m)
---------------------------------------------------------------------
Select your country and keyboard layout.

Enter the Fixit environment and use the live filesystem on your DVD.

Your usb memory stick will most likely be da0 but you can (and should)
check it with "camcontrol devlist" before you continue.

Create a new GPT partitioning scheme:
 # gpart create -s gpt da0

Create a 64KiB partition for the zfs bootcode starting at LBA 1920:
 # gpart add -b 1920 -s 128 -t freebsd-boot da0

Create a zfs partition spanning the remainder of the usb memory stick
and give it a label we can refer to:
 # gpart add -t freebsd-zfs -l FreeBSDonUSB da0

(The starting LBA for the first partition is there to align the
partitions to the flash memory's erase block size. This is
particularly important for the main zfs partition. The main partition
above will start at exactly 1MiB (LBA 2048) which will align it to any
erase block size used today. This alignment is also of great
importance if you use this guide to install FreeBSD to one of the
newer harddrives using 4096 byte sectors.)

Install the protective MBR to LBA 0 and the zfs bootcode to the first parti=
tion:
 # gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 da0

Create /boot/zfs (for zpool.cache) and load the zfs kernel modules:
 # mkdir /boot/zfs
 # kldload /dist/boot/kernel/opensolaris.ko
 # kldload /dist/boot/kernel/zfs.ko

Create a zfs pool and set its bootfs property:
 # zpool create zrootusb /dev/gpt/FreeBSDonUSB
 # zpool set bootfs=3Dzrootusb zrootusb

Switch to fletcher4 checksums and turn off access time modifications:
 # zfs set checksum=3Dfletcher4 zrootusb
 # zfs set atime=3Doff zrootusb

Create zfs mirrored data pool on SATA disks
 # zpool create zdata mirror /dev/ad4 /dev/ad6
 # zfs set canmount=3Doff zdata
 # zfs set mountpoint=3D/zrootusb zdata
 # zfs set checksum=3Dfletcher4 zdata
 # zfs create zdata/tmp
 # zfs create zdata/usr
 # zfs create zdata/usr/home
 # zfs create zdata/usr/local
 # zfs create zdata/usr/obj
 # zfs create zdata/usr/ports
 # zfs create zdata/usr/ports/distfiles
 # zfs create zdata/usr/ports/packages
 # zfs create zdata/usr/src
 # zfs create zdata/var
 # zfs create zdata/var/log
 # zfs create zdata/var/audit
 # zfs create zdata/var/tmp

Create swap zvol on zdata pool
 # zfs create -V 5G zdata/swap
 # zfs set org.freebsd:swap=3Don zdata/swap
 # zfs set checksum=3Doff zdata/swap

Extract at a minimum, base and the generic kernel:
 # cd /dist/8.1-RELEASE/base
 # DESTDIR=3D/zrootusb ./install.sh
 # cd ../kernels
 # DESTDIR=3D/zrootusb ./install.sh generic

Delete the empty, default kernel directory and move the generic kernel
into its place:
 # rmdir /zrootusb/boot/kernel
 # mv /zrootusb/boot/GENERIC /zrootusb/boot/kernel

Make sure the zfs modules are loaded at boot:
 # cat > /zrootusb/boot/loader.conf
   zfs_load=3D"YES"
   vfs.root.mountfrom=3D"zfs:zrootusb"
   kern.cam.boot_delay=3D10000
  ^d

Create /etc/rc.conf. Adjust and add to your own needs:
 # cat > /zrootusb/etc/rc.conf
   hostname=3D"sodserve"
   sshd_enable=3D"YES"
   zfs_enable=3D"YES"
   ^d

Setup your time zone:
 # cp /zrootusb/usr/share/zoneinfo/EST5EDT /zrootusb/etc/localtime

Create an empty fstab to avoid startup warnings:
 # touch /zrootusb/etc/fstab

Set the root password in the new environment:
 # cd /
 # chroot /zrootusb /bin/sh
 # passwd root
 # exit

Copy zpool.cache:
 # cp /boot/zfs/zpool.cache /zrootusb/boot/zfs

Unmount filesystems and set mountpoints:
 # zfs unmount -a
 # zfs set mountpoint=3Dlegacy zrootusb
 # zfs set mountpoint=3D/ zdata


Exit SYSINSTALL and reboot. You now have a fully functional and
bootable FreeBSD installation on your usb memory stick.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=veX-%2BS5sPXJXAHzcgTOBqsg-m8sf23LO%2B9b3_>