Date: Mon, 10 Jan 2011 09:31:24 +0000 From: krad <kraduk@gmail.com> To: Carl Chave <carl@chave.us> Cc: FreeBSD Questions <freebsd-questions@freebsd.org> Subject: Re: ZFS + GPT with root on memory stick and mirrored SATA drives Message-ID: <AANLkTikcMscZHaC7DDR5PsZrUG4JtPtjwmkSd=MySfOi@mail.gmail.com> In-Reply-To: <AANLkTi=veX-%2BS5sPXJXAHzcgTOBqsg-m8sf23LO%2B9b3_@mail.gmail.com> References: <AANLkTi=veX-%2BS5sPXJXAHzcgTOBqsg-m8sf23LO%2B9b3_@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10 January 2011 04:58, Carl Chave <carl@chave.us> wrote: > Posting the below for input. The bulk of this is from a guide that > Morgan Wesstr=F6m posted to this list. Some of it is taken from the > root on ZFS wiki entries on freebsd.org. Some from a pjd post here: > > http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-co= nfiguration/ > > And then there's this that Svein Skogen posted to the list: > > I usually (today) set up something similar. I sysinstall FreeBSD onto a > CF card with the "one-big-root" method, then create a zpool (on > spinning-metal-storage) where I create the usr, tmp, var fs'es, tar|tar > the originals over and fix the mountpoint info on the zfs'es. Then I add > swap on a zvol (since I don't know how to properly use a kernel dump, I > don't need swap to store it). > > I'm setting up a new home server and I always agonize over > partitioning. So the steps below install the base system with zfs > root on a usb stick and /tmp /usr /var and swap on mirrored sata > drives. > I've tested these steps and everything works but before I press on > with actually configuring and using the server, does anybody have any > input on whether I should or shouldn't do it this way? ZFS best > practices suggests that having elements of the root filesystem on > different pools is a bad idea. So that might be strike 1. > > Memory Stick > ------------ > / > /bin > /boot > /dev > /etc > /lib > /libexec > /media > /mnt > /proc > /rescue > /root > /sbin > /sys --> /usr/src/sys > > Hard disk zpool > --------------- > /tmp > /usr > /var > swap on zvol > > Separate zfs datasets > --------------------- > /tmp > /usr > /usr/home > /usr/local > /usr/obj > /usr/ports > /usr/ports/distfiles > /usr/ports/packages > /usr/src > /var > /var/log > /var/audit > /var/tmp > > Install Procedure (Mostly by Morgan Wesstr=F6m) > --------------------------------------------------------------------- > Select your country and keyboard layout. > > Enter the Fixit environment and use the live filesystem on your DVD. > > Your usb memory stick will most likely be da0 but you can (and should) > check it with "camcontrol devlist" before you continue. > > Create a new GPT partitioning scheme: > # gpart create -s gpt da0 > > Create a 64KiB partition for the zfs bootcode starting at LBA 1920: > # gpart add -b 1920 -s 128 -t freebsd-boot da0 > > Create a zfs partition spanning the remainder of the usb memory stick > and give it a label we can refer to: > # gpart add -t freebsd-zfs -l FreeBSDonUSB da0 > > (The starting LBA for the first partition is there to align the > partitions to the flash memory's erase block size. This is > particularly important for the main zfs partition. The main partition > above will start at exactly 1MiB (LBA 2048) which will align it to any > erase block size used today. This alignment is also of great > importance if you use this guide to install FreeBSD to one of the > newer harddrives using 4096 byte sectors.) > > Install the protective MBR to LBA 0 and the zfs bootcode to the first > partition: > # gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 da0 > > Create /boot/zfs (for zpool.cache) and load the zfs kernel modules: > # mkdir /boot/zfs > # kldload /dist/boot/kernel/opensolaris.ko > # kldload /dist/boot/kernel/zfs.ko > > Create a zfs pool and set its bootfs property: > # zpool create zrootusb /dev/gpt/FreeBSDonUSB > # zpool set bootfs=3Dzrootusb zrootusb > > Switch to fletcher4 checksums and turn off access time modifications: > # zfs set checksum=3Dfletcher4 zrootusb > # zfs set atime=3Doff zrootusb > > Create zfs mirrored data pool on SATA disks > # zpool create zdata mirror /dev/ad4 /dev/ad6 > # zfs set canmount=3Doff zdata > # zfs set mountpoint=3D/zrootusb zdata > # zfs set checksum=3Dfletcher4 zdata > # zfs create zdata/tmp > # zfs create zdata/usr > # zfs create zdata/usr/home > # zfs create zdata/usr/local > # zfs create zdata/usr/obj > # zfs create zdata/usr/ports > # zfs create zdata/usr/ports/distfiles > # zfs create zdata/usr/ports/packages > # zfs create zdata/usr/src > # zfs create zdata/var > # zfs create zdata/var/log > # zfs create zdata/var/audit > # zfs create zdata/var/tmp > > Create swap zvol on zdata pool > # zfs create -V 5G zdata/swap > # zfs set org.freebsd:swap=3Don zdata/swap > # zfs set checksum=3Doff zdata/swap > > Extract at a minimum, base and the generic kernel: > # cd /dist/8.1-RELEASE/base > # DESTDIR=3D/zrootusb ./install.sh > # cd ../kernels > # DESTDIR=3D/zrootusb ./install.sh generic > > Delete the empty, default kernel directory and move the generic kernel > into its place: > # rmdir /zrootusb/boot/kernel > # mv /zrootusb/boot/GENERIC /zrootusb/boot/kernel > > Make sure the zfs modules are loaded at boot: > # cat > /zrootusb/boot/loader.conf > zfs_load=3D"YES" > vfs.root.mountfrom=3D"zfs:zrootusb" > kern.cam.boot_delay=3D10000 > ^d > > Create /etc/rc.conf. Adjust and add to your own needs: > # cat > /zrootusb/etc/rc.conf > hostname=3D"sodserve" > sshd_enable=3D"YES" > zfs_enable=3D"YES" > ^d > > Setup your time zone: > # cp /zrootusb/usr/share/zoneinfo/EST5EDT /zrootusb/etc/localtime > > Create an empty fstab to avoid startup warnings: > # touch /zrootusb/etc/fstab > > Set the root password in the new environment: > # cd / > # chroot /zrootusb /bin/sh > # passwd root > # exit > > Copy zpool.cache: > # cp /boot/zfs/zpool.cache /zrootusb/boot/zfs > > Unmount filesystems and set mountpoints: > # zfs unmount -a > # zfs set mountpoint=3Dlegacy zrootusb > # zfs set mountpoint=3D/ zdata > > > Exit SYSINSTALL and reboot. You now have a fully functional and > bootable FreeBSD installation on your usb memory stick. > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscribe@freebsd.org" > on thing you may consider doing is mirroring the way opensolaris does its boot environments. This makes it easy to flip flop back between os versions for testing or if things go wrong. eg from my home system # zfs list | grep be system/be 4.09G 123G 29.5K /system/be system/be/root20101123 2.63G 123G 1.76G legacy system/be/root20101218 1.47G 123G 992M legacy # grep vfs /boot/loader.conf; zpool get bootfs system vfs.root.mountfrom=3D"zfs:system/be/root20101218" vfs.root.mountfrom.options=3Drw NAME PROPERTY VALUE SOURCE system bootfs system/be/root20101218 local After I have done a build world and kernel I run this script. Sending and receiving isnt 100% necessary, but I find it cleaner than snapping and cloning as it doesnt end up in dependencies. Although it does eat up more disk space (assuming no dedup) but then we aren't talking huge amounts of disk space # cat /usr/local/scripts/install_world #!/usr/local/bin/bash if [ $UID !=3D 0 ] ; then echo your not root !! ; exit 1 fi date=3D`date '+%Y%m%d'` oroot=3D`grep "vfs.root.mountfrom=3D\"zfs:system/" /boot/loader.conf | sed = -e "s#^.*\"zfs:system/be/##" -e "s#\"##"` nroot=3D"root$date" snap=3D"autoup-$RANDOM" zpool=3Dsystem export DESTDIR=3D/$zpool/be/$nroot if [ "$oroot" =3D "$nroot" ] ; then echo "i cant update twice in one day"; exit 1 fi echo building in $zpool/be/$nroot zfs snapshot $zpool/be/$oroot@$snap && \ zfs send $zpool/be/$oroot@$snap | zfs receive -vv $zpool/be/$nroot && \ cd /usr/src && \ make installkernel && \ make installworld && \ sed -i -e "s#$zpool/be/$oroot#$zpool/be/$nroot#" /$zpool/be/$nroot/boot/loader.conf && \ echo -en "\n\nNow run these two commands to make the changes live, and reboot zfs set mountpoint=3Dlegacy $zpool/be/$nroot zpool set bootfs=3D$zpool/be/$nroot $zpool\n\n"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikcMscZHaC7DDR5PsZrUG4JtPtjwmkSd=MySfOi>