From owner-freebsd-fs@FreeBSD.ORG Fri Oct 16 22:44:07 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 139B3106568B for ; Fri, 16 Oct 2009 22:44:07 +0000 (UTC) (envelope-from torbjoern@gmail.com) Received: from mail-fx0-f210.google.com (mail-fx0-f210.google.com [209.85.220.210]) by mx1.freebsd.org (Postfix) with ESMTP id 953818FC0A for ; Fri, 16 Oct 2009 22:44:06 +0000 (UTC) Received: by fxm6 with SMTP id 6so2902433fxm.43 for ; Fri, 16 Oct 2009 15:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=+RTh/bNJU/z5n1+/LdqsOoiCzL9eU5wS0keg8Uy7iG4=; b=wDFzx1/pUVH2amNeySoa24K7wicWe3gm2v/vhdolvXKrD3j+mfAl4n8Rf1QJVXLq5u fBEYzNOs0fLEtSXLfhOD07702Y/o4vYKllI1N50BraMtPemOKQ2kDR3C+XQ70AG7dBjo m0/zCj+STIBQn5kBcSSU8ToUufY6bjTvDdkvk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=EEU2cB0tHsHV0WwczYLYthV5pFJLjhqDgL2TLRNpKY9ldowwilBfkW5abIfAvsl1Cf Xt7Ncq6N4dqpq6T2Hs0U2rGuFx8uzXZ4hsLmJSH8LJFW53Zc35zLSyfZfefTzFPSDamy /9UERLJ2zvJCbaghkVtbm5yIduscAx18CiEW4= MIME-Version: 1.0 Received: by 10.204.153.220 with SMTP id l28mr1864123bkw.86.1255731506226; Fri, 16 Oct 2009 15:18:26 -0700 (PDT) Date: Sat, 17 Oct 2009 00:18:26 +0200 Message-ID: <103e3fa20910161518t57e2c3afkddf160cb1f4ecfdb@mail.gmail.com> From: Torbjorn Kristoffersen To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Advice needed with ZFS on root filesystem (installing remotely via mfsBSD) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2009 22:44:07 -0000 I am trying to install FreeBSD 8.0-RC1 on amd64, and I plan to use ZFS (mirrored) as the root filesystem. I have been struggling with this for many hours -- it just won't boot up and I can't understand what I'm doing wrong. Before I describe my steps, keep in mind that I am doing this using the Remote Install method of mfsBSD. My mfsBSD system boots up just fine and I log in via SSH. However, seeing that it's a remote installation I'm unable to see any error message if the system does not boot up correctly. A sad situation, indeed. What is wrong with my approach ? Please see below. -- First I create a GPL label on my two disks % gpart create -s gpt ad4 % gpart create -s gpt ad6 Then I add a boot-partition that will hold the ZFS bootcode % gpart add -b 34 -s 128 -t freebsd-boot ad4 % gpart add -b 34 -s 128 -t freebsd-boot ad6 I then add the freebsd-zfs partition on each disk % gpart add -b 162 -s 1465148973 -t freebsd-zfs ad4 % gpart add -b 162 -s 1465148973 -t freebsd-zfs ad6 The results: % gpart show ad4 => 34 1465149101 ad4 GPT (699G) 34 128 1 freebsd-boot (64K) 162 1465148973 2 freebsd-zfs (699G) % gpart show ad6 => 34 1465149101 ad6 GPT (699G) 34 128 1 freebsd-boot (64K) 162 1465148973 2 freebsd-zfs (699G) I add GPT bootcode to the MBR and add the ZFS bootcode: % gpart bootcode -b /boot/pbmr -p /boot/gptzfsboot -i 1 ad4 % gpart bootcode -b /boot/pbmr -p /boot/gptzfsboot -i 1 ad6 I make sure that the first partition is set to active % fdisk -a /dev/ad4 ... % fdisk -a /dev/ad6 I create zpool on ad4p2 and ad6p2 (mirrored) % zpool create zroot mirror ad4p2 ad6p2 Create filesystem % zfs create -p zroot Make sure that I enable the bootfs % zpool set bootfs=zroot zroot At this point, /zroot is mounted on my filesystem, so I go ahead and do a sysinstall and install FreeBSD. I make sure that /zroot is set as "root folder" in sysinstall's Options menu. I add zfs_load="YES" to /zroot/boot/loader.conf and vfs.root.mountfrom="zfs:zroot" to /zroot/boot/loader.conf And I also copy /boot/zfs/zpool.cache to /zroot/boot/zfs/zpool.cache Then I do % chroot /zroot % sysinstall ... I go into configuration, set the proper network settings, make sure that sshd will get started. I change root's password and create a user. My /etc/rc.conf (on zroot) looks like this: ifconfig_re0="inet xxx.xxx.121.11 netmask 255.255.255.192" defaultrouter="xxx.xxx.121.1" sshd_enable="YES" hostname="grim" At this point I have also tried compiling a kernel, making sure that GEOM_PART_BSD, GEOM_PART_MBR and other required options are included. I also installed a ZFS aware /boot/loader (**) (Note: I will create a swap partition later once I get the installation booted.) At this point % exit (from the chroot) % zfs unmount -a % reboot At this point the computer never comes back and I have to use my colo's rescue system to put mfsBSD back on. One thing I noticed at that point, is that the freebsd-boot on ad4 is gone and the freebsd-zfs partition has been replaced with a smaller freebsd-ufs. I did not touch "Partitions" or "Labels" in the sysinstall installation. What could possibly cause this? (In case it matters; ad6 is unchanged.) Kind Regards, Torbjorn Kristoffersen (**) http://wiki.freebsd.org/ZFSOnRootWithZFSboot#installLoader