From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 23 05:06:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F4CBE44 for ; Thu, 23 Apr 2015 05:06:14 +0000 (UTC) Received: from mail.metricspace.net (207-172-209-89.c3-0.arl-ubr1.sbo-arl.ma.static.cable.rcn.com [207.172.209.89]) by mx1.freebsd.org (Postfix) with ESMTP id 4692716E9 for ; Thu, 23 Apr 2015 05:06:13 +0000 (UTC) Received: from [172.16.1.102] (unknown [172.16.1.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 8182A2F257 for ; Thu, 23 Apr 2015 05:06:06 +0000 (UTC) Message-ID: <55387DBE.7040706@metricspace.net> Date: Thu, 23 Apr 2015 01:06:06 -0400 From: Eric McCorkle User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: FreeBSD and Gentoo Linux on the same ZFS volume Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2015 05:06:14 -0000 I had previously posted asking if anyone had ever gotten FreeBSD and a linux distro to share the same ZFS volume. I have since accomplished this setup with FreeBSD and Gentoo linux on a GPT disk with a single ZFS volume. I'll write a quick-and-dirty description of how to do it here; at some point in the future, I'll write a more in-depth wiki page about it. == Setting up the Volume == Assume the pool's name is "data". I have the following datasets: data/freebsd, mountpoint = legacy (root of FreeBSD) data/freebsd/*, all mountpoints = legacy data/home, mountpoint = /home (shared home dirs between linux and BSD) data/linux, mountpoint = legacy (root of linux) data/linux/*, all mountpoints = legacy Note that only the datasets that are shared between the two OSes use the ZFS mounting system. The ZFS code doesn't seem to support a "just mount everything under my root" option (if it does, someone please point it out). Thus, I have fstab entries for all the OS-specific datasets. I also have a /linux directory under the FreeBSD root, and a /freebsd directory under the linux root, at which the corresponding OS's filesystems are mounted. Thus, each OS is accessible from the other. I can see this being particularly useful to folks trying to port device drivers. Lastly, make sure to set data/freebsd as the boot dataset. Linux can specify which dataset to use as root. == Booting == Boot selection can be accomplished using GRUB. You can install GRUB using the FreeBSD port. The MBR/bios solution should work for now; I'm currently working on getting the EFI loader to work with ZFS, but it's still a ways out. Unfortunately, the port doesn't seem to be particularly well-documented. The following procedure will set up an EFI-based GRUB with a linux and a freebsd option (the freebsd option currently won't work with EFI and ZFS) 1) Install the sysutils/grub2-efi port 2) Edit /usr/local/etc/grub.d/40-custom file, add the following: menuentry "Gentoo Linux" { search.fs_label data ZFS_PART linux ($ZFS_PART)/linux@/boot/kernel dozfs=force root=ZFS=data/linux initrd ($ZFS_PART)/linux@/boot/initramfs } You'll also need to add a FreeBSD entry, as the script to detect FreeBSD partitions doesn't seem to recognize ZFS. 3) Do "grub-mkconfig > /boot/grub.cfg" to generate the config file 4) Mount the EFI system partition under /mnt 5) Run "grub-install --efi-directory=/mnt --target=x86_64-efi --modules=part_gpt". This will install the binary at /mnt/efi/grub/grubx64.efi 6) Run "cp /mnt/efi/grub/grubx64.efi /mnt/efi/boot/bootx64.efi" to copy GRUB into position to be run by the EFI loader. It should be possible to rework this for legacy BIOS boot, which will actually allow you to select the FreeBSD option and have it work. == Installing FreeBSD == I currently have a legacy BIOS boot set up with a GPT disk, which boots FreeBSD from the ZFS partition. There are guides for how to do this, and they actually work. You can do a MBR/BSDlabel setup if your BIOS can't do legacy boots with a GPT disk (Apple). Be sure to update /etc/fstab too. == Installing Gentoo Linux == The recent livedvd images have ZFS support, and will support the current pool version. Older livedvds and other distros' installers sometimes only support the older pool versions. Once you get a livedvd with the right ZFS version, you can mount the pool and follow the gentoo install guide for the most part. You'll have to build a new kernel, install the ZFS ports, then build the initramfs, which can be a bit tricky. Try to get the kernel/modules/initramfs setup working enough to boot, then circle back and install the rest of the OS. Also, make sure you update /etc/fstab. == Provisos == * I wouldn't recommend sharing swap space, as at least FreeBSD might have saved a crash dump there (and I'm pretty sure linux swap has to be prepared first) * Sharing a L2ARC or intent log is also probably a bad idea. The disjoint access patterns of the two OSes will mess up the locality properties. == Running == Once you've got everything set up, you should be able to boot either OS, use a common /home, and then access the other OS from the /linux or /freebsd directories. As there isn't ZFS support in the EFI loader (yet!), I currently have to switch to legacy BIOS boot mode to boot FreeBSD, and switch back to EFI to boot gentoo (via GRUB).