From owner-svn-doc-all@FreeBSD.ORG Mon May 5 16:03:31 2014 Return-Path: Delivered-To: svn-doc-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 433C3CEE; Mon, 5 May 2014 16:03:31 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2ED101D81; Mon, 5 May 2014 16:03:31 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s45G3VCa069275; Mon, 5 May 2014 16:03:31 GMT (envelope-from dru@svn.freebsd.org) Received: (from dru@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s45G3VOG069274; Mon, 5 May 2014 16:03:31 GMT (envelope-from dru@svn.freebsd.org) Message-Id: <201405051603.s45G3VOG069274@svn.freebsd.org> From: Dru Lavigne Date: Mon, 5 May 2014 16:03:31 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r44762 - head/en_US.ISO8859-1/books/handbook/virtualization X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire doc trees \(except for " user" , " projects" , and " translations" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 16:03:31 -0000 Author: dru Date: Mon May 5 16:03:30 2014 New Revision: 44762 URL: http://svnweb.freebsd.org/changeset/doc/44762 Log: Add section on bhyve to Virtualization chapter in prep for closing PR189216. Editorial review to follow. Submitted by: Allan Jude Sponsored by: iXsystems Modified: head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml Modified: head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml ============================================================================== --- head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml Mon May 5 15:48:07 2014 (r44761) +++ head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml Mon May 5 16:03:30 2014 (r44762) @@ -20,6 +20,16 @@ Contributed by + + + + + Allan + Jude + + bhyve section by + + @@ -1110,8 +1120,8 @@ EndSection - - &os; as a Host + + &os; as a Host with <application>VirtualBox</application> &virtualbox; is an actively developed, complete virtualization package, that is available @@ -1273,7 +1283,310 @@ perm pass* 0660 &prompt.root; service devfs restart + + + + &os; as a Host with + <application>bhyve</application> + + Starting with &os; 10.0-RELEASE the BSD licensed hypervisor + bhyve is part of the base system. + bhyve supports a number of guests + including &os;, OpenBSD, and many flavors of &linux;. + bhyve currently only supports a + serial console and does not emulate a graphical console. + bhyve is a legacy-free hypervisor, + meaning that instead of translating instructions, and manually + managing memory mappings, it relies on the virtualization + offload features of newer CPUs. + bhyve also avoids emulating + compatible hardware for the guest, and instead relies on the + paravirtualization drivers, called + VirtIO. + + Due to the design of bhyve, it is + only possible to use bhyve on + computers with newer processors that support &intel; + EPT (Extended Page Tables) or &amd; + RVI (Rapid Virtualization Indexing, also know + as NPT or Nested Page Tables). Most newer + processors, specifically the &intel; &core; i3/i5/i7 and + &intel; &xeon; E3/E5/E7 support this feature, for a + complete list of &intel; processors that support + EPT see the &intel; + ARK. RVI is found on the 3rd + generation and later of the &amd.opteron; (Barcelona) + processors. The easiest way to check for support of + EPT or RVI on a system is + to look for the POPCNT processor feature flag + on the Features2 line in + dmesg or + /var/run/dmesg.boot. + + + Preparing the Host + + The first step to creating a virtual machine in + bhyve is configuring the host + system. Load the bhyve kernel + module called vmm. Create a tap + interface for the network device in the virtual machine to + attach to. Optionally create a bridge interface and add the + tap interface as well as the physical + interface as members to allow the virtual machine to have + access to the network. + + &prompt.root; kldload vmm +&prompt.root; kldload nmdm +&prompt.root; ifconfig tap0 create +&prompt.root; sysctl net.link.tap.up_on_open=1 +net.link.tap.up_on_open: 0 -> 1 +&prompt.root; ifconfig bridge0 create +&prompt.root; ifconfig bridge0 addm igb0 addm tap0 +&prompt.root; ifconfig bridge0 up + + + + + Creating a FreeBSD Guest + + Create a file to use as the virtual disk for the guest + machine. + + &prompt.root; truncate -s 16G guest.img + + Download an installation image of &os; to install: + + &prompt.root; fetch ftp://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-amd64/10.0/FreeBSD-10.0-RELEASE-amd64-bootonly.iso +FreeBSD-10.0-RELEASE-amd64-bootonly.iso 100% of 209 MB 570 kBps 06m17s + + &os; comes with an example script for running a virtual + machine in bhyve. The script will + start the virtual machine and run it in a loop, so it will + automatically restart if it crashes. The script takes a + number of options to control the configuration of the machine. + controls the number of virtual CPUs. + limits the amount of memory available to + the guest. defines which + tap device to use. + indicates which disk image to use. tells + bhyve to boot from the CD image + instead of the disk, and defines which CD + image to use. Finally the last parameter is the name of the + virtual machine, used to track the running machines. Start + the virtual machine in installation mode: + + &prompt.root; sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.img -i -I FreeBSD-10.0-RELEASE-amd64-bootonly.iso guestname + + The system will boot and start the installer. After + installing a system in the virtual machine, when the system + asks about dropping in to a shell at the end of the + installation, choose Yes. A small + change needs to be made to make the system start with a serial + console. Edit /etc/ttys and replace the + existing console line with: + + console "/usr/libexec/getty std.9600" xterm on secure + + Reboot the virtual machine. Rebooting the virtual machine + causes bhyve to exit. However the + vmrun.sh script runs + bhyve in a loop and will automatically + restart it. When this happens, choose the reboot option from + the boot loader menu, and this will escape the loop. Now the + guest can be started from the virtual disk: + + &prompt.root; sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.img guestname + + + + Creating a &linux; Guest + + bhyve requires + sysutils/grub2-bhyve in order to boot + operating systems other than &os;. + + Create a file to use as the virtual disk for the guest + machine. + + &prompt.root; truncate -s 16G linux.img + + Starting a virtual machine with + bhyve is a two step process. First + a kernel must be loaded, then the guest can be started. + sysutils/grub2-bhyve is used to load the + &linux; kernel. Create a device.map that + grub will use to map the virtual + devices to the files on the host system: + + (hd0) ./linux.img +(cd0) ./somelinux.iso + + Use sysutils/grub2-bhyve to load the + &linux; kernel from the ISO image: + + &prompt.root; grub-bhyve -m device.map -r cd0 -M 1024M linuxguest + + This will start grub. If the installation CD contains a + grub.cfg then a menu will be displayed. + If not, the vmlinuz and + initrd files must be located and loaded + manually: + + grub> ls +(hd0) (cd0) (cd0,msdos1) (host) +grub> ls (cd0)/isolinux +boot.cat boot.msg grub.conf initrd.img isolinux.bin isolinux.cfg memtest +splash.jpg TRANS.TBL vesamenu.c32 vmlinuz +grub> linux (cd0)/isolinux/vmlinuz +grub> initrd (cd0)/isolinux/initrd.img +grub> boot + + Now that the &linux; kernel is loaded, the guest can be + started: + + &prompt.root; bhyve -AI -H -P \ +-s 0:0,hostbridge \ +-s 1:0,lpc \ +-s 2:0,virtio-net,tap1 \ +-s 3:0,virtio-blk,./linux.img \ +-s 4:0,ahci-cd,./somelinux.iso \ +-l com1,stdio \ +-c 4 -m 1024M linuxguest + + The system will boot and start the installer. After + installing a system in the virtual machine, reboot the virtual + machine. This will cause bhyve to + exit. The instance of the virtual machine needs to be + destroyed before it can be started again: + + &prompt.root; bhyvectl --destroy --vm=linuxguest + + Now the guest can be started directly from the virtual + disk. Load the kernel: + + &prompt.root; grub-bhyve -m device.map -r hd0,msdos1 -M 1024M linuxguest +grub> ls +(hd0) (hd0,msdos2) (hd0,msdos1) (cd0) (cd0,msdos1) (host) +(lvm/VolGroup-lv_swap) (lvm/VolGroup-lv_root) +grub> ls (hd0,msdos1)/ +lost+found/ grub/ efi/ System.map-2.6.32-431.el6.x86_64 config-2.6.32-431.el6.x +86_64 symvers-2.6.32-431.el6.x86_64.gz vmlinuz-2.6.32-431.el6.x86_64 +initramfs-2.6.32-431.el6.x86_64.img +grub> linux (hd0,msdos1)/vmlinuz-2.6.32-431.el6.x86_64 root=/dev/mapper/VolGroup-lv_root +grub> initrd (hd0,msdos1)/initramfs-2.6.32-431.el6.x86_64.img +grub> boot + + Boot the virtual machine: + + &prompt.root; bhyve -AI -H -P \ +-s 0:0,hostbridge \ +-s 1:0,lpc \ +-s 2:0,virtio-net,tap1 \ +-s 3:0,virtio-blk,./linux.img \ +-l com1,stdio \ +-c 4 -m 1024M linuxguest + + &linux; will now boot in the virtual machine and + eventually present you with the login prompt. Login and use + the virtual machine. When you are finished, reboot the + virtual machine to exit bhyve. + Destroy the virtual machine instance: + + &prompt.root; bhyvectl --destroy --vm=linuxguest + + + + Virtual Machines Consoles + + It is advantageous to wrap the + bhyve console in a session + management tool such as sysutils/tmux or + sysutils/screen in order to detach and + reattach to the console. It is also possible to have the + console of bhyve be a null modem + device that can be accessed with cu. Load + the nmdm kernel module, and replace + with + . The + /dev/nmdm devices are created + automatically as needed, each is a pair, + /dev/nmdm1A and + /dev/nmdm1B corresponding to the two ends + of the null modem cable. See &man.nmdm.4; for more + information. + + &prompt.root; bhyve -AI -H -P \ +-s 0:0,hostbridge \ +-s 1:0,lpc \ +-s 2:0,virtio-net,tap1 \ +-s 3:0,virtio-blk,./linux.img \ +-l com1,/dev/nmdm0A \ +-c 4 -m 1024M linuxguest +&prompt.root; cu -l /dev/nmdm0B -s 9600 +Connected + +Ubuntu 13.10 handbook ttyS0 + +handbook login: + + + + Managing Virtual Machines + + A device node is created in /dev/vmm for each virtual + machine. This allows the administrator to easily see a list + of the running virtual machines: + + &prompt.root; ls -al /dev/vmm +total 1 +dr-xr-xr-x 2 root wheel 512 Mar 17 12:19 ./ +dr-xr-xr-x 14 root wheel 512 Mar 17 06:38 ../ +crw------- 1 root wheel 0x1a2 Mar 17 12:20 guestname +crw------- 1 root wheel 0x19f Mar 17 12:19 linuxguest +crw------- 1 root wheel 0x1a1 Mar 17 12:19 otherguest + + Virtual machines can be destroyed using + bhyvectl: + + &prompt.root; bhyvectl --destroy --vm=guestname + + + + Persistent Configuration + + In order to make the system able to start + bhyve guests at boot time, the + following configurations must be made in the specified + files: + + + + <filename>/etc/sysctl.conf</filename> + + net.link.tap.up_on_open=1 + + + + <filename>/boot/loader.conf</filename> + + vmm_load="YES" +nmdm_load="YES" +if_bridge_load="YES" +if_tap_load="YES" + + + + <filename>/etc/rc.conf</filename> + + cloned_interfaces="bridge0 tap0" +ifconfig_bridge0="addm igb0 addm tap0" + + +