From owner-freebsd-virtualization@FreeBSD.ORG Thu Oct 23 19:58:38 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 417195E5 for ; Thu, 23 Oct 2014 19:58:38 +0000 (UTC) Received: from mail-yh0-x22f.google.com (mail-yh0-x22f.google.com [IPv6:2607:f8b0:4002:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0248390B for ; Thu, 23 Oct 2014 19:58:37 +0000 (UTC) Received: by mail-yh0-f47.google.com with SMTP id c41so230386yho.34 for ; Thu, 23 Oct 2014 12:58:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=W+Eoi72pkb5AePXuOAjLFNBX6kMmiiEryCb4l+h9S4o=; b=xWrIsTBjCg59W7oR+LpkHsdJWfk+csWzjEN0UADmpyXATUzo/4jM/MU7wq0akaIhOl Mwcere58LxE5vixTc87a4GZ94Fp9cyyg+OfuAFtx8vz4FN7X5ZfIabcW+wrI82pT2JDw 5dXVl7X0Cd6C2ObBgPmLINh21EnhfQdw0SpSbLYDkrOuV9eL2zkU4nPBLry9jPoPrUPE es/WQzIgloJUVmmA7xJXzyzu+eHkOEHrRLwpwidCF8sZ3LePQEBqgot+3UT0SAMaZkoL idtExYtIhYNY92FlWDiAtLMxMSE69GiJxWa4ssDYVdAK9zdNKauIOHUVqYUiuaaxStl4 331w== X-Received: by 10.170.150.197 with SMTP id r188mr927554ykc.1.1414094317201; Thu, 23 Oct 2014 12:58:37 -0700 (PDT) Received: from m.west.isilon.com (c-98-209-117-232.hsd1.mi.comcast.net. [98.209.117.232]) by mx.google.com with ESMTPSA id x9sm2642586igl.10.2014.10.23.12.58.35 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Oct 2014 12:58:36 -0700 (PDT) From: Conrad Meyer To: libvir-list@redhat.com Subject: [PATCH 1/2] bhyve: Support /domain/bootloader configuration for non-FreeBSD guests. Date: Thu, 23 Oct 2014 15:58:03 -0400 Message-Id: <1414094284-29055-2-git-send-email-cse.cem@gmail.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1414094284-29055-1-git-send-email-cse.cem@gmail.com> References: <1414094284-29055-1-git-send-email-cse.cem@gmail.com> Cc: Conrad Meyer , freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Oct 2014 19:58:38 -0000 Also, flip Bhyve /domain/os/type support from HVM to Xen. Bhyve only supports paravirtualized guests, and 'xen' is closest to that. We still default to bhyveloader(1) if no explicit bootloader configuration is supplied in the domain. If the /domain/bootloader looks like grub-bhyve and the user doesn't supply /domain/bootloader_args, we make an intelligent guess and try chainloading the first partition on the disk. Caveats: - We can't install from CD without explicit bootloader_args. - We leave a device.map file lying around in /tmp. I don't see a good way not to do so without reworking the API somewhat. Sponsored by: EMC / Isilon storage division Signed-off-by: Conrad Meyer --- docs/drvbhyve.html.in | 30 +++++- docs/formatdomain.html.in | 4 +- po/libvirt.pot | 4 + src/bhyve/bhyve_capabilities.c | 2 +- src/bhyve/bhyve_command.c | 107 +++++++++++++++++++-- tests/bhyvexml2argvdata/bhyvexml2argv-acpiapic.xml | 2 +- tests/bhyvexml2argvdata/bhyvexml2argv-base.xml | 2 +- tests/bhyvexml2argvdata/bhyvexml2argv-console.xml | 2 +- .../bhyvexml2argvdata/bhyvexml2argv-disk-cdrom.xml | 2 +- .../bhyvexml2argv-disk-virtio.xml | 2 +- tests/bhyvexml2argvdata/bhyvexml2argv-macaddr.xml | 2 +- tests/bhyvexml2argvdata/bhyvexml2argv-metadata.xml | 2 +- tests/bhyvexml2argvdata/bhyvexml2argv-serial.xml | 2 +- .../bhyvexml2xmlout-metadata.xml | 2 +- 14 files changed, 139 insertions(+), 26 deletions(-) diff --git a/docs/drvbhyve.html.in b/docs/drvbhyve.html.in index 39afdf5..c6c79d7 100644 --- a/docs/drvbhyve.html.in +++ b/docs/drvbhyve.html.in @@ -37,8 +37,7 @@ bhyve+ssh://root@example.com/system (remote access, SSH tunnelled)

Example config

The bhyve driver in libvirt is in its early stage and under active development. So it supports -only limited number of features bhyve provides. All the supported features could be found -in this sample domain XML. +only limited number of features bhyve provides.

@@ -50,13 +49,13 @@ up to 31 PCI devices.

 <domain type='bhyve'>
-  <name>bhyve</name>
-  <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
+    <name>bhyve</name>
+    <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
     <memory>219136</memory>
     <currentMemory>219136</currentMemory>
     <vcpu>1</vcpu>
     <os>
-       <type>hvm</type>
+       <type>xen</type>
     </os>
     <features>
       <apic/>
@@ -157,5 +156,26 @@ An example of domain XML device entry for that will look like:

Please refer to the Storage documentation for more details on storage management.

+

Using grub2-bhyve or Alternative Bootloaders

+ +

It's possible to boot non-FreeBSD guests by specifying an explicit +bootloader, e.g. grub-bhyve(1). Arguments to the bootloader may be +specified as well. If no arguments are given and bootloader is +grub-bhyve, libvirt will try and boot from the first partition of +the disk image.

+ +
+  ...
+    <bootloader>/usr/local/sbin/grub-bhyve</bootloader>
+    <bootloader_args>...</bootloader_args>
+  ...
+
+ +

(Of course, to install from a CD a user will have to supply explicit +arguments to grub-bhyve.)

+ +

Caveat: bootloader_args does not support any quoting. +Filenames, etc, must not have spaces or they will be tokenized incorrectly.

+ diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 0099ce7..b7b6c46 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -217,7 +217,9 @@ a BIOS, and instead the host is responsible to kicking off the operating system boot. This may use a pseudo-bootloader in the host to provide an interface to choose a kernel for the guest. - An example is pygrub with Xen. + An example is pygrub with Xen. The Bhyve hypervisor + also uses a host bootloader, either bhyveload or + grub-bhyve.

diff --git a/po/libvirt.pot b/po/libvirt.pot
index 0b44ad7..d8c9a4d 100644
--- a/po/libvirt.pot
+++ b/po/libvirt.pot
@@ -851,6 +851,10 @@ msgstr ""
 msgid "domain should have at least one disk defined"
 msgstr ""
 
+#: src/bhyve/bhyve_command.c:407
+msgid "Custom loader requires explicit %s configuration"
+msgstr ""
+
 #: src/bhyve/bhyve_device.c:50
 msgid "PCI bus 0 slot 1 is reserved for the implicit LPC PCI-ISA bridge"
 msgstr ""
diff --git a/src/bhyve/bhyve_capabilities.c b/src/bhyve/bhyve_capabilities.c
index 132ce91..b37a24f 100644
--- a/src/bhyve/bhyve_capabilities.c
+++ b/src/bhyve/bhyve_capabilities.c
@@ -85,7 +85,7 @@ virBhyveCapsBuild(void)
                                    false, false)) == NULL)
         return NULL;
 
-    if ((guest = virCapabilitiesAddGuest(caps, "hvm",
+    if ((guest = virCapabilitiesAddGuest(caps, "xen",
                                          VIR_ARCH_X86_64,
                                          "bhyve",
                                          NULL, 0, NULL)) == NULL)
diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c
index bea4a59..99956ae 100644
--- a/src/bhyve/bhyve_command.c
+++ b/src/bhyve/bhyve_command.c
@@ -300,6 +300,7 @@ virBhyveProcessBuildLoadCmd(virConnectPtr conn,
 {
     virCommandPtr cmd;
     virDomainDiskDefPtr disk;
+    bool bhyveload, grub_bhyve;
 
     if (def->ndisks < 1) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
@@ -326,19 +327,105 @@ virBhyveProcessBuildLoadCmd(virConnectPtr conn,
         return NULL;
     }
 
-    cmd = virCommandNew(BHYVELOAD);
+    if (def->os.bootloader == NULL) {
+        bhyveload = true;
+        grub_bhyve = false;
+        cmd = virCommandNew(BHYVELOAD);
+    } else {
+        bhyveload = false;
+        if (strstr(def->os.bootloader, "grub-bhyve") == 0)
+            grub_bhyve = true;
+        cmd = virCommandNew(def->os.bootloader);
+    }
 
-    /* Memory */
-    virCommandAddArg(cmd, "-m");
-    virCommandAddArgFormat(cmd, "%llu",
-                           VIR_DIV_UP(def->mem.max_balloon, 1024));
+    if (bhyveload && def->os.bootloaderArgs == NULL) {
+        VIR_DEBUG("%s: bhyveload with default arguments", __func__);
+
+        /* Memory (MB) */
+        virCommandAddArg(cmd, "-m");
+        virCommandAddArgFormat(cmd, "%llu",
+                               VIR_DIV_UP(def->mem.max_balloon, 1024));
+
+        /* Image path */
+        virCommandAddArg(cmd, "-d");
+        virCommandAddArg(cmd, virDomainDiskGetSource(disk));
+
+        /* VM name */
+        virCommandAddArg(cmd, def->name);
+    } else if (grub_bhyve && def->os.bootloaderArgs == NULL) {
+        char tmpmapfile[128] = "/tmp/grub-bhyve-device.map_XXXXXX";
+        FILE *f;
+        int fd;
+
+        VIR_DEBUG("%s: grub-bhyve with default arguments", __func__);
+
+        /*
+         * XXX Default grub-bhyve has some BIG caveats, but MAY work for some
+         * typical configurations. In particular:
+         *
+         *   - Can't create a new VM this way (no CD, no boot from CD)
+         *   - Assumes a GRUB install on hd0,msdos1
+         */
 
-    /* Image path */
-    virCommandAddArg(cmd, "-d");
-    virCommandAddArg(cmd, virDomainDiskGetSource(disk));
+        /* XXX cleanup this file. */
+        fd = mkstemp(tmpmapfile);
+        if (fd < 0) {
+                virReportError(VIR_ERR_OPEN_FAILED, tmpmapfile);
+                goto error;
+        }
+
+        f = VIR_FDOPEN(fd, "wb+");
+        if (f == NULL) {
+                VIR_FORCE_CLOSE(fd);
+                virReportError(VIR_ERR_OPEN_FAILED, tmpmapfile);
+                goto error;
+        }
+
+        /* Grub device.map */
+        fprintf(f, "(hd0) %s\n", virDomainDiskGetSource(disk));
+        /* XXX CDs would look like: "(cd0) /path/to/CD" */
+
+        if (VIR_FCLOSE(f) < 0) {
+                virReportSystemError(errno, "%s", _("failed to close file"));
+                goto error;
+        }
 
-    /* VM name */
-    virCommandAddArg(cmd, def->name);
+
+        virCommandAddArg(cmd, "--device-map");
+        virCommandAddArg(cmd, tmpmapfile);
+
+        /* Memory in MB */
+        virCommandAddArg(cmd, "--memory");
+        virCommandAddArgFormat(cmd, "%llu",
+                               VIR_DIV_UP(def->mem.max_balloon, 1024));
+
+        /* To boot from CD, "cd0" here. */
+        virCommandAddArg(cmd, "--root");
+        virCommandAddArg(cmd, "hd0,msdos1");
+
+        /* VM name */
+        virCommandAddArg(cmd, def->name);
+    } else if (def->os.bootloaderArgs == NULL) {
+        virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+                       _("Custom loader requires explicit %s configuration"),
+                       "bootloader_args");
+        goto error;
+    } else {
+        char **blargs, **arg;
+
+        VIR_DEBUG("%s: custom loader '%s' with arguments", __func__,
+                  def->os.bootloader);
+
+        /* XXX: Handle quoted? */
+        blargs = virStringSplit(def->os.bootloaderArgs, " ", 0);
+        for (arg = blargs; *arg; arg++)
+                virCommandAddArg(cmd, *arg);
+        virStringFreeList(blargs);
+    }
 
     return cmd;
+
+error:
+    virCommandFree(cmd);
+    return NULL;
 }
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-acpiapic.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-acpiapic.xml
index 2be970e..bda6ab5 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-acpiapic.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-acpiapic.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-base.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-base.xml
index 3d23375..9ae8194 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-base.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-base.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-console.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-console.xml
index 35206b5..f2492b3 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-console.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-console.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-disk-cdrom.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-disk-cdrom.xml
index d70219c..b30c76f 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-disk-cdrom.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-disk-cdrom.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-disk-virtio.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-disk-virtio.xml
index 773d55e..83dbd3c 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-disk-virtio.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-disk-virtio.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-macaddr.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-macaddr.xml
index b262eb7..b2714c6 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-macaddr.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-macaddr.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-metadata.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-metadata.xml
index 6436301..09a793b 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-metadata.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-metadata.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2argvdata/bhyvexml2argv-serial.xml b/tests/bhyvexml2argvdata/bhyvexml2argv-serial.xml
index cd4f25b..fff6aa8 100644
--- a/tests/bhyvexml2argvdata/bhyvexml2argv-serial.xml
+++ b/tests/bhyvexml2argvdata/bhyvexml2argv-serial.xml
@@ -4,7 +4,7 @@
   219136
   1
   
-    hvm
+    xen
   
   
     
diff --git a/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-metadata.xml b/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-metadata.xml
index 77e18d4..08ff7d2 100644
--- a/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-metadata.xml
+++ b/tests/bhyvexml2xmloutdata/bhyvexml2xmlout-metadata.xml
@@ -9,7 +9,7 @@
   219136
   1
   
-    hvm
+    xen
     
   
   
-- 
1.9.3