Date: Sat, 25 Jun 2016 20:00:58 +0200 From: Fabian Freyer <fabian.freyer@physik.tu-berlin.de> To: soc-status@FreeBSD.org Subject: Improving libvirt support for bhyve Message-ID: <9426d7b2-6690-a0ca-ee37-c723e63c486c@physik.tu-berlin.de>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --H31gqoMqHIpSmbtFwe8Lnqvk4aucJJLTS Content-Type: multipart/mixed; boundary="0I0I21xEfNgvXTPfJ5kApkbbDcn8g8KPb" From: Fabian Freyer <fabian.freyer@physik.tu-berlin.de> To: soc-status@FreeBSD.org Cc: novel@FreeBSD.org Message-ID: <9426d7b2-6690-a0ca-ee37-c723e63c486c@physik.tu-berlin.de> Subject: Improving libvirt support for bhyve --0I0I21xEfNgvXTPfJ5kApkbbDcn8g8KPb Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi everybody, unfortunately I had overlooked the fact that I should have provided weekl= y status reports to this list, so in this mail I will present what I've don= e so far and will aim to update this thread regularly from now on. Therefore t= his initial mail will be a bit lengthy. Introduction =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The primary aim of this project is to implement missing calls and functio= nality in the libvirt bhyve driver. A number of API calls are not yet implemente= d, and are still to be implemented. Details can be found on my wiki page [1]. S= ince this is mostly code that is contributed to the libvirt project, I base my= patches off the libvirt git repository [2] and host a development fork on= Github [3]. Things I have achieved so far =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D Simple calls ------------ I have implemented some simple calls for the bhyve driver and already ups= treamed them: * https://libvirt.org/git/?p=3Dlibvirt.git;a=3Dcommitdiff;h=3D126e630e853= 11a57f87709a269bc55059cb3cf44 * https://libvirt.org/git/?p=3Dlibvirt.git;a=3Dcommitdiff;h=3Da1efc9428bc= df6075fac0dac4d73ff2858216d7f * https://libvirt.org/git/?p=3Dlibvirt.git;a=3Dcommitdiff;h=3D32aa9ed3ba2= 117936b8634091330418573d9b8c8 * https://libvirt.org/git/?p=3Dlibvirt.git;a=3Dcommitdiff;h=3Def45eb9bc77= 82a1edb7237bb0307afe576d287d4 bhyveConnectDomainXMLFromNative ------------------------------- For the last weeks, the majority of my work was focused on implementing a= call that generates a libvirt domain definition XML file from a native bhyve a= nd bhyveload command string. The first three versions of this patch series h= ave already hit the libvirt mailing list [4], and I aim to submit v4 to the l= ibvirt mailing list early next week. For a detailed overview of the work I have done there, I will cite or sum= marize the commit messages of the last submitted patch series: * I found a bug (?) in gnulib on FreeBSD where several gnulib headers rel= y on features.h being included by ctype.h to provide __GNUC_PREREQ, but on s= ystems without glibc (such as FreeBSD), this is not provided. This is yet to b= e reported to gnulib, but I've created a workaround for libvirt so far. * I've imported the getopt-posix gnulib module into the libvirt build * I've created some string-handling functions that unescape the original = command line and splits it up into an argv-list for the bhyve loader and bhyver= un commands, respectively. * A simple getopt-based argument parser is added for the /usr/sbin/bhyve command, loosely based on its argument parser, which reads the followin= g from the bhyve command line string: * vm name * number of vcpus * memory size * the time offset (UTC or localtime). This includes a capability check = to see if this is actually supported by the bhyve version. * features: * acpi * ioapic: While this flag is deprecated in FreeBSD r257423, keep chec= king for it for backwards compatibiility. * the domain UUID; if not explicitely given, one will be generated. * lpc devices: for now only the com1 and com2 are supported. It is requ= ired for these to be /dev/nmdm[\d+][AB], and the slave devices are automat= ically inferred from these to be the corresponding end of the virtual null-m= odem cable: /dev/nmdm<N>A <-> /dev/nmdm<N>B * PCI devices: * Disks: these are numbered in the order they are found, for virtio a= nd ahci disks separately. The destination is set to sdX or vdX with X=3D'a'= +index; therefore only 'z'-'a' disks are supported. Disks are considered to be block devices if the path starts with /dev, otherwise they are considered to be files. * Networks: only tap devices are supported. Since it isn't possible t= o tell the type of the network, VIR_DOMAIN_NET_TYPE_ETHERNET is assumed, s= ince it is the most generic. If no mac is specified, one will be generated.= * A simple getopt-based argument parser is added for the /usr/sbin/bhyvel= oad command, loosely based on its argument parser. The boot disk is guessed by iterating over all disks and matching their sources. If any non-default arguments are foun= d, def->os.bootloaderArgs is set accordingly, and the bootloader is treate= d as a custom bootloader. Custom bootloader are supported by setting the def->os.bootloader and def->os.bootloaderArgs accordingly grub-bhyve is also treated as a custom bootloader. Since we don't get t= he device map in the native format anyways, we can't reconstruct the compl= ete boot order. While it is possible to check what type the grub boot disk = is by checking if the --root argument is "cd" or "hd0,msdos1", and then just = use the first disk found, implementing the grub-bhyve argument parser as-is in = the grub-bhyve source would mean adding a dependency to argp or duplicating= lots of the code of argp. Therefore it's not really worth implementing that = now. In the last week I have been working on incorporating the feedback I got = on my previous patches as well as creating a number of unit tests for the patch= series. Currently I only have positive-tests (check that things work) for= most of the options of the bhyverun command, but none yet for the loader comma= nd. I also aim to implement some negative tests to check that things that shoul= d fail or report a warning do so; and then re-submit my patches to the libvirt-m= ailing list. My latest work on this feature can be found in the gsoc/rebase/bhyveConnectDomainXMLFromNative_v4 branch on Github. Bhyve UEFI Support ------------------ As a secondary focus (mostly to get a bit of distance to the code for the= Native to XML feature to get a different perspective on it) I have been playing = around with the bhyve uefi boot support, and discussing how best to change the l= ibvirt driver for bhyve to support booting using UEFI firmwares. I've introduced a capability check whether bhyve supports -l bootrom (it = is seemingly not very easy to check what a given bhyve binary supports or ev= en what version it is apart from checking uname; I think this is one of the = reasons why my mentor novel@ created a patch [5] to see what PCI and LPC devices = are supported - if this would get into 11 it would be a huge help), and discu= ssed how the bootrom best be specified in the domain definiton, namely by spec= ifying the firmware blob in a <loader type=3D"pflash"> tag within the <os> tag. However, when booting using an UEFI bootrom, no loader command should be specified; this means a number of things in the logic that builds a bhyve= command string from a libvirt domain definition need to be changed. I am currently still trying to see how best to do this in a way that keeps eve= rything else working. However, I have deliberately designed the Native->XML code in a way that = adding this feature would be simple, and plan to integrate that should that patc= h serios get accepted. Takeaways and Conclusions =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =46rom the work I've done so far I have developed a better understanding = of the libvirt code base and the time and effort it takes to implement features = in a libvirt driver. This enables me to more realistically plan my second half= of the coding period, during which I plan to finish my work on the Native->XML f= eature as well as UEFI boot support. While I will further incorporate feedback o= n the Native->XML feature, it will no longer be a key focus of my work (since m= ost of it will be done - hopefully) and instead I will shift to working on getti= ng reboot and reset working. Further details on my plan will be published on the wiki page [1] as they= become clear. Fabian Links: [1] https://wiki.freebsd.org/SummerOfCode2016/ImprovingLibvirtSupportForB= hyve [2] https://libvirt.org/git/?p=3Dlibvirt.git;a=3Dsummary [3] https://github.com/fabianfreyer/libvirt [4] https://www.redhat.com/archives/libvir-list/2016-June/msg00001.html https://www.redhat.com/archives/libvir-list/2016-June/msg00728.html https://www.redhat.com/archives/libvir-list/2016-June/msg01741.html [5] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210111 --0I0I21xEfNgvXTPfJ5kApkbbDcn8g8KPb-- --H31gqoMqHIpSmbtFwe8Lnqvk4aucJJLTS Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJXbsbdAAoJEDlu9CkMwdG7a6oP+wVCBkzukOVTdxB2TFkpbHDG Bcjqjr3q05d2hbEv/wO7RHe/kM6PyVVbtADho/J31AOmMR3OlDWzmoDNtfvE1Keu gnbTlXUL+U+C7DvNsZhQ0twzkizeA4AJw4WqahoEFLeydiHWbu12dqPDCS/zfbmb fw1g9Q9pIs33fY7UxQVbt58nlB9GCshq9aiQSCGGbyUwAXvfLNwQWg2qAu9AeXDr C+BJnSkhj1o5ERGcm9UlzWkl3SfCGH4Lz/ppIbK1dd1rGR7BSmbXaFO4HAcTECxb YxvuFQHtwS84nGhJUmEsf3GKK4jRknqrzPo21nxpAq6RC0scwf736uKuSmkrjkVY V0GCjNjV4W6rwoExBlU2jfIN6dYYiYqkX6qNr4g/QO6o+OayFo7XACAtqzMUQ+fL 391bevF6YooylwMVoPc1rPZfV7sxgLoXIFUyAbeehNgC3QPb9F76TrjTjglNRpQO zouZkzEYJPEFyIfSjMGsE+k7U0/ZfBg5tVww88rb2BdrrjxAKNxyad706J86gnnH jvtwZvhnk5R3DMLq0HWsURv6A6XcdkiYJg7oqodezG3xtifBzIJ5w+yXfJU2FfhQ 7HbcpQ+egA2agnxAcAOe9MV4KZwcFTdMcbcIMMm7YS3yaTZfGlJWoRhL6oNuGGWh bTlvba6kQXlfc53HKci8 =Mql2 -----END PGP SIGNATURE----- --H31gqoMqHIpSmbtFwe8Lnqvk4aucJJLTS--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9426d7b2-6690-a0ca-ee37-c723e63c486c>