From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 8 06:22:26 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66781FA6 for ; Mon, 8 Dec 2014 06:22:26 +0000 (UTC) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by mx1.freebsd.org (Postfix) with ESMTP id EBD74D6A for ; Mon, 8 Dec 2014 06:22:25 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id 1E3521263A; Mon, 8 Dec 2014 16:22:18 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id CAH44223 (AUTH peterg@ptree32.com.au); Mon, 8 Dec 2014 16:22:17 +1000 Message-ID: <54854396.9000703@freebsd.org> Date: Sun, 07 Dec 2014 22:22:14 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Tinker Subject: Re: Can a host OS user process create a zillion BHyVe VM:s and microcontrol them? References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> <5fa49b79a601363b471babbfc577590d@openmailbox.org> <5483BA9F.2000905@freebsd.org> <0fd8dd84712a2d78e4397ac89d78326c@openmailbox.org> In-Reply-To: <0fd8dd84712a2d78e4397ac89d78326c@openmailbox.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2014 06:22:26 -0000 > Right, so a VT-x takes time - if we're talking 2500 cycles per > roundtrip, then you get approx one million of them per second - fair > enough. Not much time left for anything else though at that exit rate :( > As you point out the VM is launched absolutely bare. > > So then, strategies would need to be devised for how to run the code. > > I guess this would range all the way from running a barebones memory > management and pthreads implemenation (both those exist at least for > amd64, if I got it right, not sure how up to date they are though), > > ..to running a minimal BSD/Linux kernel within the VM for maxing > compatibility. > > Perhaps some neat memory dumping drick could be applied to zero the > BSD/Linux kernel boot time, i.e. the kernel doesn't boot but it's just > loaded from a hibernated memory image. Yep, that's possible. > Okay, so, there are two aspects to this problem now: > > (1) The BHyVe specific parts, about how to use BHyVe's interfaces to > deliver the virtualization/sandbox aspect, and > > (2) The guest specific parts, as in how run the intended > libraries/binaries as close to the metal as possible and with as low RAM > overhead as possible (and CPU overhead) within the sandbox. > > > I guess (2) is off-topic to this mailing list so should be looked into > separately. > > If you have any spontaneous thoughts on that one though, of course feel > free to share them :) There are many examples of small embedded o/s's that could easily be run in this environment - a reasonable example is the FreeBSD loader itself. > And, (1) is on this ML's topic, so I can direct any subsequent questions > about that to here. > > If you have any particular references with regard to header files, man > page or example program source files, feel free to pass them! Not a lot of documentation other than the bhyve source code itself. lib/libvmmapi/vmmapi_freebsd.c is an example of how to set up a 64-bit environment that the FreeBSD/amd64 kernel can be directly executed from. https://github.com/grehan-freebsd/grub2-bhyve/blob/master/grub-core/kern/emu/bhyve_hostif.c sets up a flat 32-bit environment that Linux/OpenBSD/NetBSD kernels expect to be invoked in. usr.sbin/bhyve/bhyverun.c has the main run loop for bhyve, which demonstrates how exits are handled. later, Peter.