From owner-freebsd-virtualization@FreeBSD.ORG Tue Jan 13 22:22:11 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7AA8BAC for ; Tue, 13 Jan 2015 22:22:11 +0000 (UTC) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by mx1.freebsd.org (Postfix) with ESMTP id ABF9768E for ; Tue, 13 Jan 2015 22:22:11 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id 08CDC1277F; Wed, 14 Jan 2015 08:22:10 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id CAX03441 (AUTH peterg@ptree32.com.au); Wed, 14 Jan 2015 08:22:09 +1000 Message-ID: <54B59A8E.8090206@freebsd.org> Date: Tue, 13 Jan 2015 14:22:06 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Zaphod Beeblebrox Subject: re: There's a whole lot of X34x0's out there that fail. Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 22:22:12 -0000 Hi, (moving this to the -virtualization list) > From what I've read around here, an entire swatch of CPUs have a > problem --- they have all the virtualization gunk save the ability to run > in16 bit mode in emulation. An example would be the X34x0 ... the > cornerstone processor for an entire generation of servers. I, for > instance, have four of them. While it is unfortunate that bhyve doesn't fully support Nehalems, we decided that the amount of work required to complete the support wasn't worth it, given that all follow-on models have 16-bit support. That being said, I'm certainly willing to provide help to anyone who wants to take on that work. > The "fail" in the subject line is about their inability to do two > things under bhyve: > > 1) launch linux through grub > > 2) have a VM with more than one processor > > Can we adopt what linux does to work around it? The various Linux VM solutions use an instruction emulator to run until the CPU is about to transition to 32-bit paged mode, and then cut over to using VT-x. Pulling this in to bhyve is a decidely non-trivial amount of work, especially since the majority of x86 emulator codebases are GPL'd. > Is it even necessary to boot linux with Grub? It's not Grub that is the issue, but the fact that the processor can't run in anything less than 32-bit paged mode. Grub boots Linux in either 16-bit real mode or 32-bit protected mode with paging disabled. However, here is a 64-bit entry point available for Linux/x64 (see "64-bit BOOT PROTOCOL" at https://www.kernel.org/doc/Documentation/x86/boot.txt). For anyone willing to have a go at this, my recommendation would be to modify grub-bhyve to boot Linux/x64 directly into 64-bit mode. This is already done for kFreeBSD/amd64 in grub (and for FreeBSD in bhyveload/libvmmapi) so that setup code could be reused. For MP support, the work would be to look at the Linux/FreeBSD MP trampolines and write an emulator that could either recognize those and skip them, or emulate only the instructions required to bring the vCPU into 32-bit protected mode, and then switch over to VT-x h/w emulation. later, Peter.