Date: Tue, 17 Jun 2014 10:59:14 -0600 From: John Nielsen <lists@jnielsen.net> To: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org> Subject: Re: consistent VM hang during reboot Message-ID: <7D960087-1839-4295-AEE3-5D8F60D9D710@jnielsen.net> In-Reply-To: <0238084D-FD0F-42A5-85F5-597A590E666C@jnielsen.net> References: <BED233F2-EAFF-41A3-9C5B-869041A9AED8@jnielsen.net> <201405081303.17079.jhb@freebsd.org> <E97C3027-79CF-45F9-B5ED-3339D7AE0B5F@jnielsen.net> <af0f4c6348d64ab0b5ea56d2ea777e99@BY2PR05MB582.namprd05.prod.outlook.com> <2CCD4068-A9CB-442C-BB91-ADBF62FF22C6@jnielsen.net> <83DA2398-0004-49EC-8AC1-9AA64F33A194@jnielsen.net> <0238084D-FD0F-42A5-85F5-597A590E666C@jnielsen.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Jun 13, 2014, at 4:23 PM, John Nielsen <lists@jnielsen.net> wrote:
> On Wednesday, May 07, 2014 7:15:43 pm John Nielsen wrote:
>> I am trying to solve a problem with amd64 FreeBSD virtual machines running on a Linux+KVM hypervisor. To be honest I'm not sure if the problem is in FreeBSD or the hypervisor, but I'm trying to rule out the OS first.
>>
>> The _second_ time FreeBSD boots in a virtual machine with more than one core, the boot hangs just before the kernel would normally print e.g. "SMP: AP CPU #1 Launched!" (The last line on the console is "usbus0: 12Mbps Full Speed USB v1.0", but the problem persists even without USB). The VM will boot fine a first time, but running either "shutdown -r now" OR "reboot" will lead to a hung second boot. Stopping and starting the host qemu-kvm process is the only way to continue.
...
> Following up on the off chance anyone else is interested. I installed -HEAD on a host that was having the problem ("v2" Xeon CPU) and ran a FreeBSD 9 VM under bhyve. The problem did _not_ persist. That's not entirely conclusive but it does point the finger at Qemu a bit more strongly. I have filed a bug with them:
> https://bugs.launchpad.net/qemu/+bug/1329956
With some help from the Qemu and KVM folks I've finally made some headway. The salient difference between the working and non-working CPUs above seems to be support for APIC virtualization. Loading the intel_kvm module (on the Linux host) with "enable_apicv=N" works around the reboot problem I've been having.
Since this now looks like a Linux KVM bug I won't follow up here any more, but I wanted to wrap up the thread for the archives.
JN
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7D960087-1839-4295-AEE3-5D8F60D9D710>