Date: Fri, 16 Sep 2016 23:54:06 -0600 From: Trent Thompson <trentnthompson@gmail.com> To: FreeBSD virtualization <freebsd-virtualization@freebsd.org> Subject: High vCPU Counts in bhyve Message-ID: <CAE7bWBoH0JWR5m5dFpEW=fg8XUaKjPWak7yTra_PWnJ8GCQLbA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Good day to all!
I was recently given the hint that I could edit vmm.h to allow for more
than 16 vCPUs in bhyve, so I did just that to see what would happen. I
understand it's not supported, just wanted to try it out. When trying to
boot FreeBSD 12-CURRENT with anything up to 25 vCPUs the guest will boot
and run just fine. At 26+ vCPUs the guest OS will panic at boot time.
I have posted all of these outputs and more to GitHub [1].
I started by installing FreeBSD with 16 vCPUs and then incrementing until
it wouldn't go any further. You can find the dmesg.boot output of the guest
at 16 vCPUs on GitHub [2].
Here is the relevant line in /usr/src/sys/amd64/include/vmm.h:
#define VM_MAXCPU 32 /* maximum virtual cpus */
Here is what appears in the serial console when attempting to boot with 26
vCPUs (The full output can also be found on GitHub [3]):
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x60
fault code = supervisor write data, page not present
instruction pointer = 0x20:0xffffffff80ff3859
stack pointer = 0x28:0xffffffff821c19d0
frame pointer = 0x28:0xffffffff821c1a50
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 0 (swapper)
[ thread pid 0 tid 100000 ]
Stopped at acpi_install_wakeup_handler+0x1d9: movq %rdi,0x60(%r14)
db>
If anyone has any hints on how to get more information from this or ideas
on the apparent APIC error would be greatly appreciated.
Thanks and happy hacking!
-Trent
@pr1ntf
[1] https://gist.github.com/pr1ntf/270efd6ec3fe376e68d1f9dad2deef31
[2]
https://gist.githubusercontent.com/pr1ntf/270efd6ec3fe376e68d1f9dad2deef31/raw/09dde8362a5e488794e6d7c8b70bb0368b964b49/3-guest-dmesg.boot.txt
[3]
https://gist.githubusercontent.com/pr1ntf/270efd6ec3fe376e68d1f9dad2deef31/raw/09dde8362a5e488794e6d7c8b70bb0368b964b49/4-guest-26vcpucrash.txt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAE7bWBoH0JWR5m5dFpEW=fg8XUaKjPWak7yTra_PWnJ8GCQLbA>