Date: Mon, 22 Jun 2015 14:14:13 +0300 From: Andriy Gapon <avg@FreeBSD.org> To: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@FreeBSD.org> Subject: bhyve: centos 7.1 with multiple virtual processors Message-ID: <5587EE05.2020001@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
If I run a CentOS 7.1 VM with more than one CPU more often than not it would
hang on startup and bhyve would start spinning.
The following are the last messages seen in the VM:
Switching to clocksource hpet
------------[ cut here ]------------
WARNING: at kernel/time/clockevents.c:239 clockevents_program_event+0xdb/0xf0()
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.10.0-229.4.2.el7.x86_64 #1
Hardware name: BHYVE, BIOS 1.00 03/14/2014
0000000000000000 00000000cab5bdb6 ffff88003fc03e08 ffffffff81604eaa
ffff88003fc03e40 ffffffff8106e34b 80000000000f423f 80000000000f423f
ffffffff81915440 0000000000000000 0000000000000000 ffff88003fc03e50
Call Trace:
<IRQ> [<ffffffff81604eaa>] dump_stack+0x19/0x1b
[<ffffffff8106e34b>] warn_slowpath_common+0x6b/0xb0
[<ffffffff8106e49a>] warn_slowpath_null+0x1a/0x20
[<ffffffff810ce6eb>] clockevents_program_event+0xdb/0xf0
[<ffffffff810cf211>] tick_handle_periodic_broadcast+0x41/0x50
[<ffffffff81016525>] timer_interrupt+0x15/0x20
[<ffffffff8110b5ee>] handle_irq_event_percpu+0x3e/0x1e0
[<ffffffff8110b7cd>] handle_irq_event+0x3d/0x60
[<ffffffff8110e467>] handle_edge_irq+0x77/0x130
[<ffffffff81015cff>] handle_irq+0xbf/0x150
[<ffffffff81077df7>] ? irq_enter+0x17/0xa0
[<ffffffff816172af>] do_IRQ+0x4f/0xf0
[<ffffffff8160c4ad>] common_interrupt+0x6d/0x6d
<EOI> [<ffffffff8126e359>] ? selinux_inode_alloc_security+0x59/0xa0
[<ffffffff811de58f>] ? __d_instantiate+0xbf/0x100
[<ffffffff811de56f>] ? __d_instantiate+0x9f/0x100
[<ffffffff811de60d>] d_instantiate+0x3d/0x70
[<ffffffff8124d748>] debugfs_mknod.isra.5.part.6.constprop.15+0x98/0x130
[<ffffffff8124da82>] __create_file+0x1c2/0x2c0
[<ffffffff81a6c6bf>] ? set_graph_function+0x1f/0x1f
[<ffffffff8124dbcb>] debugfs_create_dir+0x1b/0x20
[<ffffffff8112c1ce>] tracing_init_dentry_tr+0x7e/0x90
[<ffffffff8112c250>] tracing_init_dentry+0x10/0x20
[<ffffffff81a6c6d2>] ftrace_init_debugfs+0x13/0x1fd
[<ffffffff81a6c6bf>] ? set_graph_function+0x1f/0x1f
[<ffffffff810020e8>] do_one_initcall+0xb8/0x230
[<ffffffff81a45203>] kernel_init_freeable+0x18b/0x22a
[<ffffffff81a449db>] ? initcall_blacklist+0xb0/0xb0
[<ffffffff815f33f0>] ? rest_init+0x80/0x80
[<ffffffff815f33fe>] kernel_init+0xe/0xf0
[<ffffffff81614d3c>] ret_from_fork+0x7c/0xb0
[<ffffffff815f33f0>] ? rest_init+0x80/0x80
---[ end trace d5caa1cab8e7e98d ]---
At the same time sometimes there is one or more of spurious NMIs on the _host_
system:
NMI ISA c, EISA ff
NMI ... going to debugger
bhyve seems to spin here:
vmm.ko`svm_vmrun+0x894
vmm.ko`vm_run+0xbb7
vmm.ko`vmmdev_ioctl+0x5a4
kernel`devfs_ioctl_f+0x13b
kernel`kern_ioctl+0x1e1
kernel`sys_ioctl+0x16a
kernel`amd64_syscall+0x3ca
kernel`0xffffffff8088997b
(kgdb) list *svm_vmrun+0x894
0xffffffff813c9194 is in svm_vmrun
(/usr/src/sys/modules/vmm/../../amd64/vmm/amd/svm.c:1895).
1890
1891 static __inline void
1892 enable_gintr(void)
1893 {
1894
1895 __asm __volatile("stgi");
1896 }
1897
1898 /*
1899 * Start vcpu with specified RIP.
--
Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5587EE05.2020001>