Date: Tue, 20 Feb 2018 23:58:56 +0100 From: "Fabian Freyer" <fabian.freyer@physik.tu-berlin.de> To: freebsd-virtualization@freebsd.org Subject: VMX exit reason=33 and general userboot.so questions Message-ID: <BA57D57B-08B0-45DB-B200-C5C31994AAC7@physik.tu-berlin.de>
next in thread | raw e-mail | index | archive | help
--=_MailMate_36086495-52E9-42F0-92E1-29907CBCFF01_=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi!
I=E2=80=99m currently writing a userboot.so-compatible boot loader [1] an=
d am slowly getting to a point were I want to do some testing with runnin=
g test kernels in bhyve.
At the moment, I=E2=80=99m getting the following error after loading my k=
ernel:
---8< snip
vm exit[0]
reason VMX
rip 0x000000000010000c
inst_length 0
status 0
exit_reason 33
qualification 0x0000000000000000
inst_type 0
inst_error 0
[1] [PID] abort bhyve -H -P -s 0,hostbridge -s 31,lpc -c 1 -m 128=
M testing
--->8 snap
This is the register state I=E2=80=99m setting up before running bhyve:
---8< snip
efer[0] 0x0000000000000000
cr0[0] 0x0000000000000021
cr3[0] 0x0000000000000000
cr4[0] 0x0000000000000000
dr7[0] 0x0000000000000000
rsp[0] 0x0000000000000000
rip[0] 0x000000000010000c
rax[0] 0x000000002badb002
rbx[0] 0x0000000000100952
rcx[0] 0x0000000000000000
rdx[0] 0x0000000000000000
rsi[0] 0x0000000000000000
rdi[0] 0x0000000000000000
rbp[0] 0x0000000000000000
r8[0] 0x0000000000000000
r9[0] 0x0000000000000000
r10[0] 0x0000000000000000
r11[0] 0x0000000000000000
r12[0] 0x0000000000000000
r13[0] 0x0000000000000000
r14[0] 0x0000000000000000
r15[0] 0x0000000000000000
rflags[0] 0x0000000000000002
ds desc[0] 0x0000000000000000/0xffffffff/0x0000c093
es desc[0] 0x0000000000000000/0xffffffff/0x0000c093
fs desc[0] 0x0000000000000000/0xffffffff/0x0000c093
gs desc[0] 0x0000000000000000/0xffffffff/0x0000c093
ss desc[0] 0x0000000000000000/0xffffffff/0x0000c093
cs desc[0] 0x0000000000000000/0xffffffff/0x0000c09b
tr desc[0] 0x0000000000000000/0x00000000/0x00000000
ldtr desc[0] 0x0000000000000000/0x00000000/0x00000000
gdtr[0] 0x0000000000000000/0x00000000
idtr[0] 0x0000000000000000/0x00000000
cs[0] 0x0000
ds[0] 0x0000
es[0] 0x0000
fs[0] 0x0000
gs[0] 0x0000
ss[0] 0x0000
tr[0] 0x0000
ldtr[0] 0x0000
[... omitted some, not sure if relevant]
--->8 snap
And here=E2=80=99s a diff of the register state before and after running =
bhyve:
---8< snip
--- before 2018-02-20 22:36:16.001919000 +0000
+++ after 2018-02-20 22:36:27.442941000 +0000
--- before 2018-02-20 22:36:16.001919000 +0000
+++ after 2018-02-20 22:36:27.442941000 +0000
@@ -56 +56 @@
-procbased_ctls[0] 0x00000000b5186572
+procbased_ctls[0] 0x00000000f51865f2
@@ -67 +67 @@
-host_cr3[0] 0x0000000000000000
+host_cr3[0] 0x0000000389cac09a
@@ -101,4 +101,4 @@
-exit_reason[0] 0
-rtc nvram[000]: 0x05
-rtc time 0x5: Thu Jan 01 00:00:05 1970
-Capability "hlt_exit" is not set on vcpu 0
+exit_reason[0] 0x80000021
+rtc nvram[000]: 0x26
+rtc time 0x5a8ca2ea: Tue Feb 20 22:36:26 2018
+Capability "hlt_exit" is set on vcpu 0
@@ -106 +106 @@
-Capability "pause_exit" is not set on vcpu 0
+Capability "pause_exit" is set on vcpu 0
@@ -109 +109 @@
-active cpus: (none)
+active cpus: 0
@@ -125 +125 @@
-number of vm exits for unknown reason 0
+number of vm exits for unknown reason 1
@@ -128 +128 @@
-number of vm exits handled in userspace 0
+number of vm exits handled in userspace 1
@@ -131 +131 @@
-vcpu total runtime 0
+vcpu total runtime 11904
@@ -165,3 +165,3 @@
-Number of vpid invalidations done 0
-vcpu migration across host cpus 0
-total number of vm exits 0
+Number of vpid invalidations done 1
+vcpu migration across host cpus 1
+total number of vm exits 1
--->8 snap
The code at that RIP point also looks correct:
xxd -s 0x000000000010000c -l 5 /dev/vmm/testing
0010000c: bc00 2000 00
Which decompiles to:
0010000c: bc00200000 mov esp,0x2000
I=E2=80=99ve been looking at what userboot.so, grub2-bhyve and vm_setup_f=
reebsd_registers is doing. However, I left all registers that don=E2=80=99=
t have a defined state in the multiboot specification [2] in the state th=
ey are.
How would I best start debugging this?
Fabian
[1] https://github.com/fabianfreyer/bhyve-multiboot/tree/multiboot/info
[2] https://www.gnu.org/software/grub/manual/multiboot/multiboot.html
--=_MailMate_36086495-52E9-42F0-92E1-29907CBCFF01_=
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename=signature.asc
Content-Type: application/pgp-signature; name=signature.asc
-----BEGIN PGP SIGNATURE-----
iQJWBAEBCgBAFiEEX6JoxdmEemcFacQZmealkcs85+YFAlqMqDAiHGZhYmlhbi5m
cmV5ZXJAcGh5c2lrLnR1LWJlcmxpbi5kZQAKCRCZ5qWRyzzn5jimEADKTpWioyeS
b5HMGfrwKvMwQHA73KKTE86c7Wo1poRuTSKxqAT4WeDio211I5uALJueyCHkFzWf
xyI4XoPw97Ueq93y1Kx+/hjxU1/u8Fy64Jk/I9Nt0DA80hCTULzb4m+jl0tqPRNM
Z/4JIQWu/RHTlyX4wHFg4w7vfobvg60X/qqXWjm7uaec5eQEgJdNpe1/x4wuyZ5D
OgDFoks8gR/QUaXu3A9onc8lw0VAr59HyHWb+kty0QCHl1RKX7QDEPoqJ6lbt7f/
hfzvqi4F/pRHdy7a8CAz0iqUMsMfWdT5t+jZbsB2b3MvG3ZEa2UMX3uO8N38I5TP
MEVxPkujR4hM4Yq/S0FpZD3R/5UFbg9d9zXJopI9byv8+gs1mH3d4GETn3DsovWw
dv44QMKhOw8l0zmArX4qpGXzq+85CFo0sQ2vgCjdVqDCVgJZxpyGRxfYy42Gajpx
PS/6scHL1CCRCAKsJrYHP3rBd+E94J1MCfo71/OVa4ppUnE1oCmu1KF0aoVCAKb5
OGDPdsfvvkCYbTEcWQ8KQwEFMBcFFHuK+stIN8k+J3E3v25lle9M5MQWXhX7DCxh
1QANGZwM9/iJne0QbfzwS2z/3gMZf0T1rv70aEZ1lhqGW1/1K6ZkL77xUE2x/7cJ
kf+8ILy/Ry09LlkOCjWIkQx5pWXFnsTYzA==
=y/K5
-----END PGP SIGNATURE-----
--=_MailMate_36086495-52E9-42F0-92E1-29907CBCFF01_=--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BA57D57B-08B0-45DB-B200-C5C31994AAC7>