From owner-freebsd-virtualization@freebsd.org Sun Mar 4 21:00:30 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7561CF322CD
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sun, 4 Mar 2018 21:00:30 +0000 (UTC)
(envelope-from bugzilla-noreply@FreeBSD.org)
Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.ysv.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 15F1A6F26A
for <freebsd-virtualization@FreeBSD.org>; Sun, 4 Mar 2018 21:00:30 +0000 (UTC)
(envelope-from bugzilla-noreply@FreeBSD.org)
Received: from kenobi.freebsd.org (kenobi.freebsd.org
[IPv6:2001:1900:2254:206a::16:76])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 67C0814445
for <freebsd-virtualization@FreeBSD.org>; Sun, 4 Mar 2018 21:00:29 +0000 (UTC)
(envelope-from bugzilla-noreply@FreeBSD.org)
Received: from kenobi.freebsd.org ([127.0.1.118])
by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w24L0TUr076248
for <freebsd-virtualization@FreeBSD.org>; Sun, 4 Mar 2018 21:00:29 GMT
(envelope-from bugzilla-noreply@FreeBSD.org)
Received: (from bugzilla@localhost)
by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w24L0TtV076238
for freebsd-virtualization@FreeBSD.org; Sun, 4 Mar 2018 21:00:29 GMT
(envelope-from bugzilla-noreply@FreeBSD.org)
Message-Id: <201803042100.w24L0TtV076238@kenobi.freebsd.org>
X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to
bugzilla-noreply@FreeBSD.org using -f
From: bugzilla-noreply@FreeBSD.org
To: freebsd-virtualization@FreeBSD.org
Subject: Problem reports for freebsd-virtualization@FreeBSD.org that need
special attention
Date: Sun, 4 Mar 2018 21:00:29 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.25
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 04 Mar 2018 21:00:30 -0000
To view an individual PR, use:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id).
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.
Status | Bug Id | Description
------------+-----------+---------------------------------------------------
New | 212820 | FreeBSD 10-STABLE from latest HEAD and 11-RELEASE
1 problems total for which you should take action.
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 06:52:22 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9A04F343FA
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 06:52:22 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de
[130.149.50.25])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 9C2DC8594F
for <freebsd-virtualization@freebsd.org>; Tue, 6 Mar 2018 06:52:19 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from [192.168.119.1] (wlan-141-23-171-244.tubit.tu-berlin.de
[141.23.171.244])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 99AEF61FA1;
Tue, 6 Mar 2018 06:45:12 +0000 (UTC)
From: "Fabian Freyer" <fabian.freyer@physik.tu-berlin.de>
To: freebsd-virtualization@freebsd.org, rumpkernel-users@freelists.org
Subject: rumpkernel and bhyve: triple faults
Date: Tue, 06 Mar 2018 07:45:00 +0100
X-Mailer: MailMate (1.10r5443)
Message-ID: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
MIME-Version: 1.0
Content-Type: multipart/signed;
boundary="=_MailMate_F308673C-73D9-40FB-8BCF-C62E1950BE85_=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 06:52:23 -0000
This is an OpenPGP/MIME signed message (RFC 3156 and 4880).
--=_MailMate_F308673C-73D9-40FB-8BCF-C62E1950BE85_=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hello lists,
I=E2=80=99m currently playing around with getting rump kernels built with=
rumpkernel(7) running on FreeBSD=E2=80=99s bhyve(4). I=E2=80=99m using a=
custom boot loader [1] which builds on some patches to bhyveload / user =
boot [2].
To test, I=E2=80=99m using a simple =E2=80=9Chelloer=E2=80=9D unikernel f=
rom the tutorial [3]:
#include <stdio.h>
#include <unistd.h>
int
main()
{
printf("Hello, Rumprun ... I'm feeling tired\n");
sleep(2);
printf("much better!\n");
return 0;
}
I=E2=80=99m compiling this for the hw_virtio platform:
x86_64-rumprun-netbsd-gcc -o helloer-rumprun helloer.c
rumprun-bake hw_virtio helloer-rumprun.elf helloer-rumprun
Starting this kernel in bhyve gives me the following VM Exit:
=46rom the Intel SDM 3D, Appendix C [4], VM Exit 2 is a triple fault.
To further debug this, I=E2=80=99m using the ktr(4) support introduced in=
FreeBSD r276098 [5].
I=E2=80=99ve appended a full KTR trace further down in my mail, but I=E2=80=
=99ll go along the trace here with some commentary in execution order, an=
d attempt to annotate the source code lines from rumpkernel:
0x104000 is the entry address given in rump kernel=E2=80=99s multi boot h=
eader.
0000000000104000 <_start>:
632 0 350886885990 vm testing[0]: Setting nextrip to 0x104000
636 0 350886961656 vm testing[0]: Resume execution at 0x104000
This is the cpuid instruction in x86_cpuid which is called from hyperviso=
r_detect:
rumprun/platform/hw/arch/x86/x86_subr.c: 91
0000000000102f90 <x86_cpuid>:
102f90: 49 89 d2 mov r10,rdx
102f93: 49 89 c9 mov r9,rcx
102f96: 53 push rbx
102f97: 89 f8 mov eax,edi
102f99: 0f a2 cpuid
rumprun/platform/hw/arch/x86/hypervisor.c: 38
637 0 350887061536 vm testing[0]: cpuid 0,0x100fd8
638 0 350887074874 vm testing[0]: handled cpuid vmexit at 0x102f=
99
639 0 350887079146 vm testing[0]: Resume execution at 0x102f9b
rumprun/platform/hw/arch/x86/hypervisor.c: 40
640 0 350887102692 vm testing[0]: cpuid 0x1,0x100fd8
641 0 350887117424 vm testing[0]: handled cpuid vmexit at 0x102f=
99
642 0 350887121556 vm testing[0]: Resume execution at 0x102f9b
rumprun/platform/hw/arch/x86/hypervisor.c: 51
643 0 350887143374 vm testing[0]: cpuid 0x40000000,0x100fd8
644 0 350887145218 vm testing[0]: handled cpuid vmexit at 0x102f=
99
645 0 350887149146 vm testing[0]: Resume execution at 0x102f9b
The next vexit is in cons_init, directly after the call to hypervisor_det=
ect.
0000000000102a50 <cons_init>:
102a50: 53 push rbx
102a51: e8 ea 0a 00 00 call 103540 <hypervisor_detect>=
102a56: 66 83 3d a4 d5 ef ff cmp WORD PTR [rip-0x102a5c],0x=
0 # 2 <current_lwp+0x2>
Due to compiler optimisations, the check here isn=E2=80=99t the
(hypervisor =3D=3D HYPERVISOR_XEN) check directly after the call to hyper=
visor_detect,
but the check (bios_crtc_base =3D=3D 0) in
rumprun/platform/hw/arch/x86/cons.c:59:
649 0 350887182668 vm testing[0]: handled exception vmexit at 0x=
102a56
Therefore, I=E2=80=99m assuming this is the origin of the fault.
Tracking down bios_crtc_base, I find that it=E2=80=99s loaded in
rumprun/platform/hw/arch/amd64/locore.S:70:
/* save BIOS data area values */
movw BIOS_COM1_BASE, %bx
movw %bx, bios_com1_base
movw BIOS_CRTC_BASE, %bx
movw %bx, bios_crtc_base
Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking the b=
hyve
device node in /dev/vmm with xxd(1), I find the words at these addresses =
to be
Uninitialised:
00000400: 0000 ..
00000483: 0000 ..
I=E2=80=99m not sure where to go from here. Is this a bug in bhyve(4), sh=
ould these
values be initialised somehow, or should I patch rumpkernel(7) to skip th=
is check
when running on bhyve(4)?
Fabian
Full KTR trace:
index cpu timestamp trace
------ --- ---------------- -----
673 0 350887364560 vm testing[0]: vcpu state changed from frozen=
to idle
672 0 350887363766 vm testing[0]: retu 0/14
671 0 350887362932 vm testing[0]: All vcpus suspended
670 0 350887357212 vm testing[0]: vcpu state changed from runnin=
g to frozen
669 0 350887350444 vm testing[0]: returning from vmx_run: exitco=
de 14
668 0 350887348806 vm testing: virtual machine successfully susp=
ended 4
667 0 350887347712 vm testing[0]: triple fault: info1(0x80000b08=
), info2(0x80000b0e)
666 0 350887347590 vm testing[0]: Exception 14 delivered: 0x8000=
0b0e
665 0 350887346312 vm testing[0]: handled exception vmexit at 0x=
102a56
664 0 350887346178 vm testing[0]: Exception 14 pending
663 0 350887346060 vm testing[0]: Setting intr_shadow to 0 succe=
eded
662 0 350887341876 vm testing[0]: Reflecting exception 14/0 into=
the guest
661 0 350887339570 vm testing[0]: vm_exit_intinfo: info1(0x80000=
b08)
660 0 350887315326 vm testing[0]: Resume execution at 0x102a56
659 0 350887311168 vm testing[0]: vm_entry_intinfo: info1(0x8000=
0b0e), info2(0x80000b0e), retinfo(0x80000b08)
658 0 350887310988 vm testing[0]: Exception 14 delivered: 0x8000=
0b0e
657 0 350887309700 vm testing[0]: handled exception vmexit at 0x=
102a56
656 0 350887309570 vm testing[0]: Exception 14 pending
655 0 350887309442 vm testing[0]: Setting intr_shadow to 0 succe=
eded
654 0 350887305126 vm testing[0]: Reflecting exception 14/0 into=
the guest
653 0 350887302436 vm testing[0]: vm_exit_intinfo: info1(0x80000=
b0e)
652 0 350887248280 vm testing[0]: Resume execution at 0x102a56
651 0 350887184160 vm testing[0]: vm_entry_intinfo: info1(0), in=
fo2(0x80000b0e), retinfo(0x80000b0e)
650 0 350887184040 vm testing[0]: Exception 14 delivered: 0x8000=
0b0e
649 0 350887182668 vm testing[0]: handled exception vmexit at 0x=
102a56
648 0 350887182374 vm testing[0]: Exception 14 pending
647 0 350887182240 vm testing[0]: Setting intr_shadow to 0 succe=
eded
646 0 350887176682 vm testing[0]: Reflecting exception 14/0 into=
the guest
645 0 350887149146 vm testing[0]: Resume execution at 0x102f9b
644 0 350887145218 vm testing[0]: handled cpuid vmexit at 0x102f=
99
643 0 350887143374 vm testing[0]: cpuid 0x40000000,0x100fd8
642 0 350887121556 vm testing[0]: Resume execution at 0x102f9b
641 0 350887117424 vm testing[0]: handled cpuid vmexit at 0x102f=
99
640 0 350887102692 vm testing[0]: cpuid 0x1,0x100fd8
639 0 350887079146 vm testing[0]: Resume execution at 0x102f9b
638 0 350887074874 vm testing[0]: handled cpuid vmexit at 0x102f=
99
637 0 350887061536 vm testing[0]: cpuid 0,0x100fd8
636 0 350886961656 vm testing[0]: Resume execution at 0x104000
635 0 350886900428 vm testing[0]: vcpu state changed from frozen=
to running
634 0 350886895942 vm testing[0]: vcpu state changed from idle t=
o frozen
633 0 350886886630 vm testing[0]: vcpu state changed from frozen=
to idle
632 0 350886885990 vm testing[0]: Setting nextrip to 0x104000
631 0 350886802596 vm testing[0]: vcpu state changed from idle t=
o frozen
630 3 350886414966 vm testing[0]: vcpu state changed from frozen=
to idle
629 3 350886414184 vm testing[0]: activated
628 3 350886412730 vm testing[0]: vcpu state changed from idle t=
o frozen
627 3 350886137831 vm testing[0]: vcpu state changed from frozen=
to idle
626 3 350885989045 vm testing[0]: vcpu state changed from idle t=
o frozen
625 3 350885773367 vm testing: RTC time set to 0x5a9e154e
624 3 350885773017 vm testing: Updating RTC base uptime from 0x3=
1c2762314 to 0x8bd6b051a2
623 3 350885772543 vm testing: Updating RTC secs from 0x5a9e14f4=
to 0x5a9e154e
622 3 350885017699 vm testing: RTC nvram write 0 to offset 0x5d
621 3 350885015953 vm testing: RTC nvram write 0 to offset 0x5c
620 3 350885013951 vm testing: RTC nvram write 0 to offset 0x5b
619 3 350885010615 vm testing: RTC nvram write 0x1 to offset 0x3=
5
618 3 350885008163 vm testing: RTC nvram write 0 to offset 0x34
[1] https://github.com/fubarnetes/bhyve-multiboot
[2] https://reviews.FreeBSD.org/D14473
[3] https://github.com/rumpkernel/wiki/wiki/Tutorial%3A-Building-Rumprun-=
Unikernels#2-building-applications
[4] https://www.intel.com/content/dam/www/public/us/en/documents/manuals/=
64-ia-32-architectures-software-developer-vol-3d-part-4-manual.pdf
[5] https://svnweb.freebsd.org/base?view=3Drevision&revision=3D276098
--=_MailMate_F308673C-73D9-40FB-8BCF-C62E1950BE85_=
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename=signature.asc
Content-Type: application/pgp-signature; name=signature.asc
-----BEGIN PGP SIGNATURE-----
iQJWBAEBCgBAFiEEX6JoxdmEemcFacQZmealkcs85+YFAlqeOOwiHGZhYmlhbi5m
cmV5ZXJAcGh5c2lrLnR1LWJlcmxpbi5kZQAKCRCZ5qWRyzzn5owLD/915dsZZNlz
stLP2tVgPifGtY8yCD5U/WfpWznOxzDmeyBlJszTdGoKRdbTDoZHSHsEype/PjAw
fn6cQKHChmsW7GwZSMZ/ETgu9xYzPaS9tDQPpRNcRpku0Npvw4Kifv4QzSibwP7e
6vfTFk59SoCkScinVXuKbjvylwnEtxbzIxWvSo24aXebcEdqSKsX8zKemGra00XM
S9NSny4Sa5VOpuMsPMhuvwexEP3CJW888h5/3xPNU9N5/vbLc1FDHarS2QNQF8fB
iy1F0lyIEP4iAzwO6STLtJv+tMiZh//2V/o6tKxFQMtgDlc6bb/vw40OlG/Z8Es/
Z9Gbeg6YZVdrxLGVE5sEgiC4BHkmgbFGp1yqkA23OMF3nFqTiLulftUun90S66uT
Vy2vor/tnVH6FrhnZ+f8Xh2s1movStz6DpEUFlZfmOxE6Z5lftn5089KYRZeMs++
i4P5x9ajhIhGNeqkKIeeWgjZ7CNh+CcG/ZJqvWmcln/j8/lPSIktCYSHPbCOYdzb
1RigTNstbX9I+2fZU4caaMlxcDE712zdt4HA55j7UbTPjeMRcXah1H/szPBcHWQK
O923YtXpiYRQaRTIAzt9++BWtjU60P2YQ4KQnZ2hzs6DlUKBSr33FFU4xUsFUt6Q
FrBzA4skxpa29GEsgeL2sreOREmNY6aQwg==
=qScN
-----END PGP SIGNATURE-----
--=_MailMate_F308673C-73D9-40FB-8BCF-C62E1950BE85_=--
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 08:06:34 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BD71F3CE41
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 08:06:34 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id B464968F8C
for <freebsd-virtualization@freebsd.org>; Tue, 6 Mar 2018 08:06:33 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2686TF1047820;
Tue, 6 Mar 2018 00:06:29 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2686Shu047819;
Tue, 6 Mar 2018 00:06:28 -0800 (PST) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803060806.w2686Shu047819@pdx.rh.CN85.dnsmgr.net>
Subject: Re: rumpkernel and bhyve: triple faults
In-Reply-To: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
To: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Date: Tue, 6 Mar 2018 00:06:28 -0800 (PST)
CC: freebsd-virtualization@freebsd.org, rumpkernel-users@freelists.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 08:06:34 -0000
> Hello lists,
>
> I?m currently playing around with getting rump kernels built with rumpkernel(7) running on FreeBSD?s bhyve(4). I?m using a custom boot loader [1] which builds on some patches to bhyveload / user boot [2].
>
> To test, I?m using a simple ?helloer? unikernel from the tutorial [3]:
>
... excelent discription of your debug process removed for breif reply ...
> Due to compiler optimisations, the check here isn?t the
> (hypervisor == HYPERVISOR_XEN) check directly after the call to hypervisor_detect,
> but the check (bios_crtc_base == 0) in
bios_crtc_base would be part of the isa legacy vga
controller card. Bhyve does not, at this time, or
in the near future expect to have, support for this
legacy device.
> rumprun/platform/hw/arch/x86/cons.c:59:
> 649 0 350887182668 vm testing[0]: handled exception vmexit at 0x102a56
>
> Therefore, I?m assuming this is the origin of the fault.
>
> Tracking down bios_crtc_base, I find that it?s loaded in
> rumprun/platform/hw/arch/amd64/locore.S:70:
>
> /* save BIOS data area values */
> movw BIOS_COM1_BASE, %bx
> movw %bx, bios_com1_base
> movw BIOS_CRTC_BASE, %bx
> movw %bx, bios_crtc_base
>
> Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking the bhyve
> device node in /dev/vmm with xxd(1), I find the words at these addresses to be
> Uninitialised:
>
> 00000400: 0000 ..
> 00000483: 0000 ..
Typo here, should this be 00000463?
> I?m not sure where to go from here. Is this a bug in bhyve(4),
No, it is not a bug, it is an unimplemented device.
> should these
> values be initialised somehow, or should I patch rumpkernel(7) to skip this check
> when running on bhyve(4)?
rumpkernel is assuming or requiring the presence of legacy isa hardware,
it should probably be taught that this may not exist. You could simply
skip this check, but I expect you would then have a harder to find
failure later when it tries to use the hardware it expects to be
there.
>
> Fabian
... Full KTR trace removed for breif reply ...
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 08:28:40 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39D1BF3F442
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 08:28:40 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id A101F6A389
for <freebsd-virtualization@freebsd.org>; Tue, 6 Mar 2018 08:28:39 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w268Sbrm047943;
Tue, 6 Mar 2018 00:28:37 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w268SbZX047942;
Tue, 6 Mar 2018 00:28:37 -0800 (PST) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803060828.w268SbZX047942@pdx.rh.CN85.dnsmgr.net>
Subject: Re: rumpkernel and bhyve: triple faults
In-Reply-To: <201803060806.w2686Shu047819@pdx.rh.CN85.dnsmgr.net>
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Date: Tue, 6 Mar 2018 00:28:37 -0800 (PST)
CC: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>,
rumpkernel-users@freelists.org, freebsd-virtualization@freebsd.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 08:28:40 -0000
> > Hello lists,
> >
> > I?m currently playing around with getting rump kernels built with rumpkernel(7) running on FreeBSD?s bhyve(4). I?m using a custom boot loader [1] which builds on some patches to bhyveload / user boot [2].
> >
> > To test, I?m using a simple ?helloer? unikernel from the tutorial [3]:
> >
> ... excelent discription of your debug process removed for breif reply ...
>
> > Due to compiler optimisations, the check here isn?t the
> > (hypervisor == HYPERVISOR_XEN) check directly after the call to hypervisor_detect,
> > but the check (bios_crtc_base == 0) in
>
> bios_crtc_base would be part of the isa legacy vga
> controller card. Bhyve does not, at this time, or
> in the near future expect to have, support for this
> legacy device.
I am wrong on this, the framebuffer device does
infact have support for the legacy i/o addresses
that this should point to. You should see the
"vgaconf" section of the FrameBuffer section
of the bhyve(8) manpage.
I believe you need to be running bhyve with the
uefi bios options, the with CMS version, and
have vgaconf=on to get your code to work as is.
> > rumprun/platform/hw/arch/x86/cons.c:59:
> > 649 0 350887182668 vm testing[0]: handled exception vmexit at 0x102a56
> >
> > Therefore, I?m assuming this is the origin of the fault.
> >
> > Tracking down bios_crtc_base, I find that it?s loaded in
> > rumprun/platform/hw/arch/amd64/locore.S:70:
> >
> > /* save BIOS data area values */
> > movw BIOS_COM1_BASE, %bx
> > movw %bx, bios_com1_base
> > movw BIOS_CRTC_BASE, %bx
> > movw %bx, bios_crtc_base
> >
> > Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking the bhyve
> > device node in /dev/vmm with xxd(1), I find the words at these addresses to be
> > Uninitialised:
> >
> > 00000400: 0000 ..
> > 00000483: 0000 ..
> Typo here, should this be 00000463?
>
>
> > I?m not sure where to go from here. Is this a bug in bhyve(4),
> No, it is not a bug, it is an unimplemented device.
>
> > should these
> > values be initialised somehow, or should I patch rumpkernel(7) to skip this check
> > when running on bhyve(4)?
> rumpkernel is assuming or requiring the presence of legacy isa hardware,
> it should probably be taught that this may not exist. You could simply
> skip this check, but I expect you would then have a harder to find
> failure later when it tries to use the hardware it expects to be
> there.
>
> >
> > Fabian
>
> ... Full KTR trace removed for breif reply ...
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 08:53:25 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B2C0F4136C
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 08:53:25 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de
[130.149.50.25])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id E24A26B818
for <freebsd-virtualization@freebsd.org>; Tue, 6 Mar 2018 08:53:24 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from [192.168.119.1] (wlan-141-23-171-244.tubit.tu-berlin.de
[141.23.171.244])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id EB56762069;
Tue, 6 Mar 2018 08:53:21 +0000 (UTC)
From: "Fabian Freyer" <fabian.freyer@physik.tu-berlin.de>
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Cc: rumpkernel-users@freelists.org, freebsd-virtualization@freebsd.org
Subject: Re: rumpkernel and bhyve: triple faults
Date: Tue, 06 Mar 2018 09:53:21 +0100
X-Mailer: MailMate (1.10r5443)
Message-ID: <256A0D22-46D6-44DA-9ACF-631FE981D0EC@physik.tu-berlin.de>
In-Reply-To: <201803060828.w268SbZX047942@pdx.rh.CN85.dnsmgr.net>
References: <201803060828.w268SbZX047942@pdx.rh.CN85.dnsmgr.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 08:53:25 -0000
On 6 Mar 2018, at 9:28, Rodney W. Grimes wrote:
>> bios_crtc_base would be part of the isa legacy vga
>> controller card. Bhyve does not, at this time, or
>> in the near future expect to have, support for this
>> legacy device.
>
> I am wrong on this, the framebuffer device does
> infact have support for the legacy i/o addresses
> that this should point to. You should see the
> "vgaconf" section of the FrameBuffer section
> of the bhyve(8) manpage.
>
> I believe you need to be running bhyve with the
> uefi bios options, the with CMS version, and
> have vgaconf=on to get your code to work as is.
For diskless multiboot kernels I’m going with a
separate userboot.so-compatible loader. Specifying
a UEFI bootrom implicitly resets the CPU.
(See usr.sbin/bhyve/bhyverun.c:960)
I think deciding to use the serial output (which is
what most of rumpkernel’s cons_init is doing) based
on the hypervisor is probably the right way to go.
Something similar is already done for XEN:
/*
* If running under Xen use the serial console.
*/
if (hypervisor == HYPERVISOR_XEN)
prefer_serial = 1;
>>> rumprun/platform/hw/arch/x86/cons.c:59:
>>> 649 0 350887182668 vm testing[0]: handled exception vmexit
>>> at 0x102a56
>>>
>>> Therefore, I?m assuming this is the origin of the fault.
>>>
>>> Tracking down bios_crtc_base, I find that it?s loaded in
>>> rumprun/platform/hw/arch/amd64/locore.S:70:
>>>
>>> /* save BIOS data area values */
>>> movw BIOS_COM1_BASE, %bx
>>> movw %bx, bios_com1_base
>>> movw BIOS_CRTC_BASE, %bx
>>> movw %bx, bios_crtc_base
>>>
>>> Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking
>>> the bhyve
>>> device node in /dev/vmm with xxd(1), I find the words at these
>>> addresses to be
>>> Uninitialised:
>>>
>>> 00000400: 0000 ..
>>> 00000483: 0000 ..
>> Typo here, should this be 00000463?
Yes, sorry about that.
Fabian
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 09:18:17 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D995CF4306B
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 09:18:16 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 48FF86C4F2
for <freebsd-virtualization@freebsd.org>; Tue, 6 Mar 2018 09:18:15 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w269ICbw048091;
Tue, 6 Mar 2018 01:18:12 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w269IAl1048090;
Tue, 6 Mar 2018 01:18:10 -0800 (PST) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803060918.w269IAl1048090@pdx.rh.CN85.dnsmgr.net>
Subject: Re: rumpkernel and bhyve: triple faults
In-Reply-To: <256A0D22-46D6-44DA-9ACF-631FE981D0EC@physik.tu-berlin.de>
To: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Date: Tue, 6 Mar 2018 01:18:10 -0800 (PST)
CC: rumpkernel-users@freelists.org, freebsd-virtualization@freebsd.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 09:18:17 -0000
> On 6 Mar 2018, at 9:28, Rodney W. Grimes wrote:
>
> >> bios_crtc_base would be part of the isa legacy vga
> >> controller card. Bhyve does not, at this time, or
> >> in the near future expect to have, support for this
> >> legacy device.
> >
> > I am wrong on this, the framebuffer device does
> > infact have support for the legacy i/o addresses
> > that this should point to. You should see the
> > "vgaconf" section of the FrameBuffer section
> > of the bhyve(8) manpage.
> >
> > I believe you need to be running bhyve with the
> > uefi bios options, the with CMS version, and
> > have vgaconf=on to get your code to work as is.
>
> For diskless multiboot kernels I?m going with a
> separate userboot.so-compatible loader. Specifying
> a UEFI bootrom implicitly resets the CPU.
> (See usr.sbin/bhyve/bhyverun.c:960)
Well in that case my original claim that there
is not a legacy isa vga device avaliable would
be correct for this environment, and you should
just eliminate anything that tries to use it,
or make it understand that this device may not
exist. I am not sure if bhyve maps any of these
legacy addresses if your not using bhyveloader
or uefi bios code. 0x400 and 0x483 being
unmapped could lead to your tripple fault.
>
> I think deciding to use the serial output (which is
> what most of rumpkernel?s cons_init is doing) based
> on the hypervisor is probably the right way to go.
> Something similar is already done for XEN:
>
> /*
> * If running under Xen use the serial console.
> */
> if (hypervisor == HYPERVISOR_XEN)
> prefer_serial = 1;
>
> >>> rumprun/platform/hw/arch/x86/cons.c:59:
> >>> 649 0 350887182668 vm testing[0]: handled exception vmexit
> >>> at 0x102a56
> >>>
> >>> Therefore, I?m assuming this is the origin of the fault.
> >>>
> >>> Tracking down bios_crtc_base, I find that it?s loaded in
> >>> rumprun/platform/hw/arch/amd64/locore.S:70:
> >>>
> >>> /* save BIOS data area values */
> >>> movw BIOS_COM1_BASE, %bx
> >>> movw %bx, bios_com1_base
> >>> movw BIOS_CRTC_BASE, %bx
> >>> movw %bx, bios_crtc_base
> >>>
> >>> Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking
> >>> the bhyve
> >>> device node in /dev/vmm with xxd(1), I find the words at these
> >>> addresses to be
> >>> Uninitialised:
> >>>
> >>> 00000400: 0000 ..
> >>> 00000483: 0000 ..
> >> Typo here, should this be 00000463?
> Yes, sorry about that.
>
> Fabian
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org"
>
>
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 09:50:28 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A08DF45117
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 09:50:28 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de
[130.149.50.25])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 2ADEE6D935;
Tue, 6 Mar 2018 09:50:24 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from [192.168.119.1] (wlan-141-23-171-244.tubit.tu-berlin.de
[141.23.171.244])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 215CE61FA1;
Tue, 6 Mar 2018 09:50:24 +0000 (UTC)
From: "Fabian Freyer" <fabian.freyer@physik.tu-berlin.de>
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Cc: freebsd-virtualization@freebsd.org, tychon@Freebsd.org
Subject: Re: rumpkernel and bhyve: triple faults
Date: Tue, 06 Mar 2018 10:50:22 +0100
X-Mailer: MailMate (1.10r5443)
Message-ID: <43C0E566-19BC-4AB9-96BD-EE9608DA63F5@physik.tu-berlin.de>
In-Reply-To: <201803060918.w269IAl1048090@pdx.rh.CN85.dnsmgr.net>
References: <201803060918.w269IAl1048090@pdx.rh.CN85.dnsmgr.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 09:50:28 -0000
(un-CC’ing rumpkernel-users@, since this part of the
discussion is getting very bhyve-specific)
On 6 Mar 2018, at 10:18, Rodney W. Grimes wrote:
>> On 6 Mar 2018, at 9:28, Rodney W. Grimes wrote:
>>
>>>> bios_crtc_base would be part of the isa legacy vga
>>>> controller card. Bhyve does not, at this time, or
>>>> in the near future expect to have, support for this
>>>> legacy device.
>>>
>>> I am wrong on this, the framebuffer device does
>>> infact have support for the legacy i/o addresses
>>> that this should point to. You should see the
>>> "vgaconf" section of the FrameBuffer section
>>> of the bhyve(8) manpage.
>>>
>>> I believe you need to be running bhyve with the
>>> uefi bios options, the with CMS version, and
>>> have vgaconf=on to get your code to work as is.
>>
>> For diskless multiboot kernels I?m going with a
>> separate userboot.so-compatible loader. Specifying
>> a UEFI bootrom implicitly resets the CPU.
>> (See usr.sbin/bhyve/bhyverun.c:960)
>
> Well in that case my original claim that there
> is not a legacy isa vga device avaliable would
> be correct for this environment, and you should
> just eliminate anything that tries to use it,
> or make it understand that this device may not
> exist. I am not sure if bhyve maps any of these
> legacy addresses if your not using bhyveloader
> or uefi bios code. 0x400 and 0x483 being
> unmapped could lead to your tripple fault.
According to lib/libvmmapi/vmmapi.c:409, (and some
code / comments in grub2-bhyve), bhyve maps up to
two memory segments: [0, lowmem) and if more than
3 GiB of memory is given, [4GiB, 4GiB + highmem).
According to the comments in
grub2-bhyve/grub-core/kern/emu/bhyve_hostif.c:149
The area from 640kiB to 1MiB is reserved as the
vga hole and BIOS.
0x400 and 0x463 are in the bios data area [1], which is
mapped, but zeroed out:
40:00 word COM1 port address
40:63 word Base port address for active 6845 CRT controller
I’m not sure whether it might not be useful for
bhyve to populate the bios data area with the
information that it does have available (e.g. serial
ports).
On a related note, I saw that there is a relevant
GSoC Project idea [2] to be mentored by tychon@ (CC’d)
that mentions VGA/VESA device emulation independent
of the UEFI frame buffer. I’ll try to look into that
more, and possibly submit a proposal :).
Fabian
[1] http://stanislavs.org/helppc/bios_data_area.html
[2]
https://wiki.freebsd.org/SummerOfCodeIdeas#VGA_emulation_improvements_for_bhyve
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 15:15:49 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7D93F39C57
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 15:15:49 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from vito.onthenet.com.au (vito.onthenet.com.au [203.22.124.72])
by mx1.freebsd.org (Postfix) with ESMTP id 088FE7B33A
for <freebsd-virtualization@freebsd.org>; Tue, 6 Mar 2018 15:15:48 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
by vito.onthenet.com.au (Postfix) with ESMTP id 86A712076FB0
for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 01:15:46 +1000 (AEST)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
[203.13.68.150])
by alto.onthenet.com.au (Postfix) with ESMTPS id 6819C20B4A0C
for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 01:15:46 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
by iredmail.onthenet.com.au (Postfix) with ESMTP id 5C210281A4B
for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 01:15:46 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id qtSbALU1VroP for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 01:15:46 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net
[67.180.92.13])
by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 6482F2804D6;
Wed, 7 Mar 2018 01:15:43 +1000 (AEST)
Subject: Re: rumpkernel and bhyve: triple faults
To: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
References: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
From: Peter Grehan <grehan@freebsd.org>
Cc: freebsd-virtualization@freebsd.org, rumpkernel-users@freelists.org
Message-ID: <651856d3-3c34-9930-cda1-62d41091f91f@freebsd.org>
Date: Tue, 6 Mar 2018 07:15:41 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0
a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=BRlsKWOS0KQfm44lbqQA:9
a=y3tL4Oqu7xm67Kjg:21 a=9dlVhwN69FE26nys:21 a=QEXdDO2ut3YA:10 wl=host:3
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=Mvd8FVSe c=1 sm=1 tr=0
a=mJOSnoNX3k71adV6TmU0eQ==:117 a=5eVCmCvhg37cu/pjidAGzw==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=BRlsKWOS0KQfm44lbqQA:9
a=y3tL4Oqu7xm67Kjg:21 a=9dlVhwN69FE26nys:21 a=QEXdDO2ut3YA:10
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 15:15:50 -0000
Hi Fabian,
> 657 0 350887309700 vm testing[0]: handled exception vmexit at 0x102a56
> 656 0 350887309570 vm testing[0]: Exception 14 pending
> 655 0 350887309442 vm testing[0]: Setting intr_shadow to 0 succeeded
> 654 0 350887305126 vm testing[0]: Reflecting exception 14/0 into the guest
> 653 0 350887302436 vm testing[0]: vm_exit_intinfo: info1(0x80000b0e)
> 652 0 350887248280 vm testing[0]: Resume execution at 0x102a56
> 651 0 350887184160 vm testing[0]: vm_entry_intinfo: info1(0), info2(0x80000b0e), retinfo(0x80000b0e)
> 650 0 350887184040 vm testing[0]: Exception 14 delivered: 0x80000b0e
> 649 0 350887182668 vm testing[0]: handled exception vmexit at 0x102a56
Exception 14 is a page fault (SDM Vol3 ch 6.15). The exception type is
"fault" which means it is delivered at the address it was detected at.
This cascaded very quickly into a triple-fault, so it looks like it
could possibly be an issue with the stack. One debug tool you do have is
to get a register dump on exit, with 'bhyvectl --get-all --vm=<your vn
name>'.
For a page-fault, the virtual address that resulted in the fault will
be in the CR2 register.
From the code at the faulting address:
> 0000000000102a50 <cons_init>:
> 102a50: push rbx
> 102a51: call 103540 <hypervisor_detect>
> 102a56: cmp WORD PTR [rip-0x102a5c],0x0 # 2
<current_lwp+0x2>
It's using RIP-relative addressing here, but objdump seems to think
this may be an offset in the current_lwp structure - is it possible that
may have an uninitialized value ?
(I don't believe this has anything to do with VGA).
later,
Peter.
From owner-freebsd-virtualization@freebsd.org Tue Mar 6 16:22:07 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9116F3FB04
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 6 Mar 2018 16:22:07 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de
[130.149.50.25])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 545DF7E578;
Tue, 6 Mar 2018 16:22:06 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from [192.168.119.1] (wlan-141-23-171-244.tubit.tu-berlin.de
[141.23.171.244])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 674E161FA2;
Tue, 6 Mar 2018 16:22:03 +0000 (UTC)
From: "Fabian Freyer" <fabian.freyer@physik.tu-berlin.de>
To: "Peter Grehan" <grehan@freebsd.org>
Cc: freebsd-virtualization@freebsd.org, rumpkernel-users@freelists.org
Subject: Re: rumpkernel and bhyve: triple faults
Date: Tue, 06 Mar 2018 17:22:03 +0100
X-Mailer: MailMate (1.10r5443)
Message-ID: <2AAD4069-7D37-4325-8990-21C5DFD80B3B@physik.tu-berlin.de>
In-Reply-To: <651856d3-3c34-9930-cda1-62d41091f91f@freebsd.org>
References: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
<651856d3-3c34-9930-cda1-62d41091f91f@freebsd.org>
MIME-Version: 1.0
Content-Type: multipart/signed;
boundary="=_MailMate_915AEEFC-27FB-4C2F-88C0-FF431A3BA11F_=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Mar 2018 16:22:08 -0000
This is an OpenPGP/MIME signed message (RFC 3156 and 4880).
--=_MailMate_915AEEFC-27FB-4C2F-88C0-FF431A3BA11F_=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi Peter,
On 6 Mar 2018, at 16:15, Peter Grehan wrote:
> Exception 14 is a page fault (SDM Vol3 ch 6.15). The exception type is=
"fault" which means it is delivered at the address it was detected at.
>
> This cascaded very quickly into a triple-fault, so it looks like it co=
uld possibly be an issue with the stack. One debug tool you do have is to=
get a register dump on exit, with 'bhyvectl --get-all --vm=3D<your vn na=
me>'.
>
> For a page-fault, the virtual address that resulted in the fault will =
be in the CR2 register.
I don=E2=80=99t see a CR2 register in the output of bhyvectl --get-all, I=
was looking for that too.
> From the code at the faulting address:
>
> > 0000000000102a50 <cons_init>:
> > 102a50: push rbx
> > 102a51: call 103540 <hypervisor_detect>
> > 102a56: cmp WORD PTR [rip-0x102a5c],0x0 # 2 <curr=
ent_lwp+0x2>
>
> It's using RIP-relative addressing here, but objdump seems to think th=
is may be an offset in the current_lwp structure - is it possible that ma=
y have an uninitialized value ?
I=E2=80=99m pretty sure it=E2=80=99s tooling that=E2=80=99s displaying so=
mething off, since hopper is showing me this as
0x0000000000102a56 cmp word [0x2], 0x0
Which is very similar to what r2 is giving me:
;-- cons_init:
0x00102a50 53 push rbx ; /arch/x86:43=
0x00102a51 e8ea0a0000 call sym.hypervisor_detect ; /arch/x86:47=
0x00102a56 66833da4d5ef. cmp word [0x00000002], 0 ; /arch/x86:62=
> (I don't believe this has anything to do with VGA).
Maybe I=E2=80=99m off with my analysis of the actual fault here, but how =
I understand
the source (assuming compilers work as I would expect, which is not alway=
s true)
the values here are initialised from values in the bios data area (which =
is
zeroed out on bhyve):
#define BIOS_COM1_BASE 0x400
#define BIOS_CRTC_BASE 0x463
=2E..
movw BIOS_COM1_BASE, %bx
movw %bx, bios_com1_base
movw BIOS_CRTC_BASE, %bx
movw %bx, bios_crtc_base
=2E..
/*
* If the BIOS says no CRTC is present use the serial console if
* available.
*/
if (bios_crtc_base =3D=3D 0)
prefer_serial =3D 1;
Here=E2=80=99s my full output from bhyvectl --get-all:
ID Length Name
0 128MB sysmem
Address Length Segment Offset Prot Flags
0 128MB sysmem 0 RWX
efer[0] 0x0000000000000500
cr0[0] 0x0000000080010031
cr3[0] 0x000000000010b000
cr4[0] 0x0000000000002620
dr7[0] 0x0000000000000400
rsp[0] 0x0000000000100ff0
rip[0] 0x0000000000102a56
rax[0] 0x0000000000000000
rbx[0] 0x00000000003eaa2b
rcx[0] 0x0000000068622065
rdx[0] 0x0000000020657679
rsi[0] 0x0000000000100fd0
rdi[0] 0x0000000040000000
rbp[0] 0x0000000000000000
r8[0] 0x0000000000100fdc
r9[0] 0x0000000000100fd8
r10[0] 0x0000000000100fd4
r11[0] 0x0000000000000000
r12[0] 0x0000000000000000
r13[0] 0x0000000000000000
r14[0] 0x0000000000000000
r15[0] 0x0000000000000000
rflags[0] 0x0000000000010006
ds desc[0] 0x0000000000000000/0xffffffff/0x0000c093
es desc[0] 0x0000000000000000/0xffffffff/0x0000c093
fs desc[0] 0x0000000000000000/0xffffffff/0x0001c001
gs desc[0] 0x0000000000000000/0xffffffff/0x0001c001
ss desc[0] 0x0000000000000000/0xffffffff/0x0000c093
cs desc[0] 0x0000000000000000/0xffffffff/0x0000a09b
tr desc[0] 0x0000000000000000/0x00000000/0x0000008b
ldtr desc[0] 0x0000000000000000/0x0000ffff/0x00000082
gdtr[0] 0x0000000000378040/0x0000002f
idtr[0] 0x0000000000000000/0x0000ffff
cs[0] 0x0008
ds[0] 0x0018
es[0] 0x0018
fs[0] 0x0000
gs[0] 0x0000
ss[0] 0x0018
tr[0] 0x0000
ldtr[0] 0x0000
cr0_mask[0] 0xffffffff60000020
cr0_shadow[0] 0x0000000000000021
cr4_mask[0] 0xffffffffffe8f800
cr4_shadow[0] 0x0000000000000000
cr3_target_count[0] 0x0000000000000000
cr3_target0[0] 0x0000000000000000
cr3_target1[0] 0x0000000000000000
cr3_target2[0] 0x0000000000000000
cr3_target3[0] 0x0000000000000000
pinbased_ctls[0] 0x000000000000003f
procbased_ctls[0] 0x00000000f51865f2
procbased_ctls2[0] 0x00000000000010a2
gla[0] 0xfffffe0000c41000
gpa[0] 0x0000000000000000
entry_interruption_info[0] 0x0000000000000000
tpr_threshold[0] 0x0000000000000000
instruction_error[0] 0x0000000000000000
exit_ctls[0] 0x000000000033efff
entry_ctls[0] 0x00000000000093ff
host_pat[0] 0x0001050600070406
host_cr0[0] 0x000000008005003b
host_cr3[0] 0x0000000038045054
host_cr4[0] 0x00000000001726e0
host_rip[0] 0xffffffff81435290
host_rsp[0] 0xfffffe003218d700
vmcs_pointer[0] 0xffffffffffffffff
vmcs_exit_interruption_info[0] 0x0000000080000b0e
vmcs_exit_interruption_error[0] 0x0000000000000000
vmcs_guest_interruptibility[0] 0x0000000000000000
vmcs_exit_inst_length[0] 0x00000003
vmcs_exit_qualification[0] 0x0000000000000080
x2apic_state[0] 0
eptp[0] 0x000000003817905e
exception_bitmap[0] 0xffffffff
io_bitmap_a[0] 0
io_bitmap_b[0] 0
tsc_offset[0] 0x0000000000000000
msr_bitmap[0] 0x1adbc000
MSR_TSC [0] R-
MSR_EFER [0] RW
MSR_STAR [0] RW
MSR_LSTAR [0] RW
MSR_CSTAR [0] RW
MSR_SF_MASK [0] RW
MSR_FSBASE [0] RW
MSR_GSBASE [0] RW
MSR_KGSBASE [0] RW
MSR_SYSENTER_CS_MSR [0] RW
MSR_SYSENTER_ESP_MSR[0] RW
MSR_SYSENTER_EIP_MSR[0] RW
vpid[0] 0x0011
guest_pat[0] 0x0000000000000000
guest_sysenter_cs[0] 0
guest_sysenter_sp[0] 0
guest_sysenter_ip[0] 0
exit_reason[0] 0
rtc nvram[000]: 0x34
rtc time 0x5a9ebfd2: Tue Mar 06 16:20:34 2018
Capability "hlt_exit" is set on vcpu 0
Capability "mtrap_exit" is not set on vcpu 0
Capability "pause_exit" is set on vcpu 0
Capability "unrestricted_guest" is set on vcpu 0
Capability "enable_invpcid" is set on vcpu 0
active cpus: 0
suspended cpus: 0
pending: n/a
current: n/a
vcpu0 stats:
number of times in/out was intercepted 0
number of times cpuid was intercepted 3
vm exits due to nested page fault 13
vm exits for instruction emulation 0
number of vm exits for unknown reason 0
number of times astpending at exit 0
number of times idle requested at exit 0
number of vm exits handled in userspace 14
number of times rendezvous pending at exit 0
number of vm exits due to exceptions 3
number of NMIs delivered to vcpu 0
number of ExtINTs delivered to vcpu 0
Resident memory 69632
Wired memory 0
vcpu total runtime 3112708
EOI without any in-service interrupt 0
error interrupts generated by vlapic 0
timer interrupts generated by vlapic 0
corrected machine check interrupts generated by vlapic 0
lvts triggered[0] 0
lvts triggered[1] 0
lvts triggered[2] 0
lvts triggered[3] 0
lvts triggered[4] 0
lvts triggered[5] 0
lvts triggered[6] 0
ipis sent to vcpu[0] 0
ipis sent to vcpu[1] 0
ipis sent to vcpu[2] 0
ipis sent to vcpu[3] 0
ipis sent to vcpu[4] 0
ipis sent to vcpu[5] 0
ipis sent to vcpu[6] 0
ipis sent to vcpu[7] 0
ipis sent to vcpu[8] 0
ipis sent to vcpu[9] 0
ipis sent to vcpu[10] 0
ipis sent to vcpu[11] 0
ipis sent to vcpu[12] 0
ipis sent to vcpu[13] 0
ipis sent to vcpu[14] 0
ipis sent to vcpu[15] 0
number of ticks vcpu was idle 0
vcpu migration across host cpus 1
total number of vm exits 19
vm exits due to external interrupt 0
Number of vpid invalidations saved 0
Number of vpid invalidations done 1
number of times hlt was intercepted 0
number of times %cr access was intercepted 0
number of times rdmsr was intercepted 0
number of times wrmsr was intercepted 0
number of monitor trap exits 0
number of times pause was intercepted 0
vm exits due to interrupt window opening 0
vm exits due to nmi window opening 0
Fabian
--=_MailMate_915AEEFC-27FB-4C2F-88C0-FF431A3BA11F_=
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename=signature.asc
Content-Type: application/pgp-signature; name=signature.asc
-----BEGIN PGP SIGNATURE-----
iQJWBAEBCgBAFiEEX6JoxdmEemcFacQZmealkcs85+YFAlqewCsiHGZhYmlhbi5m
cmV5ZXJAcGh5c2lrLnR1LWJlcmxpbi5kZQAKCRCZ5qWRyzzn5munD/9HLibTUH3S
0sRR4LvpKXUZIzoYkIEqHNDtrGh4Y0NaNAJKQx1J6FAzf5lZBHKkcDx83vSfbRs7
C0BT76kq+jxxCgMSPm1gxXmjOJ5wuuM64tFKalEIn8AVwdKfeCiqvL4COM3DbTds
obw3MWifiBf4U8p7mZMm+AQmUxZgllmV4Lglzkq79rKWOxHjyUz7yP/sJHOUS+wN
+0QP9iFjdv2DFuFJTfa9/d84nhzY6bbdajmwYD2jtGfLnCkQW2JDjkMyk6Q5YT0g
fBDard4y2EyOsUx+RZpOWvC09MnuyTfZEVPbVCpyfH9tyRxjDw9ARnOOuWo1267u
R742vuZrZumKWV6TygloUS6DDpc6P7DhYPCZKHsrpsBUWwq+HCd52SbcYe3WS+rb
DqOHW800KLQlvopg2mQ2nix+f6Bb1pmYmJgvRcp14f+QDihjszJGjTXo8Y3npQqm
JsHBXLJsa6Txo1SlESWPg/uTB5fiNt63hL7aI9ElWSeDLalvQwRfPlMLWLXxntIX
fIFdRAF5d9nKAKYUYf9O9K94z8yFoIDTirFmm44zaWvwOosHk7WC0iCvdHZL71Xv
pXnWOhU5AbOANZgS3xPRgPZNhn2DmJAtUCLTtBiugdzU5IVv92Wmo0ifiNbmihjx
nK/IlfQ/CDc9+KcaFexiEkuqOdKaYTTqTw==
=z1en
-----END PGP SIGNATURE-----
--=_MailMate_915AEEFC-27FB-4C2F-88C0-FF431A3BA11F_=--
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 03:03:34 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D954F4804D
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 03:03:34 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from vito.onthenet.com.au (vito.onthenet.com.au [203.22.124.72])
by mx1.freebsd.org (Postfix) with ESMTP id C248C795AF
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 03:03:33 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
by vito.onthenet.com.au (Postfix) with ESMTP id 6A6B12095C5E
for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 13:03:31 +1000 (AEST)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
[203.13.68.150])
by alto.onthenet.com.au (Postfix) with ESMTPS id 519D920B1AE1
for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 13:03:31 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
by iredmail.onthenet.com.au (Postfix) with ESMTP id 4BCAA281A4B
for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 13:03:31 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id 80kxVTW9g82y for <freebsd-virtualization@freebsd.org>;
Wed, 7 Mar 2018 13:03:31 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local
(96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65])
by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 6C776280A18;
Wed, 7 Mar 2018 13:03:28 +1000 (AEST)
Subject: Re: rumpkernel and bhyve: triple faults
To: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Cc: freebsd-virtualization@freebsd.org, rumpkernel-users@freelists.org
References: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
<651856d3-3c34-9930-cda1-62d41091f91f@freebsd.org>
<2AAD4069-7D37-4325-8990-21C5DFD80B3B@physik.tu-berlin.de>
From: Peter Grehan <grehan@freebsd.org>
Message-ID: <1f1161ee-4543-6802-59a5-290e6c8c0329@freebsd.org>
Date: Tue, 6 Mar 2018 19:03:24 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <2AAD4069-7D37-4325-8990-21C5DFD80B3B@physik.tu-berlin.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0
a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=GcetEkhIoyJB3HI8B2UA:9
a=hRAa9tYipjSx2e2k:21 a=p7G98uB1is8chpi9:21 a=QEXdDO2ut3YA:10 wl=host:3
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=Mvd8FVSe c=1 sm=1 tr=0
a=mJOSnoNX3k71adV6TmU0eQ==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=GcetEkhIoyJB3HI8B2UA:9
a=hRAa9tYipjSx2e2k:21 a=p7G98uB1is8chpi9:21 a=QEXdDO2ut3YA:10
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 03:03:34 -0000
Hi Fabian,
>> For a page-fault, the virtual address that resulted in the fault
>> will
be in the CR2 register.
>=20
> I don=E2=80=99t see a CR2 register in the output of bhyvectl --get-all,=
I
> was looking for that too.
Oops, I'll add that to bhyvectl.
> I=E2=80=99m pretty sure it=E2=80=99s tooling that=E2=80=99s displaying =
something off, since
>hopper is showing me this as
>=20
> 0x0000000000102a56 cmp word [0x2], 0x0
>=20
> Which is very similar to what r2 is giving me:
>=20
> ;-- cons_init:
> 0x00102a50 53 push rbx ; /arch/x86:=
43
> 0x00102a51 e8ea0a0000 call sym.hypervisor_detect ; /arch/x86:=
47
> 0x00102a56 66833da4d5ef. cmp word [0x00000002], 0 ; /arch/x86:=
62
This is reading the 16-bit value from memory location 0x2. Hard to see=20
why this would generate a page-fault - the zero page is often mapped=20
read-only. Perhaps rumpkernel doesn't have a mapping for it, but then,=20
the offset for the access would be incorrect (maybe a linking issue with=20
the location of variables ?).
> Maybe I=E2=80=99m off with my analysis of the actual fault here, but ho=
w I understand
> the source (assuming compilers work as I would expect, which is not alw=
ays true)
> the values here are initialised from values in the bios data area (whic=
h is
> zeroed out on bhyve):
It shouldn't matter that those were zero. Loading them into a memory=20
location shouldn't be a problem.
> Here=E2=80=99s my full output from bhyvectl --get-all:
>=20
> ID Length Name
> 0 128MB sysmem
> Address Length Segment Offset Prot Flags
> 0 128MB sysmem 0 RWX
> efer[0] 0x0000000000000500
Ok, the guest is in 64-bit mode - the LMA bit is set. This implies=20
that rumpkernel has set up it's own mappings, since the multiboot loader=20
entered the guest in flat 32-bit mode.
> cr0[0] 0x0000000080010031
Paging is enabled (bit 31) as expected.
later,
PEter.
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 16:42:39 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A1EBF3DDB4
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 16:42:39 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 7348A7EF96
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 16:42:38 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org
[IPv6:2610:1c1:1:607c::16:b])
by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 5BF1B24318
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 16:42:38 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346)
id 5016B143726; Wed, 7 Mar 2018 16:42:38 +0000 (UTC)
Date: Wed, 7 Mar 2018 16:42:38 +0000
To: freebsd-virtualization@freebsd.org
From: "fabian.freyer_physik.tu-berlin.de (Fabian Freyer)"
<phabric-noreply@FreeBSD.org>
Reply-to: D14473+333+002e492985d67ce8@reviews.freebsd.org
Subject: [Differential] D14473: userboot: add callbacks to set unrestricted
guest mode
Message-ID: <05c3479bcf80d767afc3f065016357dd@localhost.localdomain>
X-Priority: 3
X-Phabricator-Sent-This-Message: Yes
X-Mail-Transport-Agent: MetaMTA
X-Auto-Response-Suppress: All
X-Phabricator-Mail-Tags: <differential-reviewers>
X-Herald-Rules: <28>, <67>, none
X-Phabricator-Projects: <#bhyve>
X-Phabricator-To: <PHID-USER-zbt5xib46bp3urfrrquz>
X-Phabricator-To: <PHID-USER-q5lmute3rwskeizvu5gf>
X-Phabricator-To: <PHID-USER-5gvndaunomzzajpx5qoc>
X-Phabricator-To: <PHID-PROJ-cwudjt2qcr7oyit224ue>
X-Phabricator-To: <PHID-USER-2monqlbk24m6mwkbdamf>
X-Phabricator-Cc: <PHID-USER-2monqlbk24m6mwkbdamf>
X-Phabricator-Cc: <PHID-USER-q5lmute3rwskeizvu5gf>
X-Phabricator-Cc: <PHID-USER-6ps3unnxvniqcn5kdjn5>
X-Phabricator-Cc: <PHID-PROJ-xwfypuwkmupl5yhy5m3b>
Precedence: bulk
Thread-Topic: PHID-DREV-z4kdgdlru2lrsvfkw4bo
X-Phabricator-Mail-ID: 987791
X-Phabricator-Send-Attempt: yolnde3jxua5o4eu
In-Reply-To: <differential-rev-PHID-DREV-z4kdgdlru2lrsvfkw4bo-req@FreeBSD.org>
References: <differential-rev-PHID-DREV-z4kdgdlru2lrsvfkw4bo-req@FreeBSD.org>
Thread-Index: OWY4MjZiMDdiMWRiNGU2MWQzZDM0N2I0N2FiIFqgFn4=
X-Phabricator-Stamps: actor(@fabian.freyer_physik.tu-berlin.de)
application(Differential) author(@fabian.freyer_physik.tu-berlin.de)
herald(H28) herald(H67) monogram(D14473) object-type(DREV)
phid(PHID-DREV-z4kdgdlru2lrsvfkw4bo) reviewer(#bhyve) reviewer(@grehan)
reviewer(@imp) reviewer(@rgrimes) revision-status(needs-review)
subscriber(#contributor_reviews_base)
subscriber(@freebsd-virtualization-list) subscriber(@grehan) subscriber(@imp)
tag(#bhyve) via(web)
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 16:42:39 -0000
ZmFiaWFuLmZyZXllcl9waHlzaWsudHUtYmVybGluLmRlIGFkZGVkIDEgYmxvY2tpbmcgcmV2aWV3
ZXIocyk6IGdyZWhhbi4KClJFVklTSU9OIERFVEFJTAogIGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNk
Lm9yZy9EMTQ0NzMKCkVNQUlMIFBSRUZFUkVOQ0VTCiAgaHR0cHM6Ly9yZXZpZXdzLmZyZWVic2Qu
b3JnL3NldHRpbmdzL3BhbmVsL2VtYWlscHJlZmVyZW5jZXMvCgpUbzogZmFiaWFuLmZyZXllcl9w
aHlzaWsudHUtYmVybGluLmRlLCBpbXAsIHJncmltZXMsICNiaHl2ZSwgZ3JlaGFuCkNjOiBncmVo
YW4sIGltcCwgZnJlZWJzZC12aXJ0dWFsaXphdGlvbi1saXN0LCAjY29udHJpYnV0b3JfcmV2aWV3
c19iYXNlCg==
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 16:55:42 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6DCA5F3F1F6
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 16:55:42 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 0A61680827
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 16:55:42 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org
[IPv6:2610:1c1:1:607c::16:b])
by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 01D792477A
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 16:55:42 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346)
id F31F8147AFA; Wed, 7 Mar 2018 16:55:41 +0000 (UTC)
Date: Wed, 7 Mar 2018 16:55:41 +0000
To: freebsd-virtualization@freebsd.org
From: "grehan (Peter Grehan)" <phabric-noreply@FreeBSD.org>
Reply-to: D14473+333+002e492985d67ce8@reviews.freebsd.org
Subject: [Differential] D14473: userboot: add callbacks to set unrestricted
guest mode
Message-ID: <a331058e51cce708d9b83140b20622d9@localhost.localdomain>
X-Priority: 3
X-Phabricator-Sent-This-Message: Yes
X-Mail-Transport-Agent: MetaMTA
X-Auto-Response-Suppress: All
X-Phabricator-Mail-Tags: <differential-comment>
X-Herald-Rules: <28>, <67>, none, <97>
X-Phabricator-Projects: <#bhyve>
X-Phabricator-To: <PHID-USER-zbt5xib46bp3urfrrquz>
X-Phabricator-To: <PHID-USER-q5lmute3rwskeizvu5gf>
X-Phabricator-To: <PHID-USER-5gvndaunomzzajpx5qoc>
X-Phabricator-To: <PHID-PROJ-cwudjt2qcr7oyit224ue>
X-Phabricator-To: <PHID-USER-2monqlbk24m6mwkbdamf>
X-Phabricator-Cc: <PHID-USER-2monqlbk24m6mwkbdamf>
X-Phabricator-Cc: <PHID-USER-q5lmute3rwskeizvu5gf>
X-Phabricator-Cc: <PHID-USER-6ps3unnxvniqcn5kdjn5>
X-Phabricator-Cc: <PHID-PROJ-xwfypuwkmupl5yhy5m3b>
Precedence: bulk
Thread-Topic: PHID-DREV-z4kdgdlru2lrsvfkw4bo
X-Phabricator-Mail-ID: 988558
X-Phabricator-Send-Attempt: 6ww5csxw64vpztnx
In-Reply-To: <differential-rev-PHID-DREV-z4kdgdlru2lrsvfkw4bo-req@FreeBSD.org>
References: <differential-rev-PHID-DREV-z4kdgdlru2lrsvfkw4bo-req@FreeBSD.org>
Thread-Index: OWY4MjZiMDdiMWRiNGU2MWQzZDM0N2I0N2FiIFqgGY0=
X-Phabricator-Stamps: actor(@grehan) application(Differential)
author(@fabian.freyer_physik.tu-berlin.de) blocking-reviewer(@grehan)
herald(H28) herald(H67) herald(H97) monogram(D14473) object-type(DREV)
phid(PHID-DREV-z4kdgdlru2lrsvfkw4bo) reviewer(#bhyve) reviewer(@grehan)
reviewer(@imp) reviewer(@rgrimes) revision-status(needs-review)
subscriber(#contributor_reviews_base)
subscriber(@freebsd-virtualization-list) subscriber(@grehan) subscriber(@imp)
tag(#bhyve) via(web)
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 16:55:42 -0000
Z3JlaGFuIGFkZGVkIGEgY29tbWVudC4KCgogIFNvIGEgY29tbWVudCBvbiB0aGlzOiBpbiBnZW5l
cmFsLCBhcGkncyBhcmUgbm90IGFkZGVkIHRvIEZyZWVCU0QgdGhhdCBkb24ndCBoYXZlIGJhc2Ut
c3lzdGVtIGNsaWVudHMgdGhhdCB1c2UgdGhlbSwgb3IgZm9yIGEgZ29vZCByZWFzb24uIEkgdGhp
bmsgdGhpcyBmYWxscyBpbnRvIHRoZSBsYXR0ZXIgYnV0IEknZCBsaWtlIHRvIGN1dCBpdCBkb3du
IGEgYml0LgogIAogIC0gY2FuIHRoZSBnZXRfdW5yZXN0cmljdGVkX2d1ZXN0KCkgcm91dGluZSBi
ZSByZW1vdmVkID8gVGhlcmUgaXMgYW4gZXJyb3IgcmV0dXJuIG9uIHRoZSBzZXQsIHdoaWNoIHNl
ZW1zIGxpa2UgaXQgY2FuIGJlIHVzZWQgdG8gZGV0ZXJtaW5lIGlmIHVucmVzdHJpY3RlZCBtb2Rl
IGlzIG5vdCBhdmFpbGFibGUgKGUuZy4gdGhhdCdzIGhvdyBiaHl2ZSB1c2VzIHRoZSBpb2N0bCku
CiAgLSBpcyB0aGVyZSBhIG5lZWQgZm9yIHZjcHVfcmVzZXQoKSA/IFRoZSBCU1Agc2hvdWxkIGJl
IGluaXRpYWxpemVkIHRvIHBvd2VyLW9uIHN0YXRlLi4gVGhhdCBjb3VsZCBiZSBhIGJ1ZyBpbiBi
aHl2ZSBhbmQgYmV0dGVyIHRvIGhhdmUgaXQgZml4ZWQgdGhlcmUuCgpSRVZJU0lPTiBERVRBSUwK
ICBodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDE0NDczCgpFTUFJTCBQUkVGRVJFTkNFUwog
IGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9zZXR0aW5ncy9wYW5lbC9lbWFpbHByZWZlcmVu
Y2VzLwoKVG86IGZhYmlhbi5mcmV5ZXJfcGh5c2lrLnR1LWJlcmxpbi5kZSwgaW1wLCByZ3JpbWVz
LCAjYmh5dmUsIGdyZWhhbgpDYzogZ3JlaGFuLCBpbXAsIGZyZWVic2QtdmlydHVhbGl6YXRpb24t
bGlzdCwgI2NvbnRyaWJ1dG9yX3Jldmlld3NfYmFzZQo=
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 17:03:14 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5522F3FF7C
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 17:03:13 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 839B88100F
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 17:03:13 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: from reviews.nyi.freebsd.org (reviews.nyi.freebsd.org
[IPv6:2610:1c1:1:607c::16:b])
by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 7B6B62491C
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 17:03:13 +0000 (UTC)
(envelope-from daemon-user@freebsd.org)
Received: by reviews.nyi.freebsd.org (Postfix, from userid 1346)
id 7A2E514A956; Wed, 7 Mar 2018 17:03:13 +0000 (UTC)
Date: Wed, 7 Mar 2018 17:03:13 +0000
To: freebsd-virtualization@freebsd.org
From: "fabian.freyer_physik.tu-berlin.de (Fabian Freyer)"
<phabric-noreply@FreeBSD.org>
Reply-to: D14473+333+002e492985d67ce8@reviews.freebsd.org
Subject: [Differential] D14473: userboot: add callbacks to set unrestricted
guest mode
Message-ID: <2843ee914c79e89f13a257933489caf7@localhost.localdomain>
X-Priority: 3
X-Phabricator-Sent-This-Message: Yes
X-Mail-Transport-Agent: MetaMTA
X-Auto-Response-Suppress: All
X-Phabricator-Mail-Tags: <differential-comment>
X-Herald-Rules: <28>, <67>, none, <97>
X-Phabricator-Projects: <#bhyve>
X-Phabricator-To: <PHID-USER-zbt5xib46bp3urfrrquz>
X-Phabricator-To: <PHID-USER-q5lmute3rwskeizvu5gf>
X-Phabricator-To: <PHID-USER-5gvndaunomzzajpx5qoc>
X-Phabricator-To: <PHID-PROJ-cwudjt2qcr7oyit224ue>
X-Phabricator-To: <PHID-USER-2monqlbk24m6mwkbdamf>
X-Phabricator-Cc: <PHID-USER-2monqlbk24m6mwkbdamf>
X-Phabricator-Cc: <PHID-USER-q5lmute3rwskeizvu5gf>
X-Phabricator-Cc: <PHID-USER-6ps3unnxvniqcn5kdjn5>
X-Phabricator-Cc: <PHID-PROJ-xwfypuwkmupl5yhy5m3b>
Precedence: bulk
Thread-Topic: PHID-DREV-z4kdgdlru2lrsvfkw4bo
X-Phabricator-Mail-ID: 988580
X-Phabricator-Send-Attempt: n5lxdu435c2hqsnv
In-Reply-To: <differential-rev-PHID-DREV-z4kdgdlru2lrsvfkw4bo-req@FreeBSD.org>
References: <differential-rev-PHID-DREV-z4kdgdlru2lrsvfkw4bo-req@FreeBSD.org>
Thread-Index: OWY4MjZiMDdiMWRiNGU2MWQzZDM0N2I0N2FiIFqgG1E=
X-Phabricator-Stamps: actor(@fabian.freyer_physik.tu-berlin.de)
application(Differential) author(@fabian.freyer_physik.tu-berlin.de)
blocking-reviewer(@grehan) herald(H28) herald(H67) herald(H97)
mention(@grehan) monogram(D14473) object-type(DREV)
phid(PHID-DREV-z4kdgdlru2lrsvfkw4bo) reviewer(#bhyve) reviewer(@grehan)
reviewer(@imp) reviewer(@rgrimes) revision-status(needs-review)
subscriber(#contributor_reviews_base)
subscriber(@freebsd-virtualization-list) subscriber(@grehan) subscriber(@imp)
tag(#bhyve) via(web)
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 17:03:14 -0000
ZmFiaWFuLmZyZXllcl9waHlzaWsudHUtYmVybGluLmRlIGFkZGVkIGEgY29tbWVudC4KCgogIElu
IEQxNDQ3MyMzMDY4MjcgPGh0dHBzOi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9EMTQ0NzMjMzA2ODI3
PiwgQGdyZWhhbiB3cm90ZToKICAKICA+IFNvIGEgY29tbWVudCBvbiB0aGlzOiBpbiBnZW5lcmFs
LCBhcGkncyBhcmUgbm90IGFkZGVkIHRvIEZyZWVCU0QgdGhhdCBkb24ndCBoYXZlIGJhc2Utc3lz
dGVtIGNsaWVudHMgdGhhdCB1c2UgdGhlbSwgb3IgZm9yIGEgZ29vZCByZWFzb24uIEkgdGhpbmsg
dGhpcyBmYWxscyBpbnRvIHRoZSBsYXR0ZXIgYnV0IEknZCBsaWtlIHRvIGN1dCBpdCBkb3duIGEg
Yml0LgogID4KICA+IC0gY2FuIHRoZSBnZXRfdW5yZXN0cmljdGVkX2d1ZXN0KCkgcm91dGluZSBi
ZSByZW1vdmVkID8gVGhlcmUgaXMgYW4gZXJyb3IgcmV0dXJuIG9uIHRoZSBzZXQsIHdoaWNoIHNl
ZW1zIGxpa2UgaXQgY2FuIGJlIHVzZWQgdG8gZGV0ZXJtaW5lIGlmIHVucmVzdHJpY3RlZCBtb2Rl
IGlzIG5vdCBhdmFpbGFibGUgKGUuZy4gdGhhdCdzIGhvdyBiaHl2ZSB1c2VzIHRoZSBpb2N0bCku
CiAgCiAgCiAgWWVzLCBpdCBjYW4uCiAgCiAgPiAtIGlzIHRoZXJlIGEgbmVlZCBmb3IgdmNwdV9y
ZXNldCgpID8gVGhlIEJTUCBzaG91bGQgYmUgaW5pdGlhbGl6ZWQgdG8gcG93ZXItb24gc3RhdGUu
LiBUaGF0IGNvdWxkIGJlIGEgYnVnIGluIGJoeXZlIGFuZCBiZXR0ZXIgdG8gaGF2ZSBpdCBmaXhl
ZCB0aGVyZS4KICAKICBOb3QgbmVjZXNzYXJpbHksIGFzIGV2ZXJ5dGhpbmcgaW4gdmNwdV9yZXNl
dCgpIGNvdWxkIGFsc28gYmUgYWNjb21wbGlzaGVkIHdpdGggdGhlIG90aGVyIGNhbGxiYWNrcy4g
SSBkb24ndCB0aGluayBiaHl2ZXJ1biBzaG91bGQgY2FsbCB2Y3B1X3Jlc2V0KCksIGFzIGJoeXZl
bG9hZCBzZXRzIHVwIHJlZ2lzdGVycyBiZWZvcmUuIEkgZ3Vlc3MgdGhpcyBzaG91bGQgaGFwcGVu
IGJlZm9yZSBgbG9hZGVyX21haW5gIGlzIGNhbGxlZD8KClJFVklTSU9OIERFVEFJTAogIGh0dHBz
Oi8vcmV2aWV3cy5mcmVlYnNkLm9yZy9EMTQ0NzMKCkVNQUlMIFBSRUZFUkVOQ0VTCiAgaHR0cHM6
Ly9yZXZpZXdzLmZyZWVic2Qub3JnL3NldHRpbmdzL3BhbmVsL2VtYWlscHJlZmVyZW5jZXMvCgpU
bzogZmFiaWFuLmZyZXllcl9waHlzaWsudHUtYmVybGluLmRlLCBpbXAsIHJncmltZXMsICNiaHl2
ZSwgZ3JlaGFuCkNjOiBncmVoYW4sIGltcCwgZnJlZWJzZC12aXJ0dWFsaXphdGlvbi1saXN0LCAj
Y29udHJpYnV0b3JfcmV2aWV3c19iYXNlCg==
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 17:14:13 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BD46F4160B
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 17:14:13 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id A6FE681C2F
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 17:14:11 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w27HE3O5054614
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 09:14:03 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w27HE31L054613
for freebsd-virtualization@freebsd.org; Wed, 7 Mar 2018 09:14:03 -0800 (PST)
(envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803071714.w27HE31L054613@pdx.rh.CN85.dnsmgr.net>
Subject: Call for testing bhyve cpu topology additions
To: freebsd-virtualization@freebsd.org
Date: Wed, 7 Mar 2018 09:14:03 -0800 (PST)
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 17:14:13 -0000
There is a new version of the CPU topology review up at
http://reviews.freebsd.org/D9930
I would like to ask that if people can test this and provide
feedback that they do so.
It has some undesired side effects on vm-bhyve and probably
other down stream tools, I am in the process of contacting
the vm-bhyve author to see what can be done to clean up
this output (if you are here please respond to this thread):
src-topo # vm list
NAME DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE
issd30g1 default bhyveload cpus=8,sockets=2,cores=4,threads=1 1024M - No Stopped
Notice that due to the new CPU string being much more complicated than
the normal int it makes the output format ugly. I have another change
to vm-bhyve that I would like to see too, and that is to move the NAME
to the end of the line and remove the 16 character limit. I have done
that in my local copy already. This string and its parsing are designed
to be Qemu compatible.
Here is a sample of my local vm-bhyve vm list output (no topology used):
/tmp # vm list
DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE NAME
default bhyveload 1 128M - No Stopped fb-bld-10-amd64
default bhyveload 1 128M - No Stopped fb-bld-11-amd64
default bhyveload 1 128M - No Stopped fb-bld-11.0-p1-amd64
default bhyveload 1 128M - No Stopped fb-bld-11.0-p1-i386
default bhyveload 4 512M - No Stopped fb-bld-11_1-amd64
default bhyveload 4 512M - No Stopped fb-bld-11_1-i386
default bhyveload 2 256M - No Running (30227) fb-bld-head-amd64
Thoughts are to teach vm-bhyve to parse the string just as bhyve does
and only output the vCPU count. Other thoughts I have are to have
it parse the string and output either vCPU if cores/threads is 1,
or a simple S/C/T string.
If you are a downstream maintainer of one of the other vm management packages
I am open to input. The implemntation I have done should allow any existing
tool that treated -c as a string to use the new topology without changes.
This includes the in base vmrun.sh.
Also people using the sysctls:
hw.vmm.topology.cores_per_package: 1
hw.vmm.topology.threads_per_core: 1
can continue to do so at this time, but there is work in process to
deprecate these, that work includes making stable/11 emit a warning
message if they are used, and remove them in head/12.
If I can get some significant test results back I plan to commit
D9930 to ^head and merge it back to stable/11 3 days later.
Thanks,
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 17:30:53 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDA74F42F24
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 17:30:53 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from vito.onthenet.com.au (vito.onthenet.com.au [203.22.124.72])
by mx1.freebsd.org (Postfix) with ESMTP id 40D1E82DB0
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 17:30:52 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
by vito.onthenet.com.au (Postfix) with ESMTP id 3EAFE2092D29
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 03:30:50 +1000 (AEST)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
[203.13.68.150])
by alto.onthenet.com.au (Postfix) with ESMTPS id 260D920B4AA6
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 03:30:50 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
by iredmail.onthenet.com.au (Postfix) with ESMTP id 2057C281F1F
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 03:30:50 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id vSImZzrZnkF0 for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 03:30:50 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net
[67.180.92.13])
by iredmail.onthenet.com.au (Postfix) with ESMTPSA id F157C280504;
Thu, 8 Mar 2018 03:30:47 +1000 (AEST)
Subject: Re: Call for testing bhyve cpu topology additions
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
References: <201803071714.w27HE31L054613@pdx.rh.CN85.dnsmgr.net>
Cc: freebsd-virtualization@freebsd.org
From: Peter Grehan <grehan@freebsd.org>
Message-ID: <2b5b4f82-ee51-dfca-c505-6d15aa9fc95a@freebsd.org>
Date: Wed, 7 Mar 2018 09:30:45 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <201803071714.w27HE31L054613@pdx.rh.CN85.dnsmgr.net>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0
a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=CfVD2wX5spfmZtrk1A0A:9
a=QEXdDO2ut3YA:10 wl=host:3
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=Mvd8FVSe c=1 sm=1 tr=0
a=mJOSnoNX3k71adV6TmU0eQ==:117 a=5eVCmCvhg37cu/pjidAGzw==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=CfVD2wX5spfmZtrk1A0A:9
a=QEXdDO2ut3YA:10
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 17:30:54 -0000
> I would like to ask that if people can test this and provide
> feedback that they do so.
And as mentioned in the review, I'd also like to see your Windows
desktop guest test results with this change.
> If I can get some significant test results back I plan to commit
> D9930 to ^head and merge it back to stable/11 3 days later.
Standard MFC time is 3 weeks.
later,
Peter.
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 17:46:44 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58C31F44470
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 17:46:44 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id CFB3483D66;
Wed, 7 Mar 2018 17:46:43 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w27HkfQx054759;
Wed, 7 Mar 2018 09:46:41 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w27HkfU9054758;
Wed, 7 Mar 2018 09:46:41 -0800 (PST) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803071746.w27HkfU9054758@pdx.rh.CN85.dnsmgr.net>
Subject: Re: Call for testing bhyve cpu topology additions
In-Reply-To: <2b5b4f82-ee51-dfca-c505-6d15aa9fc95a@freebsd.org>
To: Peter Grehan <grehan@freebsd.org>
Date: Wed, 7 Mar 2018 09:46:41 -0800 (PST)
CC: freebsd-virtualization@freebsd.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 17:46:44 -0000
> > I would like to ask that if people can test this and provide
> > feedback that they do so.
>
> And as mentioned in the review, I'd also like to see your Windows
> desktop guest test results with this change.
I do not run any windows in bhyve as bhyve can not run the
windows I use due to missing/broken ATA support.
The person I was helping with bhyve windows regression tests has
become unavaliable.
> > If I can get some significant test results back I plan to commit
> > D9930 to ^head and merge it back to stable/11 3 days later.
>
> Standard MFC time is 3 weeks.
Can you point to this some place? My understanding is that
MFC is at the discretion of the committer, and the only
thing the big list of rules says:
6. Changes go to FreeBSD-CURRENT before FreeBSD-STABLE unless
specifically permitted by the release engineer or unless they
are not applicable to FreeBSD-CURRENT. Any non-trivial or
non-urgent change which is applicable should also be allowed
to sit in FreeBSD-CURRENT for at least 3 days before merging
so that it can be given sufficient testing.
I am making a wide call for testing, above and beyond the
normal process already. I am also about to pull this back
to my own 11.1 systems to ensure that I spot any merge
problems and if need be attach an 11.1 and 11/stable patch
to the diffential.
There is also a call for testing going out at byhvecon,
and there has already been a year to test on this,
though perhaps not in final form, at least in functional
form.
Regards,
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 18:51:11 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AD98F498DB
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 18:51:11 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from vito.onthenet.com.au (vito.onthenet.com.au [203.22.124.72])
by mx1.freebsd.org (Postfix) with ESMTP id D8EE787B3E
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 18:51:10 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
by vito.onthenet.com.au (Postfix) with ESMTP id 14AD42098676
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 04:51:08 +1000 (AEST)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
[203.13.68.150])
by alto.onthenet.com.au (Postfix) with ESMTPS id E1E7120B4A16
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 04:51:07 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
by iredmail.onthenet.com.au (Postfix) with ESMTP id DBF972809DB
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 04:51:07 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id vPwCWN050CGy for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 04:51:07 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local
(96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65])
by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 25F402809BC;
Thu, 8 Mar 2018 04:51:05 +1000 (AEST)
Subject: Re: Call for testing bhyve cpu topology additions
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Cc: freebsd-virtualization@freebsd.org
References: <201803071746.w27HkfU9054758@pdx.rh.CN85.dnsmgr.net>
From: Peter Grehan <grehan@freebsd.org>
Message-ID: <faccbdfb-5d0e-054a-a500-5086dc6b8baa@freebsd.org>
Date: Wed, 7 Mar 2018 10:51:01 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <201803071746.w27HkfU9054758@pdx.rh.CN85.dnsmgr.net>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0
a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=gI7P7j3JjJIH98DZ60YA:9
a=QEXdDO2ut3YA:10 wl=host:3
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=Mvd8FVSe c=1 sm=1 tr=0
a=mJOSnoNX3k71adV6TmU0eQ==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=gI7P7j3JjJIH98DZ60YA:9
a=QEXdDO2ut3YA:10
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 18:51:11 -0000
>> And as mentioned in the review, I'd also like to see your Windows
>> desktop guest test results with this change.
>
> I do not run any windows in bhyve as bhyve can not run the
> windows I use due to missing/broken ATA support.
> The person I was helping with bhyve windows regression tests has
> become unavaliable.
I'm staggered by this. The *only reason* for having this change is to
get past the 1/2 socket limitation when running Windows desktop guests.
It has no impact on any other guests, whatsoever.
You can easily download an eval of Windows 10 to try this out with.
You do not (and have never) required ATA support to run Windows on bhyve.
>> Standard MFC time is 3 weeks.
>
> Can you point to this some place? My understanding is that
> MFC is at the discretion of the committer, and the only
> thing the big list of rules says:
This is project culture, which you should have seen observing MFC
times for commits.
And why the rush ? I'm yet to understand what the urgency for this
work is ? Who is demanding it ?
later,
Peter.
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 19:20:13 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3AD07F4BAF8
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 19:20:13 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 7B4DE6908D;
Wed, 7 Mar 2018 19:20:12 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w27JK9AG055369;
Wed, 7 Mar 2018 11:20:09 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w27JK9JQ055368;
Wed, 7 Mar 2018 11:20:09 -0800 (PST) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803071920.w27JK9JQ055368@pdx.rh.CN85.dnsmgr.net>
Subject: Re: Call for testing bhyve cpu topology additions
In-Reply-To: <faccbdfb-5d0e-054a-a500-5086dc6b8baa@freebsd.org>
To: Peter Grehan <grehan@freebsd.org>
Date: Wed, 7 Mar 2018 11:20:09 -0800 (PST)
CC: freebsd-virtualization@freebsd.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 19:20:13 -0000
> >> And as mentioned in the review, I'd also like to see your Windows
> >> desktop guest test results with this change.
> >
> > I do not run any windows in bhyve as bhyve can not run the
> > windows I use due to missing/broken ATA support.
> > The person I was helping with bhyve windows regression tests has
> > become unavaliable.
>
> I'm staggered by this. The *only reason* for having this change is to
> get past the 1/2 socket limitation when running Windows desktop guests.
> It has no impact on any other guests, whatsoever.
I shall iterate again, this change makes no change to what the guests
sees if you use the old method sysctl hw.vmm.topology.cores_per_package
or hw.vmm.topology.threads_per_core to set these values, it is
purely a interface enhancement that makes these tuneables easy
to access from userland bhyve(8).
A guest can not tell the diffence in what way these are set.
If hw.vmm.topology.* needs testing thats not on me, that is
an existing problem, and one that has existed for far too long.
>
> You can easily download an eval of Windows 10 to try this out with.
> You do not (and have never) required ATA support to run Windows on bhyve.
I have made a call for testing, whats your issue?
Are you trying to force me to do that testing?
>
> >> Standard MFC time is 3 weeks.
> >
> > Can you point to this some place? My understanding is that
> > MFC is at the discretion of the committer, and the only
> > thing the big list of rules says:
>
> This is project culture, which you should have seen observing MFC
> times for commits.
And I consider this change rather trivial and with near 0 risk,
so choose to use a 3 day MFC timer for it. The worse that can
happen is the user would have to use the old sysctls which I
left in place with full compatibility.
> And why the rush ? I'm yet to understand what the urgency for this
> work is ? Who is demanding it ?
The users have been wanting this for well over a year, it was
a frequently requested item when I wrote it. It is long overdue.
You seem to be raising a bar higher than any other part of
FreeBSD for this rather simple user command enhancement,
can I ask what your objection is? If you think the topology
seen by the guest is going to be bad, that bug exists today
for anyone trying to use the sysctl's, perhaps we can find
that there are bugs if we get this in the hands of the user
rather than yet again delay a new feature for what appears
to be tests of code paths very minimally effected (change
of a variable name) by this patch.
Regards,
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Wed Mar 7 19:29:03 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A10DF279AB
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 7 Mar 2018 19:29:03 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from vito.onthenet.com.au (vito.onthenet.com.au [203.22.124.72])
by mx1.freebsd.org (Postfix) with ESMTP id 7D9AA6991F
for <freebsd-virtualization@freebsd.org>; Wed, 7 Mar 2018 19:29:02 +0000 (UTC)
(envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
by vito.onthenet.com.au (Postfix) with ESMTP id 0A32B209867B
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 05:29:00 +1000 (AEST)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
[203.13.68.150])
by alto.onthenet.com.au (Postfix) with ESMTPS id E066B20B4A16
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 05:28:59 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
by iredmail.onthenet.com.au (Postfix) with ESMTP id DA9EF281E00
for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 05:28:59 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id Y1dUgqQXUJ-O for <freebsd-virtualization@freebsd.org>;
Thu, 8 Mar 2018 05:28:59 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local
(96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65])
by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 8C54F2804D6;
Thu, 8 Mar 2018 05:28:57 +1000 (AEST)
Subject: Re: Call for testing bhyve cpu topology additions
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Cc: freebsd-virtualization@freebsd.org
References: <201803071920.w27JK9JQ055368@pdx.rh.CN85.dnsmgr.net>
From: Peter Grehan <grehan@freebsd.org>
Message-ID: <db620e09-ec96-7931-2776-e6477c5a3d8d@freebsd.org>
Date: Wed, 7 Mar 2018 11:28:55 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <201803071920.w27JK9JQ055368@pdx.rh.CN85.dnsmgr.net>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0
a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=n5e4Vix2roilDiFsZ4gA:9
a=ylQoYIHwv3EluTzj:21 a=GssiOoEsnPXuDgD0:21 a=QEXdDO2ut3YA:10 wl=host:3
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.3 cv=Mvd8FVSe c=1 sm=1 tr=0
a=mJOSnoNX3k71adV6TmU0eQ==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=n5e4Vix2roilDiFsZ4gA:9
a=ylQoYIHwv3EluTzj:21 a=GssiOoEsnPXuDgD0:21 a=QEXdDO2ut3YA:10
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 07 Mar 2018 19:29:03 -0000
> I shall iterate again, this change makes no change to what the guests
> sees if you use the old method sysctl hw.vmm.topology.cores_per_package
> or hw.vmm.topology.threads_per_core to set these values, it is
> purely a interface enhancement that makes these tuneables easy
> to access from userland bhyve(8).
Those sysctls were an undocumented workaround with no error checking.
You are making this a documented part of bhyve,
> A guest can not tell the diffence in what way these are set.
> If hw.vmm.topology.* needs testing thats not on me, that is
> an existing problem, and one that has existed for far too long.
Ah, no, the testing is on you, not the user community.
>> You can easily download an eval of Windows 10 to try this out with.
>> You do not (and have never) required ATA support to run Windows on bhyve.
>
> I have made a call for testing, whats your issue?
> Are you trying to force me to do that testing?
At a minimum, you are expected to test changes that you expect to commit.
> And I consider this change rather trivial and with near 0 risk,
You've never made a commit to bhyve but somehow feel qualified to make
risk assessments.
>> And why the rush ? I'm yet to understand what the urgency for this
>> work is ? Who is demanding it ?
>
> The users have been wanting this for well over a year, it was
> a frequently requested item when I wrote it. It is long overdue.
Right, so 3 weeks for MFC is perfectly acceptable in that case.
> You seem to be raising a bar higher than any other part of
> FreeBSD for this rather simple user command enhancement,
> can I ask what your objection is?
The fact that you seem to think testing this is someone else's
problem, in a subsystem where rigorous testing is a requirement.
later,
Peter.
From owner-freebsd-virtualization@freebsd.org Fri Mar 9 17:45:34 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A4F1F3D215
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 9 Mar 2018 17:45:34 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from mail.physik.tu-berlin.de (mail.physik-pool.tu-berlin.de
[130.149.50.25])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 041AA8349C
for <freebsd-virtualization@freebsd.org>; Fri, 9 Mar 2018 17:45:33 +0000 (UTC)
(envelope-from fabian.freyer@physik.tu-berlin.de)
Received: from [192.168.119.1] (wlan-141-23-171-244.tubit.tu-berlin.de
[141.23.171.244])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 0407C6206E;
Fri, 9 Mar 2018 17:45:32 +0000 (UTC)
From: "Fabian Freyer" <fabian.freyer@physik.tu-berlin.de>
To: rumpkernel-users@freelists.org
Cc: freebsd-virtualization@freebsd.org
Subject: Re: rumpkernel and bhyve: triple faults
Date: Fri, 09 Mar 2018 18:45:28 +0100
X-Mailer: MailMate (1.10r5443)
Message-ID: <A3574A92-0514-44AE-BE20-6BFDAE803407@physik.tu-berlin.de>
In-Reply-To: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
References: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
MIME-Version: 1.0
Content-Type: multipart/signed;
boundary="=_MailMate_969387E7-81FE-4CE4-B9E3-F313ED190F6E_=";
micalg=pgp-sha512; protocol="application/pgp-signature"
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Mar 2018 17:45:34 -0000
This is an OpenPGP/MIME signed message (RFC 3156 and 4880).
--=_MailMate_969387E7-81FE-4CE4-B9E3-F313ED190F6E_=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
On 6 Mar 2018, at 7:45, Fabian Freyer wrote:
> Tracking down bios_crtc_base, I find that it=E2=80=99s loaded in
> rumprun/platform/hw/arch/amd64/locore.S:70:
>
> /* save BIOS data area values */
> movw BIOS_COM1_BASE, %bx
> movw %bx, bios_com1_base
> movw BIOS_CRTC_BASE, %bx
> movw %bx, bios_crtc_base
>
> Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking the=
bhyve
> device node in /dev/vmm with xxd(1), I find the words at these addresse=
s to be
> Uninitialised:
>
> 00000400: 0000 ..
> 00000483: 0000 ..
>
> I=E2=80=99m not sure where to go from here. Is this a bug in bhyve(4), =
should these
> values be initialised somehow, or should I patch rumpkernel(7) to skip =
this check
> when running on bhyve(4)?
I=E2=80=99ve chased this bug down a bit further to what I believe is an i=
ssue with the
rumprun toolchain I am building on FreeBSD with the misc/rumprun port [1]=
=2E
objdump -t helloer-rumprun.elf list a number of symbols in the *COM* sect=
ion, which
holds unallocated C external variables [2]:
objdump -t helloer-rumprun.elf | grep \*COM\*
00000001 l O *COM* 00000001 pic1mask
00000004 l O *COM* 00000004 pgalloc_totalkb
00000004 l O *COM* 00000004 pgalloc_usedkb
00001000 l O *COM* 00000020 multiboot_cmdline
00000002 l O *COM* 00000002 bios_crtc_base
00000001 l O *COM* 00000001 pic2mask
00000002 l O *COM* 00000002 bios_com1_base
As the pagetable in pagetable.s maps the first page as non-present, acces=
sing any
of these will result in a fault. I=E2=80=99m pretty sure that these shoul=
dn=E2=80=99t be undefined.
A build on Linux (which boots fine) shows these not to be uninitialised:
00000000003e3480 g O .bss 0000000000000002 bios_com1_base
00000000003e44a0 g O .bss 0000000000000002 bios_crtc_base
Further down the rabbit hole, this goes on in rumprun.o:
On Linux, bios_crtc_base is not a local symbol:
0000000000000002 O *COM* 0000000000000002 bios_crtc_base
0000000000000002 O *COM* 0000000000000002 bios_com1_base
While on FreeBSD, they are marked as local:
0000000000000002 l O *COM* 0000000000000002 bios_crtc_base
0000000000000002 l O *COM* 0000000000000002 bios_com1_base
Fabian
[1] https://svnweb.freebsd.org/ports/head/misc/rumprun/Makefile?view=3Dma=
rkup&pathrev=3D459195
[2] http://man7.org/linux/man-pages/man5/elf.5.html / SHN_COMMON
--=_MailMate_969387E7-81FE-4CE4-B9E3-F313ED190F6E_=
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename=signature.asc
Content-Type: application/pgp-signature; name=signature.asc
-----BEGIN PGP SIGNATURE-----
iQJWBAEBCgBAFiEEX6JoxdmEemcFacQZmealkcs85+YFAlqiyDgiHGZhYmlhbi5m
cmV5ZXJAcGh5c2lrLnR1LWJlcmxpbi5kZQAKCRCZ5qWRyzzn5ve+EAC3okXKpevy
pZecEHqzMRZvT69OtLYYADqwUWxUatoXWv+X9jzyYG+L7XGf2w1EJSEAKDbGpYb9
6pLWXk3Yd38HHL1w0nd5UhOynunw/ru9Ka6vtZYJcQHi39UOVdgqIm7s9x/0HmOv
/7f25q5eDaBLRYqPFUpwLegJ46UVtCGJu2eN1EyeTifx/yNw0DTQDo96JPY0SPhE
jgoX1eTggerv5Rn1O8hhToVibdNBQLcv+9uM1CET5EgWAOwEDcvZAlgpCq+eu3x8
9yEfy5UgIKMa0wgq7aUzqfGxdhLdnE5A2pHyits3TFcev/HXhzsP471Z0lAGDe9z
ap+hG2gcCsEwLjA3UmjtGXD2nBO7yI+7l/me/jfKsSGhoyoA1F+PNa9ElUbmKabz
wBE0tylxv1CC47r/V/vWcnvszpmIklaYg+on3AYM9Y0MCmgwKzSjE+uUsgzCkIBt
mxx3JukFVGFiUQldTKhnxcorvmeArYtUf3mGh2p/wOp8ZaZ9loUJ5ZmmT046/ssC
yOll/2Tbas5uuDPFeZw+iNpfM2POeQZ5Mjm/rJ3ZoO0EYb3WXo6IiDeUrkvJwW0y
UDNTrx1v2jLoZi3VR5uqA3VBfng4RRkBWVb4GofM/N0jFfTaqzNbYzA37lypssxa
2e14RvWdO8gU9lWtcDCr/7Ah2NUJsizw1A==
=6hrC
-----END PGP SIGNATURE-----
--=_MailMate_969387E7-81FE-4CE4-B9E3-F313ED190F6E_=--
From owner-freebsd-virtualization@freebsd.org Sat Mar 10 00:32:24 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90F0BF34B50
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 10 Mar 2018 00:32:24 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.ysv.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4E6B175C8F
for <freebsd-virtualization@FreeBSD.org>; Sat, 10 Mar 2018 00:32:24 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org (kenobi.freebsd.org
[IPv6:2001:1900:2254:206a::16:76])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8CC62154E8
for <freebsd-virtualization@FreeBSD.org>; Sat, 10 Mar 2018 00:32:23 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org ([127.0.1.118])
by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w2A0WN6V038326
for <freebsd-virtualization@FreeBSD.org>; Sat, 10 Mar 2018 00:32:23 GMT
(envelope-from bugzilla-noreply@freebsd.org)
Received: (from www@localhost)
by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w2A0WN5G038325
for freebsd-virtualization@FreeBSD.org; Sat, 10 Mar 2018 00:32:23 GMT
(envelope-from bugzilla-noreply@freebsd.org)
X-Authentication-Warning: kenobi.freebsd.org: www set sender to
bugzilla-noreply@freebsd.org using -f
From: bugzilla-noreply@freebsd.org
To: freebsd-virtualization@FreeBSD.org
Subject: [Bug 213333] FreeBSD 11-RELEASE fails to boot under KVM/Qemu
Date: Sat, 10 Mar 2018 00:32:23 +0000
X-Bugzilla-Reason: AssignedTo
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: Base System
X-Bugzilla-Component: kern
X-Bugzilla-Version: 11.0-STABLE
X-Bugzilla-Keywords:
X-Bugzilla-Severity: Affects Only Me
X-Bugzilla-Who: mainland@apeiron.net
X-Bugzilla-Status: New
X-Bugzilla-Resolution:
X-Bugzilla-Priority: ---
X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields:
Message-ID: <bug-213333-27103-Q0Q9oFEdCx@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-213333-27103@https.bugs.freebsd.org/bugzilla/>
References: <bug-213333-27103@https.bugs.freebsd.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Mar 2018 00:32:24 -0000
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213333
--- Comment #19 from mainland@apeiron.net ---
I've attached a patch that fixes this regression in bug #213155.
--=20
You are receiving this mail because:
You are the assignee for the bug.=
From owner-freebsd-virtualization@freebsd.org Sat Mar 10 15:15:32 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35DCBF40A07
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 10 Mar 2018 15:15:32 +0000 (UTC)
(envelope-from bogorodskiy@gmail.com)
Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com
[IPv6:2a00:1450:4010:c07::229])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G2" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 8E3157794F
for <freebsd-virtualization@freebsd.org>; Sat, 10 Mar 2018 15:15:31 +0000 (UTC)
(envelope-from bogorodskiy@gmail.com)
Received: by mail-lf0-x229.google.com with SMTP id t132-v6so750019lfe.2
for <freebsd-virtualization@freebsd.org>; Sat, 10 Mar 2018 07:15:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=sender:date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=DohFJPVl4bcMpNSNm8k4biLxf7oh0QIT2eF9bwe4yWA=;
b=FOzH5DsSyNSv9hBEm7tBn1TT3KYUxxfeFMYmqIOhB5w+lhJ+fv7f0nhV/3+9epmBf3
c93fFpwNp+3FMWu4BVqFl/YwePipc2DayzkIcxArlYFILXTHx7ywMQaMldiAmzN4NMPd
Yy/pBr5sTuUeGRa3+yuAWxr474ceyaFhx45CurN7KMUJ91OEZaJGmZFJJ9kSaN6Feqm5
LnHokPdZLCXNBZc2MveOSzEJMVnwqczl9d2NFp9OoB1yyR7ufInIElUC3Phxtj288rKV
/+TiVOeiImKc5OQrZkWHCt9q5meyAjfRtLHUcTFeQOxet3Uve78BCU7eLSdiSQZodOpq
tmZw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
:references:mime-version:content-disposition:in-reply-to:user-agent;
bh=DohFJPVl4bcMpNSNm8k4biLxf7oh0QIT2eF9bwe4yWA=;
b=MXJh9FjYITvWxhfhDdRlbCDDvHJ2eBTDV4fCREfecMg1osBmHLoa5fud3+bjARKbaH
oI0suFRkS4BLnaYiS7JR+xzFQoobXkvbdy9UosMDnsW/Em+S3ATOxmryO9fEqYoFmNqp
stt3gl2LbV6qd7twEOyHRWZflYZW0d+CWazg8xNipr3xUQ8XZ77HzfyXzev97t2Q3Zu/
oq0qRGIzfGy42jpWEDsqOarBvGZtOzsbuQr1HmJ3MT44u9G+/GZLQizFi7eNeqa2kvXU
rYD+uEoRESYEbaKjvYoeUo/Z+k6UgYfYsvSNHCq6HCxRXe2OLcRTLjE7dySj1xI7HhQ1
RbeQ==
X-Gm-Message-State: AElRT7HshphcaA0Ep7UYwqGjULXAGW9mQkE8rOGd6lo/bfw9uT5ZgvSI
WtP3nmmO/Vrp19WF3SYYmYBOwg==
X-Google-Smtp-Source: AG47ELvvlk9jINpj1B+D41VOwFZ/yk0UBinLQ3hgorolS1LikvLEO2O36u0QbhSdl4T6YDUkb3eN9w==
X-Received: by 2002:a19:a943:: with SMTP id
s64-v6mr1495047lfe.80.1520694930102;
Sat, 10 Mar 2018 07:15:30 -0800 (PST)
Received: from kloomba ([31.29.232.197])
by smtp.gmail.com with ESMTPSA id v83sm798101lje.53.2018.03.10.07.15.28
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Sat, 10 Mar 2018 07:15:29 -0800 (PST)
Sender: Roman Bogorodskiy <bogorodskiy@gmail.com>
Date: Sat, 10 Mar 2018 19:15:22 +0400
From: Roman Bogorodskiy <novel@FreeBSD.org>
To: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Cc: freebsd-virtualization@freebsd.org
Subject: Re: Call for testing bhyve cpu topology additions
Message-ID: <20180310151520.GA19476@kloomba>
References: <201803071714.w27HE31L054613@pdx.rh.CN85.dnsmgr.net>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="X1bOJ3K7DJ5YkBrT"
Content-Disposition: inline
In-Reply-To: <201803071714.w27HE31L054613@pdx.rh.CN85.dnsmgr.net>
User-Agent: Mutt/1.9.3 (2018-01-21)
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Mar 2018 15:15:32 -0000
--X1bOJ3K7DJ5YkBrT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Rodney W. Grimes wrote:
> There is a new version of the CPU topology review up at
> http://reviews.freebsd.org/D9930
>=20
> I would like to ask that if people can test this and provide
> feedback that they do so.
>=20
> It has some undesired side effects on vm-bhyve and probably
> other down stream tools, I am in the process of contacting
> the vm-bhyve author to see what can be done to clean up
> this output (if you are here please respond to this thread):
>=20
> src-topo # vm list
> NAME DATASTORE LOADER CPU MEMORY VNC =
AUTOSTART STATE
> issd30g1 default bhyveload cpus=3D8,sockets=3D2,cores=3D=
4,threads=3D1 1024M - No Stopped
>=20
> Notice that due to the new CPU string being much more complicated than
> the normal int it makes the output format ugly. I have another change
> to vm-bhyve that I would like to see too, and that is to move the NAME
> to the end of the line and remove the 16 character limit. I have done
> that in my local copy already. This string and its parsing are designed
> to be Qemu compatible.
>=20
> Here is a sample of my local vm-bhyve vm list output (no topology used):
> /tmp # vm list
> DATASTORE LOADER CPU MEMORY VNC AUTOSTA=
RT STATE NAME
> default bhyveload 1 128M - No =
Stopped fb-bld-10-amd64
> default bhyveload 1 128M - No =
Stopped fb-bld-11-amd64
> default bhyveload 1 128M - No =
Stopped fb-bld-11.0-p1-amd64
> default bhyveload 1 128M - No =
Stopped fb-bld-11.0-p1-i386
> default bhyveload 4 512M - No =
Stopped fb-bld-11_1-amd64
> default bhyveload 4 512M - No =
Stopped fb-bld-11_1-i386
> default bhyveload 2 256M - No =
Running (30227) fb-bld-head-amd64
>=20
> Thoughts are to teach vm-bhyve to parse the string just as bhyve does
> and only output the vCPU count. Other thoughts I have are to have
> it parse the string and output either vCPU if cores/threads is 1,
> or a simple S/C/T string.
>=20
> If you are a downstream maintainer of one of the other vm management pack=
ages
> I am open to input. The implemntation I have done should allow any exist=
ing
> tool that treated -c as a string to use the new topology without changes.
> This includes the in base vmrun.sh.
My general input on adding new features to bhyve is that it would be
nice to have a way to detect if the given bhyve version supports this
specific feature.
In this specific case it looks like this could be checked by running
bhyve -c gibberish
and checking if the error message contains 'invalid cpu topology'
string.
But generally it would be good to have some more convenient way of doing
that because running bhyve to probe each feature is pretty expensive.
By the way, looking at the DR, it seems that the usage output (bhyve -h)
wasn't updated, but maybe I'm missing that without context.
> Also people using the sysctls:
> hw.vmm.topology.cores_per_package: 1
> hw.vmm.topology.threads_per_core: 1
> can continue to do so at this time, but there is work in process to
> deprecate these, that work includes making stable/11 emit a warning
> message if they are used, and remove them in head/12.
>=20
> If I can get some significant test results back I plan to commit
> D9930 to ^head and merge it back to stable/11 3 days later.
>=20
> Thanks,
> --=20
> Rod Grimes rgrimes@freebs=
d.org
Roman Bogorodskiy
--X1bOJ3K7DJ5YkBrT
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEcBAEBAgAGBQJao/aIAAoJEMltX/4IwiJq5joH/A7qOM+9sZm3SgS9rZQRjAVS
bjh0H4UGA7wzYmADqfug0TZif+h0nn8Gz5rWhH858lZLeqzHA9TdJfht5D59I9xe
0r834O3slDD9tOtMoJyiQkqZxi8IsJbA3mw5q2Kk0wQ6vLguKgV3rRJlH0ZvexeK
LI0NVUxwjtlodbKwF7/EjzqFmGm/qylDx90D5sZGhw9wevziUwJUvTSJXMpWX/uE
1/HHhgJuBkPi6V/6IxQzsC2Td9k47uESwfPtpNK4rTvG4fXeWPHA0IlaLCRdK0J/
mx00BblXss1FlvTyPeKuTktXT8wwakwBItbaJ00U4+FrrSGrc1anqvkZUjiV7Do=
=mSPD
-----END PGP SIGNATURE-----
--X1bOJ3K7DJ5YkBrT--
From owner-freebsd-virtualization@freebsd.org Sat Mar 10 15:38:36 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9280F426B6
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 10 Mar 2018 15:38:36 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 53035783A0;
Sat, 10 Mar 2018 15:38:35 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2AFcVJa070003;
Sat, 10 Mar 2018 07:38:31 -0800 (PST)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2AFcV9N070002;
Sat, 10 Mar 2018 07:38:31 -0800 (PST) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201803101538.w2AFcV9N070002@pdx.rh.CN85.dnsmgr.net>
Subject: Re: Call for testing bhyve cpu topology additions
In-Reply-To: <20180310151520.GA19476@kloomba>
To: Roman Bogorodskiy <novel@FreeBSD.org>
Date: Sat, 10 Mar 2018 07:38:31 -0800 (PST)
CC: freebsd-virtualization@FreeBSD.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Mar 2018 15:38:37 -0000
> Rodney W. Grimes wrote:
>
> > There is a new version of the CPU topology review up at
> > http://reviews.freebsd.org/D9930
> >
> > I would like to ask that if people can test this and provide
> > feedback that they do so.
> >
> > It has some undesired side effects on vm-bhyve and probably
> > other down stream tools, I am in the process of contacting
> > the vm-bhyve author to see what can be done to clean up
> > this output (if you are here please respond to this thread):
> >
> > src-topo # vm list
> > NAME DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE
> > issd30g1 default bhyveload cpus=8,sockets=2,cores=4,threads=1 1024M - No Stopped
> >
> > Notice that due to the new CPU string being much more complicated than
> > the normal int it makes the output format ugly. I have another change
> > to vm-bhyve that I would like to see too, and that is to move the NAME
> > to the end of the line and remove the 16 character limit. I have done
> > that in my local copy already. This string and its parsing are designed
> > to be Qemu compatible.
> >
> > Here is a sample of my local vm-bhyve vm list output (no topology used):
> > /tmp # vm list
> > DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE NAME
> > default bhyveload 1 128M - No Stopped fb-bld-10-amd64
> > default bhyveload 1 128M - No Stopped fb-bld-11-amd64
> > default bhyveload 1 128M - No Stopped fb-bld-11.0-p1-amd64
> > default bhyveload 1 128M - No Stopped fb-bld-11.0-p1-i386
> > default bhyveload 4 512M - No Stopped fb-bld-11_1-amd64
> > default bhyveload 4 512M - No Stopped fb-bld-11_1-i386
> > default bhyveload 2 256M - No Running (30227) fb-bld-head-amd64
> >
> > Thoughts are to teach vm-bhyve to parse the string just as bhyve does
> > and only output the vCPU count. Other thoughts I have are to have
> > it parse the string and output either vCPU if cores/threads is 1,
> > or a simple S/C/T string.
> >
> > If you are a downstream maintainer of one of the other vm management packages
> > I am open to input. The implemntation I have done should allow any existing
> > tool that treated -c as a string to use the new topology without changes.
> > This includes the in base vmrun.sh.
>
> My general input on adding new features to bhyve is that it would be
> nice to have a way to detect if the given bhyve version supports this
> specific feature.
>
> In this specific case it looks like this could be checked by running
>
> bhyve -c gibberish
>
> and checking if the error message contains 'invalid cpu topology'
> string.
>
> But generally it would be good to have some more convenient way of doing
> that because running bhyve to probe each feature is pretty expensive.
I agree that this is a nice thing to have, but I can not think of
any other command that implements that feature. In general command
line interfaces are for human consumption. This one though is being
used extensivly by other scripts and programs.
>
> By the way, looking at the DR, it seems that the usage output (bhyve -h)
> wasn't updated, but maybe I'm missing that without context.
Good catch, I infact did miss that. I'll update it and push a new
patch to the review. Thank you.
> > Also people using the sysctls:
> > hw.vmm.topology.cores_per_package: 1
> > hw.vmm.topology.threads_per_core: 1
> > can continue to do so at this time, but there is work in process to
> > deprecate these, that work includes making stable/11 emit a warning
> > message if they are used, and remove them in head/12.
> >
> > If I can get some significant test results back I plan to commit
> > D9930 to ^head and merge it back to stable/11 3 days later.
> >
> > Thanks,
> > --
> > Rod Grimes rgrimes@freebsd.org
>
> Roman Bogorodskiy
Regards,
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-virtualization@freebsd.org Sat Mar 10 22:55:51 2018
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D178F3C04A
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 10 Mar 2018 22:55:51 +0000 (UTC)
(envelope-from martin@lucina.net)
Received: from smtp.lucina.net (smtp.lucina.net [62.176.169.44])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id E37EA87DD9
for <freebsd-virtualization@freebsd.org>; Sat, 10 Mar 2018 22:55:50 +0000 (UTC)
(envelope-from martin@lucina.net)
Received: from nodbug.lucina.net (dynrak234g-65-4-67-105.inwitelecom.net
[105.67.4.65])
by smtp.lucina.net (Postfix) with ESMTPSA id B0BC9122804;
Sat, 10 Mar 2018 23:46:09 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net;
s=dkim-201309; t=1520721969;
bh=BRUVOrvnGAPexJCTG1fOM0EbqWA8R/IFXQPBn/fRiHw=;
h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
b=gIP8CV9GABkj0BF2QtE+mMUtrTLEOTpj5G/xjtpN0YBpPCa8nRZBxj9/O/LwuXRr2
Rmb29vpiGZatFU+SwexNA0aoL7ZcmZe7ryWRQsVuGvuuqVVdW1bWnTOKeKJfUNThnT
N9KLYLLB7agdOOQU1+btprYoGlE6WMm9jZWSEH4b+Wg+E0puBxtBq9yJtENqEbpM3z
MiqXR4s2H1rqXMYcZa4Cs45rwTExnYzr2NRPv2WAwYjR2bcuA7mOkTYtPr3461qyAZ
9C3XmIV9KCPibb0aNs/L6iQC3KFwydhqYZgul+TZhjNMB/sYu1xtGnVFqW7o0pAtoq
l6LngMaesuLOA==
Received: by nodbug.lucina.net (Postfix, from userid 1000)
id 9BD882175E7B; Sat, 10 Mar 2018 23:46:07 +0100 (CET)
Date: Sat, 10 Mar 2018 23:46:07 +0100
From: Martin Lucina <martin@lucina.net>
To: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Cc: rumpkernel-users@freelists.org, freebsd-virtualization@freebsd.org
Subject: Re: rumpkernel and bhyve: triple faults
Message-ID: <20180310224607.wscuqebheq5bjxww@nodbug.lucina.net>
Mail-Followup-To: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>,
rumpkernel-users@freelists.org, freebsd-virtualization@freebsd.org
References: <C49D0E56-10A4-49D8-A843-E371395831B5@physik.tu-berlin.de>
<A3574A92-0514-44AE-BE20-6BFDAE803407@physik.tu-berlin.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <A3574A92-0514-44AE-BE20-6BFDAE803407@physik.tu-berlin.de>
User-Agent: NeoMutt/20170113 (1.7.2)
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Mar 2018 22:55:51 -0000
Hi,
On Friday, 09.03.2018 at 18:45, Fabian Freyer wrote:
> On 6 Mar 2018, at 7:45, Fabian Freyer wrote:
> > Tracking down bios_crtc_base, I find that it’s loaded in
> > rumprun/platform/hw/arch/amd64/locore.S:70:
> >
> > /* save BIOS data area values */
> > movw BIOS_COM1_BASE, %bx
> > movw %bx, bios_com1_base
> > movw BIOS_CRTC_BASE, %bx
> > movw %bx, bios_crtc_base
> >
> > Where BIOS_CRTC_BASE is 0x463 and BIOS_COM1_BASE is 0x400. Checking the bhyve
> > device node in /dev/vmm with xxd(1), I find the words at these addresses to be
> > Uninitialised:
> >
> > 00000400: 0000 ..
> > 00000483: 0000 ..
> >
> > I’m not sure where to go from here. Is this a bug in bhyve(4), should these
> > values be initialised somehow, or should I patch rumpkernel(7) to skip this check
> > when running on bhyve(4)?
You probably want to use a serial console rather than VGA on bhyve in any
case, so you'll want to add the appropriate checks to hypervisor.c and
cons.c.
> I’ve chased this bug down a bit further to what I believe is an issue with the
> rumprun toolchain I am building on FreeBSD with the misc/rumprun port [1].
>
> objdump -t helloer-rumprun.elf list a number of symbols in the *COM* section, which
> holds unallocated C external variables [2]:
>
> objdump -t helloer-rumprun.elf | grep \*COM\*
> 00000001 l O *COM* 00000001 pic1mask
> 00000004 l O *COM* 00000004 pgalloc_totalkb
> 00000004 l O *COM* 00000004 pgalloc_usedkb
> 00001000 l O *COM* 00000020 multiboot_cmdline
> 00000002 l O *COM* 00000002 bios_crtc_base
> 00000001 l O *COM* 00000001 pic2mask
> 00000002 l O *COM* 00000002 bios_com1_base
>
> As the pagetable in pagetable.s maps the first page as non-present, accessing any
> of these will result in a fault. I’m pretty sure that these shouldn’t be undefined.
>
> A build on Linux (which boots fine) shows these not to be uninitialised:
> 00000000003e3480 g O .bss 0000000000000002 bios_com1_base
> 00000000003e44a0 g O .bss 0000000000000002 bios_crtc_base
When you write "which boots fine", I presume you're referring to booting on
bhyve?
> Further down the rabbit hole, this goes on in rumprun.o:
>
> On Linux, bios_crtc_base is not a local symbol:
> 0000000000000002 O *COM* 0000000000000002 bios_crtc_base
> 0000000000000002 O *COM* 0000000000000002 bios_com1_base
>
> While on FreeBSD, they are marked as local:
> 0000000000000002 l O *COM* 0000000000000002 bios_crtc_base
> 0000000000000002 l O *COM* 0000000000000002 bios_com1_base
That seems wrong. Can you try and force the toolchain to use the more
recent GNU ld from devel/binutils and see if that fixes the problem?
-mato