Date: Tue, 16 Jul 2024 13:39:47 -0700 From: Bakul Shah <bakul@iitbombay.org> To: Kyle Evans <kevans@FreeBSD.org> Cc: virtualization@freebsd.org Subject: Re: nmdm issues with bhyve Message-ID: <E0DAB7B2-2917-4900-A538-95ED2766F186@iitbombay.org> In-Reply-To: <16dfe6b3-2ef1-4859-a5b0-a36efe452804@FreeBSD.org> References: <20240716153851.60c1ea61@PolarianBSD> <16dfe6b3-2ef1-4859-a5b0-a36efe452804@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Jul 16, 2024, at 1:15=E2=80=AFPM, Kyle Evans <kevans@FreeBSD.org> =
wrote:
>=20
> On 7/16/24 09:38, Polarian wrote:
>> Hello,
>> Over the last few days I have been trying to get nmdm working with
>> bhyve. I have discussed it within #bhyve over on libera.chat however
>> none of the suggestions so far have fixed the problem, therefore I am
>> bringing it to the mailing list for further support.
>> A rundown what I have done so far, I have a shell script with the
>> following contents:
>> #!/bin/sh
>> bhyve -c 1 -m 1G -u -H \
>> -s 0,amd_hostbridge \
>> -s 3,ahci-hd,/path/to/install.img \
>> -s 4,virtio-blk,/dev/zvol/zpool-storage/dns \
>> -s 5,virtio-net,tap0 \
>> -s 31,lpc -l com1,/dev/nmdm3A \
>> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
>> dns
>> To verify the virtual machine was in fact booting, I did test stdio,
>> and the install medium for OpenBSD does successfully boot.
>> The problem, well there is many. Following the FAQ entry [1], the
>> following flag is suggested:
>> -s 31,lpc -l com1,nmdm0A
>> (where 0 can be substituted to prevent collisions)
>> Enter problem #1, when I try this by executing the script (with doas,
>> doas sh dns.sh) I get the following:
>> Unable to initialize backend 'nmdm3A' for LPC device com1
>> Device emulation initialization error: No such file or directory
>> Now, both vmm and nmdm have been loaded, checked with the following
>> commands:
>> kldstat | grep vmm
>> kldstat | grep nmdm
>> And these have been entered into /boot/loader.conf see below:
>> vmm_load=3D"YES"
>> nmdm_load=3D"YES"
>> So they have been loaded into the kernel.
>> When looking into the problem I found an article on the FreeBSD =
forums
>> which has the same error [2], which would suggest the example in the
>> FAQ is either wrong, or deprecated and no longer works.
>> The solution from this article is to ensure to specify the /dev path, =
as
>> seen in the full script at the beginning of the email, /dev/nmdm0A,
>> this DOES work and the vm does startup.
>> Problem #2, the device doesn't show up, even though the vm is, in =
fact,
>> running.
>> ls /dev | grep nmdm
>> The above returns nothing, there is no nmdm device. I have tested it =
to
>> ensure nmdm is in fact working, using cu to open both A and B and
>> verifying that the data is being passes between them. So nmdm is
>> working!
>=20
> We did some light debugging in #freebsd on this; bhyve(8) is opening =
the nmdm pair as it should and that works fine, but I guess at some =
point (perhaps right after handoff to the OpenBSD kernel) it's getting =
closed so the device isn't around anymore by the time they try to =
observe it above. They attempted a run with stdio in otherwise the same =
configuration, and it cut off right after:
>=20
> booting hd0a:/7.5/amd64/bsd.rd: 4076463+1688576+3891240+0+708608 =
[109+464016+317541]=3D0xaa40e8
> entry point at 0x1001000
> wrmsr to register 0xc0011029(0x3) on vcpu 0
This is what I used for openbsd (later upgraded to 7.5):
bhyve -c 1 \
-s 0,hostbridge \
-s 4,ahci-hd,install74.img \
-s 5,nvme,obsd.img \
-s 10,e1000,tap6 \
-s 11,fbuf,tcp=3D0.0.0.0:5907 \
-s 20,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
-m 2G -H -w \
-l com1,/dev/nmdm4B \
obsd
exit 0
Once installed, move the "-s 4,..." line below "exit 0".
Move the "-s 11,..." line below exit if you don't want
to connect via VNC. Never tried installing 7.5 from scratch.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E0DAB7B2-2917-4900-A538-95ED2766F186>