Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 4 Jan 2025 20:23:55 +0000
From:      Peter Wood <peter@alastria.net>
To:        freebsd-virtualization@freebsd.org
Subject:   bhyve/passthru for Intel dGPU (ARC A380)?
Message-ID:  <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
--000000000000562b7a062ae72d63
Content-Type: text/plain; charset="UTF-8"

Happy new year all.

I've been using bhyve happily for a year or two now, multiple machines with
VT-d running happily passing SAS cards and USB cards into VMs.

I've reached the point where I want to pass a GPU in for accelerated
encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel ARC
A380, as it's encoders/decoders are pretty decent for my use case - I'd
also seen that there had been success with people using the iGPU's in Intel
CPUs successfully.

Unfortunately after attaching the GPUs PCI device to ppt, and attempting to
start a VM with it attached, I'm greated by an error that seems to focus on
iGPUs?

/usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l
bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars
-s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s
3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s
5,virtio-net,tap9 -s 4,passthru,4/0/0 -s
6,fbuf,tcp=0.0.0.0:6007,w=1280,h=720,vga=io
-s 7,xhci,tablet test
bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. GPU
passthrough might not work properly.
bhyve: gvt_d_setup_opregion: Invalid OpRegion signature
bhyve: gvt_d_init: Unable to setup OpRegion
Device emulation initialization error: No such file or directory

The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated
onboard GPU (attached to the BMC), which the BIOS is configured to
encourage operating systems to use as the primary display - and sure enough
the loader and BSD console are presented through the BMCs ASPEED AST2500.

Digging through the freebsd source tree, it appears that that pci_gvt-d.c
is responsible for this, if it's Intel and a Display then attempt the set
up of graphics memory.
https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44

Has anyone experimented with this? Can I just patch this out, rebuild bhyve
and expect a chance of success?

Cheers,

P.
-- 
*Peter Wood*
peter@alastria.net

--000000000000562b7a062ae72d63
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Happy new year all.<br></div><div><br></div><div>I&#3=
9;ve been using bhyve happily for a year or two now, multiple machines with=
 VT-d running happily passing SAS cards and USB cards into VMs.</div><div><=
br></div><div>I&#39;ve reached the point where I want to pass a GPU in for =
accelerated encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up =
an Intel ARC A380, as it&#39;s encoders/decoders are pretty decent for my u=
se case - I&#39;d also seen that there had been success with people using t=
he iGPU&#39;s in Intel CPUs successfully.</div><div><br></div><div>Unfortun=
ately after attaching the GPUs PCI device to ppt, and attempting to start a=
 VM with it attached, I&#39;m greated by an error that seems to focus on iG=
PUs?<br></div><div><br></div><div>/usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G=
 -l com1,/dev/nmdm202B -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI=
.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars -s 0,hostbridge -s 1,lpc -s =
2,nvme,/dev/zvol/vm/k8s-worker-3 -s 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-liv=
e-server-amd64.iso -s 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=
=3D<a href=3D"http://0.0.0.0:6007">0.0.0.0:6007</a>,w=3D1280,h=3D720,vga=3D=
io -s 7,xhci,tablet test<br>bhyve: Warning: Unable to reuse host address of=
 Graphics Stolen Memory. GPU passthrough might not work properly.<br>bhyve:=
 gvt_d_setup_opregion: Invalid OpRegion signature<br>bhyve: gvt_d_init: Una=
ble to setup OpRegion<br>Device emulation initialization error: No such fil=
e or directory<br></div><div><br></div><div>The machine is an AMD EPYC 7343=
 on a ASRock ROMED8-2T, it has a dedicated onboard GPU (attached to the BMC=
), which the BIOS is configured to encourage operating systems to use as th=
e primary display - and sure enough the loader and BSD console are presente=
d through the BMCs
ASPEED AST2500.</div><div><br></div><div>Digging through the freebsd source=
 tree, it appears that that pci_gvt-d.c is responsible for this, if it&#39;=
s Intel and a Display then attempt the set up of graphics memory.</div><div=
><a href=3D"https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6c=
b79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44">https://github.=
com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.s=
bin/bhyve/amd64/pci_gvt-d.c#L44</a></div><div><br></div><div>Has anyone exp=
erimented with this? Can I just patch this out, rebuild bhyve and expect a =
chance of success?</div><div><br></div><div>Cheers,</div><div><br></div><di=
v>P.</div><span class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"=
ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=
=3D"ltr"><div><b>Peter Wood</b></div><div><div style=3D"font-size:12.8px"><=
a href=3D"mailto:peter@alastria.net" target=3D"_blank">peter@alastria.net</=
a></div></div><div><br></div></div></div></div>

--000000000000562b7a062ae72d63--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA>