Date: Sat, 4 Jan 2025 20:23:55 +0000 From: Peter Wood <peter@alastria.net> To: freebsd-virtualization@freebsd.org Subject: bhyve/passthru for Intel dGPU (ARC A380)? Message-ID: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--000000000000562b7a062ae72d63 Content-Type: text/plain; charset="UTF-8" Happy new year all. I've been using bhyve happily for a year or two now, multiple machines with VT-d running happily passing SAS cards and USB cards into VMs. I've reached the point where I want to pass a GPU in for accelerated encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel ARC A380, as it's encoders/decoders are pretty decent for my use case - I'd also seen that there had been success with people using the iGPU's in Intel CPUs successfully. Unfortunately after attaching the GPUs PCI device to ppt, and attempting to start a VM with it attached, I'm greated by an error that seems to focus on iGPUs? /usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars -s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=0.0.0.0:6007,w=1280,h=720,vga=io -s 7,xhci,tablet test bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. GPU passthrough might not work properly. bhyve: gvt_d_setup_opregion: Invalid OpRegion signature bhyve: gvt_d_init: Unable to setup OpRegion Device emulation initialization error: No such file or directory The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated onboard GPU (attached to the BMC), which the BIOS is configured to encourage operating systems to use as the primary display - and sure enough the loader and BSD console are presented through the BMCs ASPEED AST2500. Digging through the freebsd source tree, it appears that that pci_gvt-d.c is responsible for this, if it's Intel and a Display then attempt the set up of graphics memory. https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44 Has anyone experimented with this? Can I just patch this out, rebuild bhyve and expect a chance of success? Cheers, P. -- *Peter Wood* peter@alastria.net --000000000000562b7a062ae72d63 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Happy new year all.<br></div><div><br></div><div>I= 9;ve been using bhyve happily for a year or two now, multiple machines with= VT-d running happily passing SAS cards and USB cards into VMs.</div><div><= br></div><div>I've reached the point where I want to pass a GPU in for = accelerated encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up = an Intel ARC A380, as it's encoders/decoders are pretty decent for my u= se case - I'd also seen that there had been success with people using t= he iGPU's in Intel CPUs successfully.</div><div><br></div><div>Unfortun= ately after attaching the GPUs PCI device to ppt, and attempting to start a= VM with it attached, I'm greated by an error that seems to focus on iG= PUs?<br></div><div><br></div><div>/usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G= -l com1,/dev/nmdm202B -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI= .fd,/usr/local/var/cache/bmd/k8s-worker-3.vars -s 0,hostbridge -s 1,lpc -s = 2,nvme,/dev/zvol/vm/k8s-worker-3 -s 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-liv= e-server-amd64.iso -s 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp= =3D<a href=3D"http://0.0.0.0:6007">0.0.0.0:6007</a>,w=3D1280,h=3D720,vga=3D= io -s 7,xhci,tablet test<br>bhyve: Warning: Unable to reuse host address of= Graphics Stolen Memory. GPU passthrough might not work properly.<br>bhyve:= gvt_d_setup_opregion: Invalid OpRegion signature<br>bhyve: gvt_d_init: Una= ble to setup OpRegion<br>Device emulation initialization error: No such fil= e or directory<br></div><div><br></div><div>The machine is an AMD EPYC 7343= on a ASRock ROMED8-2T, it has a dedicated onboard GPU (attached to the BMC= ), which the BIOS is configured to encourage operating systems to use as th= e primary display - and sure enough the loader and BSD console are presente= d through the BMCs ASPEED AST2500.</div><div><br></div><div>Digging through the freebsd source= tree, it appears that that pci_gvt-d.c is responsible for this, if it'= s Intel and a Display then attempt the set up of graphics memory.</div><div= ><a href=3D"https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6c= b79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44">https://github.= com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.s= bin/bhyve/amd64/pci_gvt-d.c#L44</a></div><div><br></div><div>Has anyone exp= erimented with this? Can I just patch this out, rebuild bhyve and expect a = chance of success?</div><div><br></div><div>Cheers,</div><div><br></div><di= v>P.</div><span class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"= ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir= =3D"ltr"><div><b>Peter Wood</b></div><div><div style=3D"font-size:12.8px"><= a href=3D"mailto:peter@alastria.net" target=3D"_blank">peter@alastria.net</= a></div></div><div><br></div></div></div></div> --000000000000562b7a062ae72d63--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA>