Date: Sat, 4 Jan 2025 21:33:28 +0100 From: Mario Marietto <marietto2008@gmail.com> To: Peter Wood <peter@alastria.net> Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? Message-ID: <CA%2B1FSihc5EiBUSFjxoUViAYzZ3qbo%2BqpssrBkuEvQK1=O9W6uw@mail.gmail.com> In-Reply-To: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com> References: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Hi Peter, Please make sure your GPU is booted in UEFI and not CSM mode. This can be changed in the host BIOS. Cheers. On Sat, Jan 4, 2025 at 9:24 PM Peter Wood <peter@alastria.net> wrote: > Happy new year all. > > I've been using bhyve happily for a year or two now, multiple machines > with VT-d running happily passing SAS cards and USB cards into VMs. > > I've reached the point where I want to pass a GPU in for accelerated > encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel ARC > A380, as it's encoders/decoders are pretty decent for my use case - I'd > also seen that there had been success with people using the iGPU's in Intel > CPUs successfully. > > Unfortunately after attaching the GPUs PCI device to ppt, and attempting > to start a VM with it attached, I'm greated by an error that seems to focus > on iGPUs? > > /usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l > bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars > -s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s > 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s > 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=0.0.0.0:6007,w=1280,h=720,vga=io > -s 7,xhci,tablet test > bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. > GPU passthrough might not work properly. > bhyve: gvt_d_setup_opregion: Invalid OpRegion signature > bhyve: gvt_d_init: Unable to setup OpRegion > Device emulation initialization error: No such file or directory > > The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated > onboard GPU (attached to the BMC), which the BIOS is configured to > encourage operating systems to use as the primary display - and sure enough > the loader and BSD console are presented through the BMCs ASPEED AST2500. > > Digging through the freebsd source tree, it appears that that pci_gvt-d.c > is responsible for this, if it's Intel and a Display then attempt the set > up of graphics memory. > > https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44 > > Has anyone experimented with this? Can I just patch this out, rebuild > bhyve and expect a chance of success? > > Cheers, > > P. > -- > *Peter Wood* > peter@alastria.net > > -- Mario. [-- Attachment #2 --] <div dir="ltr"><p class="MsoNormal">Hi Peter,</p> <p class="MsoNormal"><u></u> <u></u></p> <p class="MsoNormal">Please make sure your GPU is booted in UEFI and not CSM mode. This can be changed in the <span class="gmail-il">host</span> BIOS.</p> <p class="MsoNormal">Cheers. <br></p><p class="MsoNormal"><br></p></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sat, Jan 4, 2025 at 9:24 PM Peter Wood <<a href="mailto:peter@alastria.net">peter@alastria.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Happy new year all.<br></div><div><br></div><div>I've been using bhyve happily for a year or two now, multiple machines with VT-d running happily passing SAS cards and USB cards into VMs.</div><div><br></div><div>I've reached the point where I want to pass a GPU in for accelerated encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel ARC A380, as it's encoders/decoders are pretty decent for my use case - I'd also seen that there had been success with people using the iGPU's in Intel CPUs successfully.</div><div><br></div><div>Unfortunately after attaching the GPUs PCI device to ppt, and attempting to start a VM with it attached, I'm greated by an error that seems to focus on iGPUs?<br></div><div><br></div><div>/usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars -s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=<a href="http://0.0.0.0:6007" target="_blank">0.0.0.0:6007</a>,w=1280,h=720,vga=io -s 7,xhci,tablet test<br>bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. GPU passthrough might not work properly.<br>bhyve: gvt_d_setup_opregion: Invalid OpRegion signature<br>bhyve: gvt_d_init: Unable to setup OpRegion<br>Device emulation initialization error: No such file or directory<br></div><div><br></div><div>The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated onboard GPU (attached to the BMC), which the BIOS is configured to encourage operating systems to use as the primary display - and sure enough the loader and BSD console are presented through the BMCs ASPEED AST2500.</div><div><br></div><div>Digging through the freebsd source tree, it appears that that pci_gvt-d.c is responsible for this, if it's Intel and a Display then attempt the set up of graphics memory.</div><div><a href="https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44" target="_blank">https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44</a></div><div><br></div><div>Has anyone experimented with this? Can I just patch this out, rebuild bhyve and expect a chance of success?</div><div><br></div><div>Cheers,</div><div><br></div><div>P.</div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><b>Peter Wood</b></div><div><div style="font-size:12.8px"><a href="mailto:peter@alastria.net" target="_blank">peter@alastria.net</a></div></div><div><br></div></div></div></div> </blockquote></div><div><br clear="all"></div><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Mario.<br></div>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSihc5EiBUSFjxoUViAYzZ3qbo%2BqpssrBkuEvQK1=O9W6uw>