Date: Sat, 4 Jan 2025 21:33:28 +0100 From: Mario Marietto <marietto2008@gmail.com> To: Peter Wood <peter@alastria.net> Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? Message-ID: <CA%2B1FSihc5EiBUSFjxoUViAYzZ3qbo%2BqpssrBkuEvQK1=O9W6uw@mail.gmail.com> In-Reply-To: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com> References: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000001dfe1062ae751d0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Peter, Please make sure your GPU is booted in UEFI and not CSM mode. This can be changed in the host BIOS. Cheers. On Sat, Jan 4, 2025 at 9:24=E2=80=AFPM Peter Wood <peter@alastria.net> wrot= e: > Happy new year all. > > I've been using bhyve happily for a year or two now, multiple machines > with VT-d running happily passing SAS cards and USB cards into VMs. > > I've reached the point where I want to pass a GPU in for accelerated > encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Intel A= RC > A380, as it's encoders/decoders are pretty decent for my use case - I'd > also seen that there had been success with people using the iGPU's in Int= el > CPUs successfully. > > Unfortunately after attaching the GPUs PCI device to ppt, and attempting > to start a VM with it attached, I'm greated by an error that seems to foc= us > on iGPUs? > > /usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l > bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cache= /bmd/k8s-worker-3.vars > -s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s > 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s > 5,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=3D0.0.0.0:6007,w=3D12= 80,h=3D720,vga=3Dio > -s 7,xhci,tablet test > bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. > GPU passthrough might not work properly. > bhyve: gvt_d_setup_opregion: Invalid OpRegion signature > bhyve: gvt_d_init: Unable to setup OpRegion > Device emulation initialization error: No such file or directory > > The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedicated > onboard GPU (attached to the BMC), which the BIOS is configured to > encourage operating systems to use as the primary display - and sure enou= gh > the loader and BSD console are presented through the BMCs ASPEED AST2500. > > Digging through the freebsd source tree, it appears that that pci_gvt-d.c > is responsible for this, if it's Intel and a Display then attempt the set > up of graphics memory. > > https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e183= 0b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44 > > Has anyone experimented with this? Can I just patch this out, rebuild > bhyve and expect a chance of success? > > Cheers, > > P. > -- > *Peter Wood* > peter@alastria.net > > --=20 Mario. --00000000000001dfe1062ae751d0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><p class=3D"MsoNormal">Hi Peter,</p> <p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p> <p class=3D"MsoNormal">Please make sure your GPU is booted in UEFI and not = CSM mode. This can be changed in the <span class=3D"gmail-il">host</span> B= IOS.</p> <p class=3D"MsoNormal">Cheers. <br></p><p class=3D"MsoNormal"><br></p></div= ><br><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" clas= s=3D"gmail_attr">On Sat, Jan 4, 2025 at 9:24=E2=80=AFPM Peter Wood <<a h= ref=3D"mailto:peter@alastria.net">peter@alastria.net</a>> wrote:<br></di= v><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;borde= r-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div>H= appy new year all.<br></div><div><br></div><div>I've been using bhyve h= appily for a year or two now, multiple machines with VT-d running happily p= assing SAS cards and USB cards into VMs.</div><div><br></div><div>I've = reached the point where I want to pass a GPU in for accelerated encoding/de= coding/etc (scrypted, tdarr, jellyfin). I picked up an Intel ARC A380, as i= t's encoders/decoders are pretty decent for my use case - I'd also = seen that there had been success with people using the iGPU's in Intel = CPUs successfully.</div><div><br></div><div>Unfortunately after attaching t= he GPUs PCI device to ppt, and attempting to start a VM with it attached, I= 'm greated by an error that seems to focus on iGPUs?<br></div><div><br>= </div><div>/usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B = -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,/usr/local/var/cach= e/bmd/k8s-worker-3.vars -s 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s= -worker-3 -s 3,ahci-cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s 5= ,virtio-net,tap9 -s 4,passthru,4/0/0 -s 6,fbuf,tcp=3D<a href=3D"http://0.0.= 0.0:6007" target=3D"_blank">0.0.0.0:6007</a>,w=3D1280,h=3D720,vga=3Dio -s 7= ,xhci,tablet test<br>bhyve: Warning: Unable to reuse host address of Graphi= cs Stolen Memory. GPU passthrough might not work properly.<br>bhyve: gvt_d_= setup_opregion: Invalid OpRegion signature<br>bhyve: gvt_d_init: Unable to = setup OpRegion<br>Device emulation initialization error: No such file or di= rectory<br></div><div><br></div><div>The machine is an AMD EPYC 7343 on a A= SRock ROMED8-2T, it has a dedicated onboard GPU (attached to the BMC), whic= h the BIOS is configured to encourage operating systems to use as the prima= ry display - and sure enough the loader and BSD console are presented throu= gh the BMCs ASPEED AST2500.</div><div><br></div><div>Digging through the freebsd source= tree, it appears that that pci_gvt-d.c is responsible for this, if it'= s Intel and a Display then attempt the set up of graphics memory.</div><div= ><a href=3D"https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6c= b79075e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44" target=3D"_blan= k">https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075e18= 30b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44</a></div><div><br></div><= div>Has anyone experimented with this? Can I just patch this out, rebuild b= hyve and expect a chance of success?</div><div><br></div><div>Cheers,</div>= <div><br></div><div>P.</div><span class=3D"gmail_signature_prefix">-- </spa= n><br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><b>P= eter Wood</b></div><div><div style=3D"font-size:12.8px"><a href=3D"mailto:p= eter@alastria.net" target=3D"_blank">peter@alastria.net</a></div></div><div= ><br></div></div></div></div> </blockquote></div><div><br clear=3D"all"></div><br><span class=3D"gmail_si= gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Ma= rio.<br></div> --00000000000001dfe1062ae751d0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSihc5EiBUSFjxoUViAYzZ3qbo%2BqpssrBkuEvQK1=O9W6uw>