Date: Mon, 6 Jan 2025 13:56:54 +0000 From: Peter Wood <peter@alastria.net> To: =?UTF-8?Q?Corvin_K=C3=B6hne?= <corvink@freebsd.org> Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? Message-ID: <CAD-E2ieq0P03tTkuAdvg-D35ua9PQT6XuT9GHOPLc7Aqa6xGvw@mail.gmail.com> In-Reply-To: <d2e6638855f263e8f613ba06c37e091c710c27be.camel@FreeBSD.org> References: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com> <CA%2B1FSihc5EiBUSFjxoUViAYzZ3qbo%2BqpssrBkuEvQK1=O9W6uw@mail.gmail.com> <CAD-E2icA%2BHkWZJL2JVjU28ECpc8BNG1KXkpQj4Zw5nB8yPfahQ@mail.gmail.com> <d2e6638855f263e8f613ba06c37e091c710c27be.camel@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000e946f0062b0a0059 Content-Type: text/plain; charset="UTF-8" Thanks for the feedback Corvin, and thank you for the hard work you've been putting into GPU passthru. I'm reaching the end of my limited knowledge here, and I have no expectation of any further assistance - but as a status update, BIOS in CSM (with legacy video op rom, so I see the console). If you want to pass the option rom to the guest, you can use the rom option > of > passthru devices: > > -s 1/2/3,passthru,1/2/3,rom=/path/to/rom I extracted the option ROM using linux, I was able to use the /sys/devices/pci*/rom route to extract it, it seems valid at a glance (768k dump) - but no idea how to really tell. Using the patched bhyve executable to bypass gvt-d: -s 4/0/0,passthru,4/0/0,rom=/mnt/vm/intel-arc-a380.bin -s 5/0/0,passthru,5/0/0 (5/0/0 is a separate audio device exposing the audio channels of the HDMI ports). Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0) VM still fails: [ 2.508656] i915 0000:00:04.0: enabling device (0000 -> 0002) [ 2.520226] i915 0000:00:04.0: [drm] Local memory IO size: 0x000000017c800000 [ 2.520232] i915 0000:00:04.0: [drm] Local memory available: 0x000000017c800000 [ 2.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none [ 2.550829] i915 0000:00:04.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin (v2.8) [ 2.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864 [ 2.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware i915/dg2_guc_70.bin version 70.20.0 [ 2.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware i915/dg2_huc_gsc.bin version 7.10.3 [ 2.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864 [ 2.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status = 0x40000056, time = 0ms, freq = 2300MHz, ret = 0 [ 2.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status: Reset = 0, BootROM = 0x2B, UKernel = 0x00, MIA = 0x00, Auth = 0x01 [ 2.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware production part check failure [ 2.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC initialization failed -ENOEXEC [ 2.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed (-5) [ 2.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to initialize GPU, declaring it wedged! [ 2.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_set_wedged_on_init+0x34/0x50 [i915] [ 2.587048] [drm] Initialized i915 1.6.0 20230929 for 0000:00:04.0 on minor 1 Interestingly intel_gpu_top will interact with the card to a degree, it shows a render utilization (of 0%), but none of the other card capabilities. There are some very similar errors in Google which may suggest it's may not be a bhyve/passthru issue, though it could be. I need to spin up a new VM with more bleeding edge linux (or maybe even Win11) to see if it can talk to the card. https://github.com/intel-analytics/ipex-llm/issues/12122 I'll post if I get any further, but I suspect this is the end for now. Peter. -- *Peter Wood* peter@alastria.net --000000000000e946f0062b0a0059 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div>Thanks for the feedback Corvin, and = thank you for the hard work you've been putting into GPU passthru.</div= ><div><br></div><div>I'm reaching the end of my limited knowledge here,= and I have no expectation of any further assistance - but as a status upda= te, BIOS in CSM (with legacy video op rom, so I see the console).<br></div>= <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">If you wan= t to pass the option rom to the guest, you can use the rom option of<br> passthru devices:<br> <br> -s 1/2/3,passthru,1/2/3,rom=3D/path/to/rom</blockquote><div><br></div><div>= I extracted the option ROM using linux, I was able to use the /sys/devices/= pci*/rom route to extract it, it seems valid at a glance (768k dump) - but = no idea how to really tell.<br></div><div><br></div><div>Using the patched = bhyve executable to bypass gvt-d: -s 4/0/0,passthru,4/0/0,rom=3D/mnt/vm/int= el-arc-a380.bin -s 5/0/0,passthru,5/0/0</div><div>(5/0/0 is a separate audi= o device exposing the audio channels of the HDMI ports).<br><br></div><div>= Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0) V= M still fails:</div><div>[ =C2=A0 =C2=A02.508656] i915 0000:00:04.0: enabli= ng device (0000 -> 0002)<br>[ =C2=A0 =C2=A02.520226] i915 0000:00:04.0: = [drm] Local memory IO size: 0x000000017c800000<br>[ =C2=A0 =C2=A02.520232] = i915 0000:00:04.0: [drm] Local memory available: 0x000000017c800000<br>[ = =C2=A0 =C2=A02.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: oldd= ecodes=3Dio+mem,decodes=3Dnone:owns=3Dnone<br>[ =C2=A0 =C2=A02.550829] i915= 0000:00:04.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin= (v2.8)<br>[ =C2=A0 =C2=A02.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS = capture alloc size changed from 32768 to 36864<br>[ =C2=A0 =C2=A02.565855] = i915 0000:00:04.0: [drm] GT0: GuC firmware i915/dg2_guc_70.bin version 70.2= 0.0<br>[ =C2=A0 =C2=A02.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware = i915/dg2_huc_gsc.bin version 7.10.3<br>[ =C2=A0 =C2=A02.565979] i915 0000:0= 0:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864<= br>[ =C2=A0 =C2=A02.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed:= status =3D 0x40000056, time =3D 0ms, freq =3D 2300MHz, ret =3D 0<br>[ =C2= =A0 =C2=A02.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status:= Reset =3D 0, BootROM =3D 0x2B, UKernel =3D 0x00, MIA =3D 0x00, Auth =3D 0x= 01<br>[ =C2=A0 =C2=A02.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware = production part check failure<br>[ =C2=A0 =C2=A02.567077] i915 0000:00:04.0= : [drm] *ERROR* GT0: GuC initialization failed -ENOEXEC<br>[ =C2=A0 =C2=A02= .567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed (-5)<br>[= =C2=A0 =C2=A02.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to ini= tialize GPU, declaring it wedged!<br>[ =C2=A0 =C2=A02.570106] i915 0000:00:= 04.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_set_wedged_o= n_init+0x34/0x50 [i915]<br>[ =C2=A0 =C2=A02.587048] [drm] Initialized i915 = 1.6.0 20230929 for 0000:00:04.0 on minor 1</div><div><br></div><div>Interes= tingly intel_gpu_top will interact with the card to a degree, it shows a re= nder utilization (of 0%), but none of the other card capabilities. There ar= e some very similar errors in Google which may suggest it's may not be = a bhyve/passthru issue, though it could be. I need to spin up a new VM with= more bleeding edge linux (or maybe even Win11) to see if it can talk to th= e card.</div><div><br></div><div><a href=3D"https://github.com/intel-analyt= ics/ipex-llm/issues/12122">https://github.com/intel-analytics/ipex-llm/issu= es/12122</a></div><div><br></div><div>I'll post if I get any further, b= ut I suspect this is the end for now.</div><div><br></div><div>Peter.<br></= div></div><span class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"= ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><b>Peter Wood</b></div= ><div><div style=3D"font-size:12.8px"><a href=3D"mailto:peter@alastria.net"= target=3D"_blank">peter@alastria.net</a></div></div><div><br></div></div><= /div></div> --000000000000e946f0062b0a0059--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAD-E2ieq0P03tTkuAdvg-D35ua9PQT6XuT9GHOPLc7Aqa6xGvw>