Date: Mon, 6 Jan 2025 13:56:54 +0000 From: Peter Wood <peter@alastria.net> To: =?UTF-8?Q?Corvin_K=C3=B6hne?= <corvink@freebsd.org> Cc: freebsd-virtualization@freebsd.org Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? Message-ID: <CAD-E2ieq0P03tTkuAdvg-D35ua9PQT6XuT9GHOPLc7Aqa6xGvw@mail.gmail.com> In-Reply-To: <d2e6638855f263e8f613ba06c37e091c710c27be.camel@FreeBSD.org> References: <CAD-E2if_q6JreqPWiFBgPc=KHeP12Pq_E2R4m3ZxvGC3g87ZHA@mail.gmail.com> <CA%2B1FSihc5EiBUSFjxoUViAYzZ3qbo%2BqpssrBkuEvQK1=O9W6uw@mail.gmail.com> <CAD-E2icA%2BHkWZJL2JVjU28ECpc8BNG1KXkpQj4Zw5nB8yPfahQ@mail.gmail.com> <d2e6638855f263e8f613ba06c37e091c710c27be.camel@FreeBSD.org>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Thanks for the feedback Corvin, and thank you for the hard work you've been putting into GPU passthru. I'm reaching the end of my limited knowledge here, and I have no expectation of any further assistance - but as a status update, BIOS in CSM (with legacy video op rom, so I see the console). If you want to pass the option rom to the guest, you can use the rom option > of > passthru devices: > > -s 1/2/3,passthru,1/2/3,rom=/path/to/rom I extracted the option ROM using linux, I was able to use the /sys/devices/pci*/rom route to extract it, it seems valid at a glance (768k dump) - but no idea how to really tell. Using the patched bhyve executable to bypass gvt-d: -s 4/0/0,passthru,4/0/0,rom=/mnt/vm/intel-arc-a380.bin -s 5/0/0,passthru,5/0/0 (5/0/0 is a separate audio device exposing the audio channels of the HDMI ports). Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0) VM still fails: [ 2.508656] i915 0000:00:04.0: enabling device (0000 -> 0002) [ 2.520226] i915 0000:00:04.0: [drm] Local memory IO size: 0x000000017c800000 [ 2.520232] i915 0000:00:04.0: [drm] Local memory available: 0x000000017c800000 [ 2.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none [ 2.550829] i915 0000:00:04.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin (v2.8) [ 2.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864 [ 2.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware i915/dg2_guc_70.bin version 70.20.0 [ 2.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware i915/dg2_huc_gsc.bin version 7.10.3 [ 2.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864 [ 2.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status = 0x40000056, time = 0ms, freq = 2300MHz, ret = 0 [ 2.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status: Reset = 0, BootROM = 0x2B, UKernel = 0x00, MIA = 0x00, Auth = 0x01 [ 2.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware production part check failure [ 2.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC initialization failed -ENOEXEC [ 2.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed (-5) [ 2.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to initialize GPU, declaring it wedged! [ 2.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_set_wedged_on_init+0x34/0x50 [i915] [ 2.587048] [drm] Initialized i915 1.6.0 20230929 for 0000:00:04.0 on minor 1 Interestingly intel_gpu_top will interact with the card to a degree, it shows a render utilization (of 0%), but none of the other card capabilities. There are some very similar errors in Google which may suggest it's may not be a bhyve/passthru issue, though it could be. I need to spin up a new VM with more bleeding edge linux (or maybe even Win11) to see if it can talk to the card. https://github.com/intel-analytics/ipex-llm/issues/12122 I'll post if I get any further, but I suspect this is the end for now. Peter. -- *Peter Wood* peter@alastria.net [-- Attachment #2 --] <div dir="ltr"><div dir="ltr"><div>Thanks for the feedback Corvin, and thank you for the hard work you've been putting into GPU passthru.</div><div><br></div><div>I'm reaching the end of my limited knowledge here, and I have no expectation of any further assistance - but as a status update, BIOS in CSM (with legacy video op rom, so I see the console).<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">If you want to pass the option rom to the guest, you can use the rom option of<br> passthru devices:<br> <br> -s 1/2/3,passthru,1/2/3,rom=/path/to/rom</blockquote><div><br></div><div>I extracted the option ROM using linux, I was able to use the /sys/devices/pci*/rom route to extract it, it seems valid at a glance (768k dump) - but no idea how to really tell.<br></div><div><br></div><div>Using the patched bhyve executable to bypass gvt-d: -s 4/0/0,passthru,4/0/0,rom=/mnt/vm/intel-arc-a380.bin -s 5/0/0,passthru,5/0/0</div><div>(5/0/0 is a separate audio device exposing the audio channels of the HDMI ports).<br><br></div><div>Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0) VM still fails:</div><div>[ 2.508656] i915 0000:00:04.0: enabling device (0000 -> 0002)<br>[ 2.520226] i915 0000:00:04.0: [drm] Local memory IO size: 0x000000017c800000<br>[ 2.520232] i915 0000:00:04.0: [drm] Local memory available: 0x000000017c800000<br>[ 2.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none<br>[ 2.550829] i915 0000:00:04.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin (v2.8)<br>[ 2.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864<br>[ 2.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware i915/dg2_guc_70.bin version 70.20.0<br>[ 2.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware i915/dg2_huc_gsc.bin version 7.10.3<br>[ 2.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864<br>[ 2.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status = 0x40000056, time = 0ms, freq = 2300MHz, ret = 0<br>[ 2.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status: Reset = 0, BootROM = 0x2B, UKernel = 0x00, MIA = 0x00, Auth = 0x01<br>[ 2.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware production part check failure<br>[ 2.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC initialization failed -ENOEXEC<br>[ 2.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed (-5)<br>[ 2.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to initialize GPU, declaring it wedged!<br>[ 2.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_set_wedged_on_init+0x34/0x50 [i915]<br>[ 2.587048] [drm] Initialized i915 1.6.0 20230929 for 0000:00:04.0 on minor 1</div><div><br></div><div>Interestingly intel_gpu_top will interact with the card to a degree, it shows a render utilization (of 0%), but none of the other card capabilities. There are some very similar errors in Google which may suggest it's may not be a bhyve/passthru issue, though it could be. I need to spin up a new VM with more bleeding edge linux (or maybe even Win11) to see if it can talk to the card.</div><div><br></div><div><a href="https://github.com/intel-analytics/ipex-llm/issues/12122">https://github.com/intel-analytics/ipex-llm/issues/12122</a></div><div><br></div><div>I'll post if I get any further, but I suspect this is the end for now.</div><div><br></div><div>Peter.<br></div></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><b>Peter Wood</b></div><div><div style="font-size:12.8px"><a href="mailto:peter@alastria.net" target="_blank">peter@alastria.net</a></div></div><div><br></div></div></div></div>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAD-E2ieq0P03tTkuAdvg-D35ua9PQT6XuT9GHOPLc7Aqa6xGvw>