Date: Fri, 10 Jul 2026 12:38:55 +0300 From: Roman Gromov <truegrom@gmail.com> To: freebsd-virtualization@freebsd.org Subject: bhyve PCI passthrough of Intel Arc (DG2) GPUs: two root causes identified, PRs filed (with PoC mitigations) Message-ID: <CAC=t6A7Yh3-th2BDorqecnXm_wsopY%2B3iPuf=Wao4Az==gemMw@mail.gmail.com>
index | next in thread | raw e-mail
Hi list,
Following up on the earlier thread about passing an Intel Arc A380
through to a bhyve guest (January 2025, "Unable to setup Graphics
Stolen Memory" on an EPYC host): I hit the same class of problem on
FreeBSD 15.1-RELEASE (AMD Ryzen/Cezanne host, AMD-Vi, Arc A380
8086:56a5, Ubuntu 24.04 guest via vm-bhyve) and was able to track it
down to two independent root causes. I have filed two PRs with full
analysis, reproduction data and proof-of-concept mitigations:
PR 296662: bhyve passthru caches the PCI config header while the
device is still recovering from FLR
DG2 GPUs do not complete FLR cleanly ("Transactions pending after
FLR!" on every VM start) and return 0xff to config space accesses
for up to ~1 second afterwards (measured with devctl reset).
Two consequences:
1. ppt(4)'s post-FLR pci_cfg_restore() writes into the
unresponsive device and is lost: afterwards the physical BARs
read back as zero and MEMEN/BUSMASTEREN are cleared, so all
guest MMIO returns 0xffffffff ("Device is non-operational" from
i915).
2. bhyve's cfginit() then copies the config header into the
emulated config space while the device still reads as 0xff, so
vendor/device are cached as 0xffff and the guest permanently
sees an empty slot -- silently, with no error from bhyve. A PCI
rescan in the guest does not help since the poisoned cache is
never refreshed.
Confirmed by manually restoring the BARs and command register with
pciconf(8) on the host + a PCI rescan in the guest: the GPU then
comes up fully with no code changes. A PoC (readiness poll in
cfginit() + BAR restore from the kernel's PCIOCGETBAR data) makes
plain vm stop/start cycles work reliably. The proper fix likely
belongs in pcie_flr() (poll for device readiness before
pci_cfg_restore(), like Linux's pci_dev_wait()), which would help
all consumers, not just bhyve.
PR 296663: gvt-d probe claims discrete Intel GPUs
gvt_d_probe() matches on vendor==Intel && class==display only, so
it also claims Arc/DG2 cards, and gvt_d_init() then fails VM
startup on GSM/IGD/OpRegion (the failure Peter reported in the
January 2025 thread). Restricting the probe to devices the code
actually knows how to handle (as Corvin suggested back then) fixes
it; a one-line PoC is attached to the PR.
Note the two bugs mask each other: with the FLR issue present,
gvt_d_probe() reads 0xffff as the vendor ID and silently declines
the device, so the VM starts but the GPU is invisible; with the
FLR issue fixed, every Arc passthrough instead dies in
gvt_d_init(). Whichever one you hit first, Arc passthrough does
not work.
With both PoCs applied, the A380 passes through cleanly: i915 in the
guest initializes fully (GuC/HuC loaded and authenticated),
/dev/dri/renderD128 works, and VA-API hardware transcoding
(H.264/HEVC/VP9/AV1) is functional under Jellyfin.
Full disclosure: the analysis was done interactively on the affected
hardware, and the PoC diffs were developed with LLM assistance. I am
not a C developer, so I am not proposing them for commit -- they are
attached to the PRs as reproduction aids and as a temporary mitigation
for affected users. The PRs contain everything needed (register dumps,
timings, logs) for someone to implement a proper fix, and I am happy
to test any proposed fix on this hardware.
Best regards,
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAC=t6A7Yh3-th2BDorqecnXm_wsopY%2B3iPuf=Wao4Az==gemMw>