From nobody Fri Jul 10 09:38:55 2026 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gxRcj49dgz6ksLN for ; Fri, 10 Jul 2026 09:39:33 +0000 (UTC) (envelope-from truegrom@gmail.com) Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gxRch6RX9z3LGF for ; Fri, 10 Jul 2026 09:39:32 +0000 (UTC) (envelope-from truegrom@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20251104 header.b=o7zAhDiF; arc=pass ("google.com:s=arc-20260327:i=1"); spf=pass (mx1.freebsd.org: domain of truegrom@gmail.com designates 2607:f8b0:4864:20::102a as permitted sender) smtp.mailfrom=truegrom@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-381891a9525so932305a91.3 for ; Fri, 10 Jul 2026 02:39:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1783676372; cv=none; d=google.com; s=arc-20260327; b=OHp1fiZbsxHyX2jqVJwU6wE6UJO31rgZGaRrm45FaJsVKEbHAD2RoMFA6cPfJHNl6+ WHs7BPhPHfI2/xl/HD4HZ0DZD4pU0vrt8cNimijcjmydzLLhF34vedSnYpEr9lXkroS0 cqtOtf16pdus0mYQQJo0O8cDi+8RKgtRTJvjW3wDkXL1UXIrBnNNpGZ4iRTMy6cfwnMD utQFoGgVZbjd7bPzACzmoaJpuHRSV1IuOqFybAsDC8kli2qACNAaTXs8vRsQunn5NG38 VKVlG9dVXl3Igg5b3ZHK3swcRmElOGe693/98+2+mvphfpkqZhbiAE4Vdn7iZUKCAuM6 s9Hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=to:subject:message-id:date:from:mime-version:dkim-signature; bh=GTXSD22dhkstENeesCINb3bc13Iuy+55Hy8VVx4LDCM=; fh=l26hILhfTsHyyw+ygmDeeLm14oRJFdCZGIazu6zV1tM=; b=pTsmL6P3NTg5FCI6m036sIikOptjYdhJ/xVajbcy+F5kwrqWHlcGPKvpTtZ16OKJv1 qemZZ8EEecRfqh94F2bhFCDr/ABtn2emxL2RfUUzN/59qgCSRQkw5cqqqrPwdrMekaku 0Df8Kd1BMXWOcKoBwbUPF15R1b+h/TZDRNmv9LWd96b6DNLd1VN2WFy3oIWXTXJWHDzR 59cVW9Styx4RjakGx/+1u8W+jg93Gfz1k65AioNRjZLHhTnzwnsbk4vyuDEgzVqwqnJn 80QOKu+OyDXNtof4suyD36JQaZLkmrwM0Le7i7FHQIOUhb/KPf2nNmwggl8PwismeARP MPJA==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783676372; x=1784281172; darn=freebsd.org; h=content-type:to:subject:message-id:date:from:mime-version:from:to :cc:subject:date:message-id:reply-to:content-type; bh=GTXSD22dhkstENeesCINb3bc13Iuy+55Hy8VVx4LDCM=; b=o7zAhDiFaZtsWoc9Gc5tPoABUHNbBVCvoNb3s1fhtAXzlid/xK3t8IRfJy0PgZeHSg J4LcJ6nmUh+Cmmxr0CvIeKAeDVCIX82YVct3dR4U8K0YJ2cUii9t3H4B9osjdNbnuAHH RSBXN9nZXowyci2t9V6GEhnYQ1y3NOmbqBSM4sKeXfFFC9ttaY0VKsq4i8pgs0djR0Kc 8OVk/B4yS19xwvY/Pxmk2oWq/Cpi4aOlMZY6fJeqyzB0BCRrbwt1sb32yYJuNoViNhYh jbpsmsOkibXcozs2ZIvm3E7n9l4+4uPLUQcL4PclRcFcPaGK7fykp2O8bNPaZ9hVQF1W jEAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783676372; x=1784281172; h=content-type:to:subject:message-id:date:from:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=GTXSD22dhkstENeesCINb3bc13Iuy+55Hy8VVx4LDCM=; b=OvA0QoPRnJQO2t/HoI2OLgP2PjS8TKiK+kWiUYuOsGJ/NVtzimJ6QUYssohJclssPZ Y2J+uI3wUrhzttWhrUAdoIYgNz51h7NB/+WcmFR9JZw6u+4DYfz8kfelSb/X5miv2Azb XZQqw2g78zo4zx5XmTQnNJsaHAqNmU7m+E9WtpaoLwGsC0hPfnoH0xMiQRbvlHCovXfg JmPppKkt4MnLfN6ETe09R36X1F92R0g2mCMz8CgJ16yplJrtq/W7wg2OKs4e/DZ3Ceiq UB4ovJULJy8Wp0Ios/DGg4yDTrPUqxOTG3euyJk/FqpotxEgvkcAzt2L42igdFhcK/qb sucA== X-Gm-Message-State: AOJu0YwjWl4QyLr0Yja6cGyxbLPH/9z0SPpKsBoZ7laUXoeJt5SGd8DG 8FZIAwxdLVv82bAfQji5a8tqI7B0QW9k63Gf5Ggw3h1HxP2OLPOYSHxJtMPE1KT7kH4MsgN642x A8SJDKE0buoqXrwNfGdyDi9MK1SfxkkTpu9mcyA4= X-Gm-Gg: AfdE7cnLyMjbYhL1l1lc07J4TaxC9rL21npYUwug4humBuMH+Wot7Mg2Xqx8wkIQe4W GPzb324uzxQhrEWx94+NQw/vQprCcYHN9lXgC5kPxjsOlmS1VlyZU2kl7LvPetd5+cMVb5qXpJV EhJQvRGEhMtUCkjXlcTLU4TvFpV50PtVss/+wTIrTRNa+tL53P0WdXwd6l+o517Tib8+LEMxqZp k0RBIAVcfRzvL1LZPlR8iwnSAKJ/9Yqsv5LaCZZEfOGSDKLtQzKO1NAWjwY4h+9Q/XJd46YNuF9 nQ== X-Received: by 2002:a17:90a:e184:b0:37f:fb1d:63fa with SMTP id 98e67ed59e1d1-3893fc693famr10967044a91.15.1783676371720; Fri, 10 Jul 2026 02:39:31 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 From: Roman Gromov Date: Fri, 10 Jul 2026 12:38:55 +0300 X-Gm-Features: AVVi8Cet7ue5TpYzIcBSbJdS5DLpS5io6NAvgOMii_H1jyewHPO6OfbitEoeZ1k Message-ID: Subject: bhyve PCI passthrough of Intel Arc (DG2) GPUs: two root causes identified, PRs filed (with PoC mitigations) To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-4.78 / 15.00]; ARC_ALLOW(-1.00)[google.com:s=arc-20260327:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.78)[-0.784]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20251104]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4864::/56:c]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::102a:from] X-Rspamd-Queue-Id: 4gxRch6RX9z3LGF X-Spamd-Bar: ---- Hi list, Following up on the earlier thread about passing an Intel Arc A380 through to a bhyve guest (January 2025, "Unable to setup Graphics Stolen Memory" on an EPYC host): I hit the same class of problem on FreeBSD 15.1-RELEASE (AMD Ryzen/Cezanne host, AMD-Vi, Arc A380 8086:56a5, Ubuntu 24.04 guest via vm-bhyve) and was able to track it down to two independent root causes. I have filed two PRs with full analysis, reproduction data and proof-of-concept mitigations: PR 296662: bhyve passthru caches the PCI config header while the device is still recovering from FLR DG2 GPUs do not complete FLR cleanly ("Transactions pending after FLR!" on every VM start) and return 0xff to config space accesses for up to ~1 second afterwards (measured with devctl reset). Two consequences: 1. ppt(4)'s post-FLR pci_cfg_restore() writes into the unresponsive device and is lost: afterwards the physical BARs read back as zero and MEMEN/BUSMASTEREN are cleared, so all guest MMIO returns 0xffffffff ("Device is non-operational" from i915). 2. bhyve's cfginit() then copies the config header into the emulated config space while the device still reads as 0xff, so vendor/device are cached as 0xffff and the guest permanently sees an empty slot -- silently, with no error from bhyve. A PCI rescan in the guest does not help since the poisoned cache is never refreshed. Confirmed by manually restoring the BARs and command register with pciconf(8) on the host + a PCI rescan in the guest: the GPU then comes up fully with no code changes. A PoC (readiness poll in cfginit() + BAR restore from the kernel's PCIOCGETBAR data) makes plain vm stop/start cycles work reliably. The proper fix likely belongs in pcie_flr() (poll for device readiness before pci_cfg_restore(), like Linux's pci_dev_wait()), which would help all consumers, not just bhyve. PR 296663: gvt-d probe claims discrete Intel GPUs gvt_d_probe() matches on vendor==Intel && class==display only, so it also claims Arc/DG2 cards, and gvt_d_init() then fails VM startup on GSM/IGD/OpRegion (the failure Peter reported in the January 2025 thread). Restricting the probe to devices the code actually knows how to handle (as Corvin suggested back then) fixes it; a one-line PoC is attached to the PR. Note the two bugs mask each other: with the FLR issue present, gvt_d_probe() reads 0xffff as the vendor ID and silently declines the device, so the VM starts but the GPU is invisible; with the FLR issue fixed, every Arc passthrough instead dies in gvt_d_init(). Whichever one you hit first, Arc passthrough does not work. With both PoCs applied, the A380 passes through cleanly: i915 in the guest initializes fully (GuC/HuC loaded and authenticated), /dev/dri/renderD128 works, and VA-API hardware transcoding (H.264/HEVC/VP9/AV1) is functional under Jellyfin. Full disclosure: the analysis was done interactively on the affected hardware, and the PoC diffs were developed with LLM assistance. I am not a C developer, so I am not proposing them for commit -- they are attached to the PRs as reproduction aids and as a temporary mitigation for affected users. The PRs contain everything needed (register dumps, timings, logs) for someone to implement a proper fix, and I am happy to test any proposed fix on this hardware. Best regards,