From nobody Mon Jan 6 08:10:32 2025 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YRRgt0xBVz5k8VJ for ; Mon, 06 Jan 2025 08:10:34 +0000 (UTC) (envelope-from corvink@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YRRgt0MW4z47Hr; Mon, 6 Jan 2025 08:10:34 +0000 (UTC) (envelope-from corvink@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1736151034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=I1mh08et75SWA9UQ/flD504P2/0uoIxi4Vj08Q7ls08=; b=KjSYA5zxAKT1V0s9chFCWYurM6EmY22YLhdq3qEAlGAZ1Q+bVEkacQNgfFmqDdqkK6JB7f LpNaK8Df/euKuvh57cZ317K8t3rDbX955DG15+RROxsDIiXc8Fdn51rneFk9sGxPDXD4b1 e/iR2TUYSpYm3BggUHoVglkedtzkitY5qdmxTYnIW+xXQjA6kSkyP8a677WQYfYI33Gmv4 lDOOaEN5D+8nZgxiXfkW+AV7c5Y8wOGJEXyVFkFg0r/lxwI7+TVIrxg7ZiprhtRjrqrXLb xQN3EBEesaAz9vfnd3o+LWKt8ilRfJclkA3GcK08zMfJlUj0YBz6cOwubgeKNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1736151034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=I1mh08et75SWA9UQ/flD504P2/0uoIxi4Vj08Q7ls08=; b=dohO9aXF8G6gSBIIHehIL2qvpXd/pNfI4FCWAd8+IWHMtIq9brIVc9iG50q0GLDJB11Km5 UQHzbN+DDa4haA99STjgSbdn8EjoFCFH5hDDVG1A7+42oF40M5Qg+BQr9NTS/1kA3D5RHI tFjz+PJjn0i3YWWTYczCDI4bB38yd1d0ldjbwWfufDtcDvDkgZufZXqJSiZTivKhZOQtP6 PSWL7sPEM6hgIUHq281Oa/gGAAGVXEYSENrD+cm8FnqlYII54tA1bYSP55M4aQFUMB3zVI oKbCEz2qu0WRtVJMaFtOlxwHt+5JRdkUW6S/Dw8NEs00vlWLvjLJJrcmdOg3ew== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1736151034; a=rsa-sha256; cv=none; b=mGTPNoH+EGOBIUxtc9lDkd0ECuWyXd+DTHSW1D26OhbfbvxE0aB2cWBLKEdXkHpvoshy2F jFrZ4EKQidu0acVOHlZPT3WMGqHAGVtLDcNLzSczO3zUm1VOPhpzwOw5Vbzpd3hFs40Pvh WTv+6JHxr/gF5/T37oOZBhvZtQF+PhpX3vG5i4BDT4V3jH9kzfJsZRWWCZEJj12JmQ2UqR mTIqdqghcH4zixAPszJuFcYMx0rJ53LJOGHqt9h21Y0kb4COgNswgEZVJPta2AI0ci7Brw BZ7xZ/7qFwgU6wu4lrOnCVqnPGslTKCkm6fZslSFYVyb/8+zT3ZU5Jj3eiFEvA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [IPv6:2001:9e8:dc01:5900:fc72:1b65:c1e1:a77c] (unknown [IPv6:2001:9e8:dc01:5900:fc72:1b65:c1e1:a77c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: corvink) by smtp.freebsd.org (Postfix) with ESMTPSA id 4YRRgs3sV2zHvL; Mon, 06 Jan 2025 08:10:33 +0000 (UTC) (envelope-from corvink@FreeBSD.org) Message-ID: Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? From: Corvin =?ISO-8859-1?Q?K=F6hne?= To: Peter Wood , Mario Marietto Cc: freebsd-virtualization@freebsd.org Date: Mon, 06 Jan 2025 09:10:32 +0100 In-Reply-To: References: Autocrypt: addr=corvink@FreeBSD.org; prefer-encrypt=mutual; keydata=mQINBGNjZaIBEADDTrDNf+0pwiuRPBdClcnZW83dH1UhuOi0u+A1J2SatEBbNaFVtXXAa vewCTuyV/ZbNidjlhq3R/pWyiKjFKvs5dj7PMCw+3z2D5OWpMdHg7TrB+fbdFPOEsu0zQVKNaO+pS KCfN0Re0m7bL3wuvl7PXvBufRwA3Guo1P4j3TXWaEkuso7VupTvE25zVGg9ONHrGOjA9RUy+Yg4Se 3NLgtUdjBgA21SBQTDvRQV4fDmVenlwvWeE0Xm8FcDcpQb6sJTihaDku78mi3Ux1HCk7rTcepVEB0 xIB6qmFxv0sLlDmVv6Z6qg1y/Q5m23Pgz60o3TulMPV4F+3Itm8ifU+wgVSzBZbD29GYkd7LKqMkF bhvfSBk+5db3vbYY5OD//+LTM5AV7e2AhXuXMvG1UNBqXqSJTTSy6KZz+qmPQO0zos0dq46p8o82l KiBEGD2Hu0p+u0OyV+MmRYo1NIBFVbOPXp2MvUVl5II0UIJ3+N9gLBmfGA+HEpVO8PnvdoT/5NQ7m 8JK1rQHzjiDub/iDPAYMqKH4C0eZ/7zO0fuY5FeRNtuNtpH1Bw/+7/5RJH7bcKkfGHHEp15FJUrGH gWNydoDLB9QBprwQc8FEldDXBjzOMXIgh6FGKLNu6DswvIPGy6M3u7DXwDakCXz+c9Ym0oFihLzZx WntrsxdswD/CwARAQABtCdDb3J2aW4gS8O2aG5lIDxjb3J2aW4ua29laG5lQGdtYWlsLmNvbT6JAl QEEwEIAD4WIQSC9FKVreba38fZT0bYVNpWMV4CagUCZArUIwIbAwUJCWYBgAULCQgHAgYVCgkICwI EFgIDAQIeAQIXgAAKCRDYVNpWMV4CalcID/44k2i/mqSSi4W6FAobSF1nFLtP/pfcRNJriWKx2UF7 cfFMKyg7Nilg7FhLb5FDB1umUW2nFfchFPTUp4FfKzgRvPzIMg4RIRcVtTpYbl3z7zs9ZXD8qS//i ibbiUG3quncm6tO2x1jLZD3ORC+8MuLGXhYQIa4O5vVF2SBHdb/U6P+wsrF+U+OpRdEdQ/4Xu9S02 kltzBGgArjcexdhUqEqW01KCCSH0+qgfN1NE+9L934ZOB+cai2b9apPbCOGuV6KcUKMj4z0RWInhl XIyMqtmhdix/P/GjrQ1REVNdp74JeweSSedM15wwc6YLMXPrtOnnExyZ1gyNFYaub+Mdo3ZQ+8386 3B0C9IlpzEW0K8bYlZrl4WBNdcMOyByETAxgQmBgP6ZFErTtaeUOH1nX2FyR6o2GLSahRgngQmnRE zScTpPbBEkMwJMDAO+rbjjGxjeKSUwD1WOfbI6QZj+MS/uBk2p08kgN7fQaCEwj6jqML/IIE+FZ9I In6TNG0hChD384VHO+YioLBno1Atgi4Q7JUWSRIHQXZW+StQajFrWPPyKQwCe1MwqpKoMpX/q8IZB lzwJgZS8ShLeFZjtzOt1jgM99TD91Neonf9OzjTSbfo7sJviSWoICMhB/MvDZjj+naMVF86uGFxnI EsjVKyBxlJd4TRHnLYPTIHFKBLQjQ29ydmluIEvDtmhuZSA8Y29ydmlua0BGcmVlQlNELm9yZz6JA lQEEwEKAD4WIQSC9FKVreba38fZT0bYVNpWMV4CagUCY2NlogIbAwUJCWYBgAULCQgHAwUVCgkICw UWAwIBAAIeAQIXgAAKCRDYVNpWMV4CavfDEACCFnXpR7H9eOgP+GJMNPtK6i9/xnqdyXi8uCZIN0h YwjN4Xzo9SMLOf4UUlQEveOB+bGqbRfHd/fGKnrlXiPd0SGpKWJC21gqL/DsIH0J8I3Whth+O8tfP WeFy0oCsvBaaGFLIrDfoIgHF9i/gqEe48xhN42weB02Z3mdR1L0d7ME/BLwS0mCXe9Zh3uHw63S6x YB3Wsjptxe/ph6TpQDUKWtRJkjC6BqXPBdThpbbfIRWmjZbp2fKEJPvtRXS14+gbUqWeJ4xCvprA3 +ae7vtrp91X775yngyW3XTw5cmDiJIjykH8+zhEIoNQXNBpFrehkQDYrcM+WoE6NGSJo+3VJvSRWh UGWDVrxdTYNkbIjmTNlkI12NINC007DiuV7OF9XHWgrbbylvuZvbODmbJRdhTFy9upAUygX1/xUAQ EIMqMiJmyTdv8i0IbZ611WElQx9XHgGeZgM6+39/laN8FwspM6gE/4NzZHIZN0LEBOerZqoF+Il6e ccQpoEWx7nb/RilJp3dUUyvkBnJWg+AJByosg857kvvmDnZ3UB+bejWpcfFvnbkiKPUBPDO7tWPb5 r2yFDpDe/Vg5x+sRbkkXGUFD6Rx0p/ZiRIneVVg8emTzhDR0IL/BZkq/uVJkdaphZli1F/31cNgf6 ZJYnjxlk86uiYSySZQR0dKLDqq7QlQ29ydmluIEvDtmhuZSA8Yy5rb2VobmVAYmVja2hvZmYuY29t PokCVAQTAQgAPhYhBIL0UpWt5trfx9lPRthU2lYxXgJqBQJkCtScAhsDBQkJZgGABQsJCAcCBhUKC QgLAgQWAgMBAh4BAheAAAoJENhU2lYxXgJqI+QQAML5PTR7KpUFV3SLG60LQJGEOHUfDmJYczxBFb IAq1U4hIbivopu1AdLty7oDDrIjCVoa2/Cy34dd99O7lhLvUmZFB/zDSUtbUg2zhDkU0YSZ11Fdrl Wzky2tFaQRgxpDvWlUP0baa3Pd4dPDRiIUI6AOSR2SL6XANk6sJh56gLVM6G8yyafGsxSyDYg6Z78 EEMFejHwB+KP2DdsahupzM+F97HeC1+bOHYxtqN+2hEkPLtQWizyumPqNg5FvZhwe7yO8V95hF3Rh uDO+9aJT+WLLvcZEb/L1bI04IvZ5FWgCLI7Levd/DuOtZI8gWapHhqGZRbXB2fuJkCoKCl6V67h/7 aWhU3LjFTsC5siJyrxPjapKcIk8a7PqZDswNCKR+24LJ5D59mPgEOnsiCCVpik1WE/kgD+rOu9dQx jpjKwuKowf4EJP1KYNkYtoy1HthzyTOqPwqXC3IUl0GVPO3xw6MhUM3irCVFruC+ecVrv85Rd37vr duT6JvgW92xjWegMsamtkDZH8Ik/cmYhH0K/qEc6OZVNea/4PTEZe0uxODJ6pbMd80AJyGqDPPVeA gWJtEIG4k6IS8XyD5v1QJtlpDVpPwP/bbFnVc3h3Oatfn8Etm1KAqYvNwyO+om2PkF1p732uapDZd LwksVmgc9s79+9pSpeP5DbIeMzhrIK Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-avojERmarfoZl3SV3vsk" User-Agent: Evolution 3.54.2 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 --=-avojERmarfoZl3SV3vsk Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Peter, unfortunately, this iGPU stuff is a bit messy. For some reason, Intel is un= able to design them in a platform independent way. That's why bhyve has to probe= for it and apply some quirks to it in it's GVT-d code. It's not expected that t= hose quirks are required for dedicated GPUs. So, please try to skip the GVT-d ch= eck you've already found. If you want to pass the option rom to the guest, you can use the rom option= of passthru devices: -s 1/2/3,passthru,1/2/3,rom=3D/path/to/rom -- Kind regards, Corvin On Sun, 2025-01-05 at 10:38 +0000, Peter Wood wrote: > Hello Mario, >=20 > Thanks for the response. Interesting, is the expectation that an dGPU ARC > would work with the gvt-d code in place? Which would reenforce I'm fighti= ng > the motherboard/BIOS. >=20 > The BIOS is infact in CSM, as you've suggested, but for reason - though t= his > isn't a -virtualization problem, though I'll explain it for any future re= ader: >=20 > My experience of the ROMED8-2T is when in pure UEFI after the FreeBSD loa= der > starts the framebuffer console there are no further updates via the AST25= 00 if > the Intel ARC is present, I suspect the framebuffer is being started on t= he > Intel ARC - but I don't have a monitor capable of checking that near the > server. >=20 > The BIOS (3.5 and 3.8 checked) only offers configuration of the preferred > graphical output if the BIOS is CSM, and even then the framebuffer only s= eems > to stay on the AST2500 if the video option rom is set to legacy mode. >=20 > For pure EFI, I haven't dug into loader yet, but what I have observed is = that > the ROMED8-2T does not output the EFI variable ConOut, which having a qui= ck > read would probably guide the loader where to push the framebuffer. I don= 't > know if it's possible for me to write that var to EFI, but may be worth > looking into. >=20 > As an aside, with CSM (video legacy option rom) I did actually patch bhyv= e to > remove the gvt-d check, and the linux guest did boot with the passthru - = I > assume with the card in legacy mode... the linux kernel was happy with it > until it tried to read the option rom, which I haven't exported yet - but= I'd > prefer to fix this the correct way if possible. >=20 > P. >=20 >=20 > On Sat, 4 Jan 2025 at 20:34, Mario Marietto wrot= e: > > Hi Peter, > > =C2=A0 > > Please make sure your GPU is booted in UEFI and not CSM mode. This can = be > > changed in the host BIOS. > > Cheers.=20 > >=20 > >=20 > > On Sat, Jan 4, 2025 at 9:24=E2=80=AFPM Peter Wood = wrote: > > > Happy new year all. > > >=20 > > > I've been using bhyve happily for a year or two now, multiple machine= s > > > with VT-d running happily passing SAS cards and USB cards into VMs. > > >=20 > > > I've reached the point where I want to pass a GPU in for accelerated > > > encoding/decoding/etc (scrypted, tdarr, jellyfin). I picked up an Int= el > > > ARC A380, as it's encoders/decoders are pretty decent for my use case= - > > > I'd also seen that there had been success with people using the iGPU'= s in > > > Intel CPUs successfully. > > >=20 > > > Unfortunately after attaching the GPUs PCI device to ppt, and attempt= ing > > > to start a VM with it attached, I'm greated by an error that seems to > > > focus on iGPUs? > > >=20 > > > /usr/sbin/bhyve -A -H -w -u -S -c 2 -m 8G -l com1,/dev/nmdm202B -l > > > bootrom,/usr/local/share/uefi- > > > firmware/BHYVE_UEFI.fd,/usr/local/var/cache/bmd/k8s-worker-3.vars -s > > > 0,hostbridge -s 1,lpc -s 2,nvme,/dev/zvol/vm/k8s-worker-3 -s 3,ahci- > > > cd,/mnt/vm/isos/ubuntu-24.04-live-server-amd64.iso -s 5,virtio-net,ta= p9 -s > > > 4,passthru,4/0/0 -s 6,fbuf,tcp=3D0.0.0.0:6007,w=3D1280,h=3D720,vga=3D= io -s > > > 7,xhci,tablet test > > > bhyve: Warning: Unable to reuse host address of Graphics Stolen Memor= y. > > > GPU passthrough might not work properly. > > > bhyve: gvt_d_setup_opregion: Invalid OpRegion signature > > > bhyve: gvt_d_init: Unable to setup OpRegion > > > Device emulation initialization error: No such file or directory > > >=20 > > > The machine is an AMD EPYC 7343 on a ASRock ROMED8-2T, it has a dedic= ated > > > onboard GPU (attached to the BMC), which the BIOS is configured to > > > encourage operating systems to use as the primary display - and sure > > > enough the loader and BSD console are presented through the BMCs ASPE= ED > > > AST2500. > > >=20 > > > Digging through the freebsd source tree, it appears that that pci_gvt= -d.c > > > is responsible for this, if it's Intel and a Display then attempt the= set > > > up of graphics memory. > > > https://github.com/freebsd/freebsd-src/blob/b662ca1d6cd82044c6cb79075= e1830b97594bef3/usr.sbin/bhyve/amd64/pci_gvt-d.c#L44 > > >=20 > > > Has anyone experimented with this? Can I just patch this out, rebuild > > > bhyve and expect a chance of success? > > >=20 > > > Cheers, > > >=20 > > > P. > > > --=20 > > > Peter Wood > > > peter@alastria.net > > >=20 > >=20 > >=20 > > --=20 > > Mario. >=20 >=20 > --=20 > Peter Wood > peter@alastria.net >=20 --=20 Kind regards, Corvin --=-avojERmarfoZl3SV3vsk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEgvRSla3m2t/H2U9G2FTaVjFeAmoFAmd7j/gACgkQ2FTaVjFe AmrFJg//aR0fFMQwh61Ogs+/d//AkS8XXsYYp6jkEJH3kxec7QvorB1rY/1oYIXn RwCbMwmk1SwHtkrWhGeUdI/G0sQy6rXV4s4ycEni6NhwTflp2ZnblGDvFAwh/yhT hS7eU7LXA5R83Hcy7/3EDFK575XaoT+to6c8+lckcbTbmvIuQfgQ5oHxh8nRZb0f 9mpi5ij7zwtcI3Skhu4NfheyXDNaHFPusD/S4O7dCf2NutURtIZDJ/ZUKm9Lt35H tLBFSlCFnkE+mwLG4FbpBJDNkKlKINaUQJXFaS6YGIFeZZ7P/PMIl7z9y4AuNEox PLCDFEF95B0sLL183tfUtVhUfv1F5eqE8A+pE1YUGMhf2BbAMYJnQ8iSBV1hH5Cu vG8NP7IaEkvUm3sltPf6rYtFmNioOaFAvU3bwPItTOjEtUy8+Oied8d51sujOg8F ZX3Zl4LZ8RUZ/Cpfq2aFlnBEAvHbhXmRO6CVreF8m0rlCwTTeRXNPOr076WCUeoL 9NQQ9bvU37EwhWTrI7S5qtk4J1OU1D17YHC+XUPSivBzffkihoB94hEgRM4rFTEk DpUGIW2gTP0dOn6GKdyNcULJvW5fU+jgiYZgqCtkULgm9dUv1hEGs5MMGfcFO2YQ AvSO7qpoFTH948dXViFI/whKX/X7UnsbHEdqC2p+XvwUMep5/e4= =IEUd -----END PGP SIGNATURE----- --=-avojERmarfoZl3SV3vsk-- From nobody Mon Jan 6 13:56:54 2025 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YRbMk5hzJz5kZy9 for ; Mon, 06 Jan 2025 13:57:06 +0000 (UTC) (envelope-from peter@alastria.net) Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YRbMj6nbPz4gpv for ; Mon, 6 Jan 2025 13:57:05 +0000 (UTC) (envelope-from peter@alastria.net) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x834.google.com with SMTP id d75a77b69052e-4678cd314b6so137877881cf.3 for ; Mon, 06 Jan 2025 05:57:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alastria.net; s=google; t=1736171825; x=1736776625; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lMpsNfZ814QUAVn5GkrGngUxZs08ZUMjI3LrUuK3GFM=; b=rGJP/a6fARF5PSklzIA+ThY+aJs3BuYlELDh5aNq35lpyvc8k1K718rGiUrz3sEOQH fixx5+0SbyqL/m4Fyj6yuyFUKyIQmAaeClcmcQ3ZMDbyfqRMwTBqP9p+L/yiQiQdsErf IK/eMyKQ8X/6fTqFSFKFWdsOAeakxn89K9wRxQp5DTzesoD9PqPzs3amklExYh6SzJoc C4aBe4NCrhGJUCZNKlGIO0Kq0MzgSQw38sgjsY3Ad2H240UFPUejKg+ES90NW/zY5Tk7 I3psdf1OC/gqOK55kTds9keVtIorjdZR8vFlWTo6fX0eF5sPuWFlg38Ur+mS2BzwQmRs Bgrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736171825; x=1736776625; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lMpsNfZ814QUAVn5GkrGngUxZs08ZUMjI3LrUuK3GFM=; b=PNV+Kwrn5Kb9RPrtLrfXlOm/KrvWxN4uPMCixdcbj2rYSTyhbDvlkY9AlxiYn5/CSa Xu+ZlLcoJsfHGBYarxwtI9+Supkyg4b77cbV8UPzrMj591J7ZfT2SeUZp/dGw36SG7OD pJJWggczJuCRgRPhwtWt6I51HY/W5ftH0X3EEsOBkA9H6CvPzXkdSkMkyZgYDm+zVz8R +wxtMssW2cKQa03qQ0fmc49qzD8/KJPjbv6oKfyy0yd/SxX9ZOq+OudSpSplWOKGvVJj HVqwvRoQFEV9h+uikRn0y2R4oG6pM3v4IdGLrK4uDchAVg74MJ8r+OmB2oGQQtKqZDRl E3Rg== X-Gm-Message-State: AOJu0Yx7k1sPEymWjdGpYGWGhtNHrhvckGzi5y2g1UAGs7DT+LwaMYB+ J6X4b5G3VJtyvaZ2kcWvu6dR5vdVmRNiN66ipFV/DrqaqJEp6FpMFlBIQWP6I50uHweaq3PyC42 PXsUo5Tm8o1CDnZVExJnF6LMf6m8LEe4h/l3zwczOBHC35/e3hsY= X-Gm-Gg: ASbGncs4khpK/hY4AQc8KcmE49HhEzImQn4pHtUBEhHl9nF8BQCJl0VI9PM83eXI/fT B/7ZuZ3/0G/gf5Yqu1L4YEs7bD+9qnUYNN+NVHuqJxdVY9ANqdlIDilQlMrOOBFB5spBPdw== X-Google-Smtp-Source: AGHT+IEkPJp5LN47HcH/oVCy36rZ30TjAcRIGSWwl8BzP1nFpGnIPUevwN4ctQgaTSRLaIPQCyICn+hHTzcVoAbSjcU= X-Received: by 2002:a05:622a:24c:b0:466:a6e4:4c1a with SMTP id d75a77b69052e-46a4a8e6c07mr772387351cf.31.1736171825214; Mon, 06 Jan 2025 05:57:05 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Peter Wood Date: Mon, 6 Jan 2025 13:56:54 +0000 Message-ID: Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? To: =?UTF-8?Q?Corvin_K=C3=B6hne?= Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000e946f0062b0a0059" X-Rspamd-Queue-Id: 4YRbMj6nbPz4gpv X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --000000000000e946f0062b0a0059 Content-Type: text/plain; charset="UTF-8" Thanks for the feedback Corvin, and thank you for the hard work you've been putting into GPU passthru. I'm reaching the end of my limited knowledge here, and I have no expectation of any further assistance - but as a status update, BIOS in CSM (with legacy video op rom, so I see the console). If you want to pass the option rom to the guest, you can use the rom option > of > passthru devices: > > -s 1/2/3,passthru,1/2/3,rom=/path/to/rom I extracted the option ROM using linux, I was able to use the /sys/devices/pci*/rom route to extract it, it seems valid at a glance (768k dump) - but no idea how to really tell. Using the patched bhyve executable to bypass gvt-d: -s 4/0/0,passthru,4/0/0,rom=/mnt/vm/intel-arc-a380.bin -s 5/0/0,passthru,5/0/0 (5/0/0 is a separate audio device exposing the audio channels of the HDMI ports). Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0) VM still fails: [ 2.508656] i915 0000:00:04.0: enabling device (0000 -> 0002) [ 2.520226] i915 0000:00:04.0: [drm] Local memory IO size: 0x000000017c800000 [ 2.520232] i915 0000:00:04.0: [drm] Local memory available: 0x000000017c800000 [ 2.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none [ 2.550829] i915 0000:00:04.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin (v2.8) [ 2.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864 [ 2.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware i915/dg2_guc_70.bin version 70.20.0 [ 2.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware i915/dg2_huc_gsc.bin version 7.10.3 [ 2.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864 [ 2.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status = 0x40000056, time = 0ms, freq = 2300MHz, ret = 0 [ 2.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status: Reset = 0, BootROM = 0x2B, UKernel = 0x00, MIA = 0x00, Auth = 0x01 [ 2.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware production part check failure [ 2.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC initialization failed -ENOEXEC [ 2.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed (-5) [ 2.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to initialize GPU, declaring it wedged! [ 2.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_set_wedged_on_init+0x34/0x50 [i915] [ 2.587048] [drm] Initialized i915 1.6.0 20230929 for 0000:00:04.0 on minor 1 Interestingly intel_gpu_top will interact with the card to a degree, it shows a render utilization (of 0%), but none of the other card capabilities. There are some very similar errors in Google which may suggest it's may not be a bhyve/passthru issue, though it could be. I need to spin up a new VM with more bleeding edge linux (or maybe even Win11) to see if it can talk to the card. https://github.com/intel-analytics/ipex-llm/issues/12122 I'll post if I get any further, but I suspect this is the end for now. Peter. -- *Peter Wood* peter@alastria.net --000000000000e946f0062b0a0059 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for the feedback Corvin, and = thank you for the hard work you've been putting into GPU passthru.

I'm reaching the end of my limited knowledge here,= and I have no expectation of any further assistance - but as a status upda= te, BIOS in CSM (with legacy video op rom, so I see the console).
=

If you wan= t to pass the option rom to the guest, you can use the rom option of
passthru devices:

-s 1/2/3,passthru,1/2/3,rom=3D/path/to/rom

= I extracted the option ROM using linux, I was able to use the /sys/devices/= pci*/rom route to extract it, it seems valid at a glance (768k dump) - but = no idea how to really tell.

Using the patched = bhyve executable to bypass gvt-d: -s 4/0/0,passthru,4/0/0,rom=3D/mnt/vm/int= el-arc-a380.bin -s 5/0/0,passthru,5/0/0
(5/0/0 is a separate audi= o device exposing the audio channels of the HDMI ports).

= Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0) V= M still fails:
[ =C2=A0 =C2=A02.508656] i915 0000:00:04.0: enabli= ng device (0000 -> 0002)
[ =C2=A0 =C2=A02.520226] i915 0000:00:04.0: = [drm] Local memory IO size: 0x000000017c800000
[ =C2=A0 =C2=A02.520232] = i915 0000:00:04.0: [drm] Local memory available: 0x000000017c800000
[ = =C2=A0 =C2=A02.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: oldd= ecodes=3Dio+mem,decodes=3Dnone:owns=3Dnone
[ =C2=A0 =C2=A02.550829] i915= 0000:00:04.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin= (v2.8)
[ =C2=A0 =C2=A02.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS = capture alloc size changed from 32768 to 36864
[ =C2=A0 =C2=A02.565855] = i915 0000:00:04.0: [drm] GT0: GuC firmware i915/dg2_guc_70.bin version 70.2= 0.0
[ =C2=A0 =C2=A02.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware = i915/dg2_huc_gsc.bin version 7.10.3
[ =C2=A0 =C2=A02.565979] i915 0000:0= 0:04.0: [drm] GT0: GUC: ADS capture alloc size changed from 32768 to 36864<= br>[ =C2=A0 =C2=A02.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed:= status =3D 0x40000056, time =3D 0ms, freq =3D 2300MHz, ret =3D 0
[ =C2= =A0 =C2=A02.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status:= Reset =3D 0, BootROM =3D 0x2B, UKernel =3D 0x00, MIA =3D 0x00, Auth =3D 0x= 01
[ =C2=A0 =C2=A02.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware = production part check failure
[ =C2=A0 =C2=A02.567077] i915 0000:00:04.0= : [drm] *ERROR* GT0: GuC initialization failed -ENOEXEC
[ =C2=A0 =C2=A02= .567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed (-5)
[= =C2=A0 =C2=A02.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to ini= tialize GPU, declaring it wedged!
[ =C2=A0 =C2=A02.570106] i915 0000:00:= 04.0: [drm:add_taint_for_CI [i915]] CI tainted:0x9 by intel_gt_set_wedged_o= n_init+0x34/0x50 [i915]
[ =C2=A0 =C2=A02.587048] [drm] Initialized i915 = 1.6.0 20230929 for 0000:00:04.0 on minor 1

Interes= tingly intel_gpu_top will interact with the card to a degree, it shows a re= nder utilization (of 0%), but none of the other card capabilities. There ar= e some very similar errors in Google which may suggest it's may not be = a bhyve/passthru issue, though it could be. I need to spin up a new VM with= more bleeding edge linux (or maybe even Win11) to see if it can talk to th= e card.


I'll post if I get any further, b= ut I suspect this is the end for now.

Peter.
--
<= /div>
--000000000000e946f0062b0a0059-- From nobody Mon Jan 6 14:40:59 2025 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YRcLP0RV6z5kdcY for ; Mon, 06 Jan 2025 14:41:01 +0000 (UTC) (envelope-from corvink@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YRcLN3Mxkz4nkv; Mon, 6 Jan 2025 14:41:00 +0000 (UTC) (envelope-from corvink@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1736174460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=FxlVfClbRmJUaBdq9k8cuoHraHryEtzToIHEDv1sJF8=; b=aYSu6z9eq1FOZMiZTG8MWviRQwCMyMoc91tjN4FBfxz3a90FYJo4E4EX6AFxyC8gN2wPXr Gajfr+sqw4+Ei4AimPXtqNowh/OySGyhj91RQ2OCHRK4pKXrHcot5d0I4OiWwLqUjk52UG W/PjqgamDdd72kSXjR22lfKjuWxVpnEFRprUb9C1plt8Jt/9+PZMJj/5t0wP+Q46DN8AId bgt8Aucye5GAzLsg2HOZ0JpHBVt0hmsxdY54HEpfO9RiMfeTNKzPiZAkGZmPQQxvPp9b4L dYOZ/504tbbrkoWXHK8M2NQvXE9lCfMDaWQcX4MGQiOr4s1BYHRmBVRymSTLGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1736174460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=FxlVfClbRmJUaBdq9k8cuoHraHryEtzToIHEDv1sJF8=; b=CcdebA2JNY/1XeNW3gVWBXoWcszFbAKAEOrpP786ulrEeMseLcbJrIxiwLQjlRdtutdXNa cDr/luObGKrIDGMvSPTq9fmldOBfwaipe7h47T6950w4NI6C2TG5MxUhkjOgaQe4mohZAx Zf3g6+PTJABAIvXvkaoxgRUlRL4TaJDvlYPIzMV7+F0uVtsOhX2gu/ae2onkm1kdZAwzlt V8GkVTVh93GvORZExsyLiPWgw3Ogbmkr7hAKZWEGWFp02mHri/aO82WkSKk9CAlIH5SU32 mL8DhQD6NbPazr5y2b7jY+DKs5hw7ft4ADbzRUijoS0YFmMZr3qssNvDcdYENg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1736174460; a=rsa-sha256; cv=none; b=ynj8M2tTMTjsLFVGliK1+uHVSxUVZRHRkQeFRvdQkGlOCoCFwmk7uxyQqywHg3gPXxfpa4 FELPvK48tTMOBVKO6eebVeVRrF82UHNeRyDuKAaEBiP9H4Z4EZPBx9+Rhx4RI6+b+wzGAh 6MIHcxNOFAWwKnj2O7XBFJIK85W7bk5EQcv/M2OymQD3l3Lydk7iKSG0aWQjbIveCcc6Tg yf7U+J5tItl5Vp/kmtUk9xRaaGzCQ9+PucOC7bS3fgV51iNmxiUa0y3zeuwwZtatlPnUpB dYyux5tPobP+mKWXlk0EBlm9QVeVCeDU5TmWG9TjzXjmCpWabHMo9elYExDZcA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [IPv6:2001:9e8:dc01:5900:fc72:1b65:c1e1:a77c] (unknown [IPv6:2001:9e8:dc01:5900:fc72:1b65:c1e1:a77c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (prime256v1) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: corvink) by smtp.freebsd.org (Postfix) with ESMTPSA id 4YRcLN0qxnzRPM; Mon, 06 Jan 2025 14:41:00 +0000 (UTC) (envelope-from corvink@FreeBSD.org) Message-ID: Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? From: Corvin =?ISO-8859-1?Q?K=F6hne?= To: Peter Wood Cc: freebsd-virtualization@freebsd.org Date: Mon, 06 Jan 2025 15:40:59 +0100 In-Reply-To: References: Autocrypt: addr=corvink@FreeBSD.org; prefer-encrypt=mutual; keydata=mQINBGNjZaIBEADDTrDNf+0pwiuRPBdClcnZW83dH1UhuOi0u+A1J2SatEBbNaFVtXXAa vewCTuyV/ZbNidjlhq3R/pWyiKjFKvs5dj7PMCw+3z2D5OWpMdHg7TrB+fbdFPOEsu0zQVKNaO+pS KCfN0Re0m7bL3wuvl7PXvBufRwA3Guo1P4j3TXWaEkuso7VupTvE25zVGg9ONHrGOjA9RUy+Yg4Se 3NLgtUdjBgA21SBQTDvRQV4fDmVenlwvWeE0Xm8FcDcpQb6sJTihaDku78mi3Ux1HCk7rTcepVEB0 xIB6qmFxv0sLlDmVv6Z6qg1y/Q5m23Pgz60o3TulMPV4F+3Itm8ifU+wgVSzBZbD29GYkd7LKqMkF bhvfSBk+5db3vbYY5OD//+LTM5AV7e2AhXuXMvG1UNBqXqSJTTSy6KZz+qmPQO0zos0dq46p8o82l KiBEGD2Hu0p+u0OyV+MmRYo1NIBFVbOPXp2MvUVl5II0UIJ3+N9gLBmfGA+HEpVO8PnvdoT/5NQ7m 8JK1rQHzjiDub/iDPAYMqKH4C0eZ/7zO0fuY5FeRNtuNtpH1Bw/+7/5RJH7bcKkfGHHEp15FJUrGH gWNydoDLB9QBprwQc8FEldDXBjzOMXIgh6FGKLNu6DswvIPGy6M3u7DXwDakCXz+c9Ym0oFihLzZx WntrsxdswD/CwARAQABtCdDb3J2aW4gS8O2aG5lIDxjb3J2aW4ua29laG5lQGdtYWlsLmNvbT6JAl QEEwEIAD4WIQSC9FKVreba38fZT0bYVNpWMV4CagUCZArUIwIbAwUJCWYBgAULCQgHAgYVCgkICwI EFgIDAQIeAQIXgAAKCRDYVNpWMV4CalcID/44k2i/mqSSi4W6FAobSF1nFLtP/pfcRNJriWKx2UF7 cfFMKyg7Nilg7FhLb5FDB1umUW2nFfchFPTUp4FfKzgRvPzIMg4RIRcVtTpYbl3z7zs9ZXD8qS//i ibbiUG3quncm6tO2x1jLZD3ORC+8MuLGXhYQIa4O5vVF2SBHdb/U6P+wsrF+U+OpRdEdQ/4Xu9S02 kltzBGgArjcexdhUqEqW01KCCSH0+qgfN1NE+9L934ZOB+cai2b9apPbCOGuV6KcUKMj4z0RWInhl XIyMqtmhdix/P/GjrQ1REVNdp74JeweSSedM15wwc6YLMXPrtOnnExyZ1gyNFYaub+Mdo3ZQ+8386 3B0C9IlpzEW0K8bYlZrl4WBNdcMOyByETAxgQmBgP6ZFErTtaeUOH1nX2FyR6o2GLSahRgngQmnRE zScTpPbBEkMwJMDAO+rbjjGxjeKSUwD1WOfbI6QZj+MS/uBk2p08kgN7fQaCEwj6jqML/IIE+FZ9I In6TNG0hChD384VHO+YioLBno1Atgi4Q7JUWSRIHQXZW+StQajFrWPPyKQwCe1MwqpKoMpX/q8IZB lzwJgZS8ShLeFZjtzOt1jgM99TD91Neonf9OzjTSbfo7sJviSWoICMhB/MvDZjj+naMVF86uGFxnI EsjVKyBxlJd4TRHnLYPTIHFKBLQjQ29ydmluIEvDtmhuZSA8Y29ydmlua0BGcmVlQlNELm9yZz6JA lQEEwEKAD4WIQSC9FKVreba38fZT0bYVNpWMV4CagUCY2NlogIbAwUJCWYBgAULCQgHAwUVCgkICw UWAwIBAAIeAQIXgAAKCRDYVNpWMV4CavfDEACCFnXpR7H9eOgP+GJMNPtK6i9/xnqdyXi8uCZIN0h YwjN4Xzo9SMLOf4UUlQEveOB+bGqbRfHd/fGKnrlXiPd0SGpKWJC21gqL/DsIH0J8I3Whth+O8tfP WeFy0oCsvBaaGFLIrDfoIgHF9i/gqEe48xhN42weB02Z3mdR1L0d7ME/BLwS0mCXe9Zh3uHw63S6x YB3Wsjptxe/ph6TpQDUKWtRJkjC6BqXPBdThpbbfIRWmjZbp2fKEJPvtRXS14+gbUqWeJ4xCvprA3 +ae7vtrp91X775yngyW3XTw5cmDiJIjykH8+zhEIoNQXNBpFrehkQDYrcM+WoE6NGSJo+3VJvSRWh UGWDVrxdTYNkbIjmTNlkI12NINC007DiuV7OF9XHWgrbbylvuZvbODmbJRdhTFy9upAUygX1/xUAQ EIMqMiJmyTdv8i0IbZ611WElQx9XHgGeZgM6+39/laN8FwspM6gE/4NzZHIZN0LEBOerZqoF+Il6e ccQpoEWx7nb/RilJp3dUUyvkBnJWg+AJByosg857kvvmDnZ3UB+bejWpcfFvnbkiKPUBPDO7tWPb5 r2yFDpDe/Vg5x+sRbkkXGUFD6Rx0p/ZiRIneVVg8emTzhDR0IL/BZkq/uVJkdaphZli1F/31cNgf6 ZJYnjxlk86uiYSySZQR0dKLDqq7QlQ29ydmluIEvDtmhuZSA8Yy5rb2VobmVAYmVja2hvZmYuY29t PokCVAQTAQgAPhYhBIL0UpWt5trfx9lPRthU2lYxXgJqBQJkCtScAhsDBQkJZgGABQsJCAcCBhUKC QgLAgQWAgMBAh4BAheAAAoJENhU2lYxXgJqI+QQAML5PTR7KpUFV3SLG60LQJGEOHUfDmJYczxBFb IAq1U4hIbivopu1AdLty7oDDrIjCVoa2/Cy34dd99O7lhLvUmZFB/zDSUtbUg2zhDkU0YSZ11Fdrl Wzky2tFaQRgxpDvWlUP0baa3Pd4dPDRiIUI6AOSR2SL6XANk6sJh56gLVM6G8yyafGsxSyDYg6Z78 EEMFejHwB+KP2DdsahupzM+F97HeC1+bOHYxtqN+2hEkPLtQWizyumPqNg5FvZhwe7yO8V95hF3Rh uDO+9aJT+WLLvcZEb/L1bI04IvZ5FWgCLI7Levd/DuOtZI8gWapHhqGZRbXB2fuJkCoKCl6V67h/7 aWhU3LjFTsC5siJyrxPjapKcIk8a7PqZDswNCKR+24LJ5D59mPgEOnsiCCVpik1WE/kgD+rOu9dQx jpjKwuKowf4EJP1KYNkYtoy1HthzyTOqPwqXC3IUl0GVPO3xw6MhUM3irCVFruC+ecVrv85Rd37vr duT6JvgW92xjWegMsamtkDZH8Ik/cmYhH0K/qEc6OZVNea/4PTEZe0uxODJ6pbMd80AJyGqDPPVeA gWJtEIG4k6IS8XyD5v1QJtlpDVpPwP/bbFnVc3h3Oatfn8Etm1KAqYvNwyO+om2PkF1p732uapDZd LwksVmgc9s79+9pSpeP5DbIeMzhrIK Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-ROMEs70aYfmzhmJO/I+c" User-Agent: Evolution 3.54.2 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 --=-ROMEs70aYfmzhmJO/I+c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 2025-01-06 at 13:56 +0000, Peter Wood wrote: > Thanks for the feedback Corvin, and thank you for the hard work you've be= en > putting into GPU passthru. >=20 > I'm reaching the end of my limited knowledge here, and I have no expectat= ion > of any further assistance - but as a status update, BIOS in CSM (with leg= acy > video op rom, so I see the console). >=20 > > If you want to pass the option rom to the guest, you can use the rom op= tion > > of > > passthru devices: > >=20 > > -s 1/2/3,passthru,1/2/3,rom=3D/path/to/rom > >=20 >=20 >=20 > I extracted the option ROM using linux, I was able to use the > /sys/devices/pci*/rom route to extract it, it seems valid at a glance (76= 8k > dump) - but no idea how to really tell. >=20 > Using the patched bhyve executable to bypass gvt-d: -s > 4/0/0,passthru,4/0/0,rom=3D/mnt/vm/intel-arc-a380.bin -s 5/0/0,passthru,5= /0/0 > (5/0/0 is a separate audio device exposing the audio channels of the HDMI > ports). >=20 > Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8.0)= VM > still fails: > [ =C2=A0 =C2=A02.508656] i915 0000:00:04.0: enabling device (0000 -> 0002= ) > [ =C2=A0 =C2=A02.520226] i915 0000:00:04.0: [drm] Local memory IO size: > 0x000000017c800000 > [ =C2=A0 =C2=A02.520232] i915 0000:00:04.0: [drm] Local memory available: > 0x000000017c800000 > [ =C2=A0 =C2=A02.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: > olddecodes=3Dio+mem,decodes=3Dnone:owns=3Dnone > [ =C2=A0 =C2=A02.550829] i915 0000:00:04.0: [drm] Finished loading DMC fi= rmware > i915/dg2_dmc_ver2_08.bin (v2.8) > [ =C2=A0 =C2=A02.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture a= lloc size > changed from 32768 to 36864 > [ =C2=A0 =C2=A02.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware i915/= dg2_guc_70.bin > version 70.20.0 > [ =C2=A0 =C2=A02.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware i915/= dg2_huc_gsc.bin > version 7.10.3 > [ =C2=A0 =C2=A02.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture a= lloc size > changed from 32768 to 36864 > [ =C2=A0 =C2=A02.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed: = status =3D > 0x40000056, time =3D 0ms, freq =3D 2300MHz, ret =3D 0 > [ =C2=A0 =C2=A02.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: = status: Reset =3D > 0, BootROM =3D 0x2B, UKernel =3D 0x00, MIA =3D 0x00, Auth =3D 0x01 > [ =C2=A0 =C2=A02.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware prod= uction part > check failure > [ =C2=A0 =C2=A02.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC initia= lization failed > -ENOEXEC > [ =C2=A0 =C2=A02.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling u= c failed (-5) > [ =C2=A0 =C2=A02.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to = initialize GPU, > declaring it wedged! > [ =C2=A0 =C2=A02.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i915]]= CI tainted:0x9 > by intel_gt_set_wedged_on_init+0x34/0x50 [i915] > [ =C2=A0 =C2=A02.587048] [drm] Initialized i915 1.6.0 20230929 for 0000:0= 0:04.0 on minor > 1 >=20 If possible, it might be a good idea to check if it's running on a Linux ho= st with QEMU properly. If yes, we may be able to check if QEMU has some specia= l quirks for those devices (don't see one yet). > Interestingly intel_gpu_top will interact with the card to a degree, it s= hows > a render utilization (of 0%), but none of the other card capabilities. Th= ere > are some very similar errors in Google which may suggest it's may not be = a > bhyve/passthru issue, though it could be. I need to spin up a new VM with= more > bleeding edge linux (or maybe even Win11) to see if it can talk to the ca= rd. >=20 > https://github.com/intel-analytics/ipex-llm/issues/12122 >=20 > I'll post if I get any further, but I suspect this is the end for now. >=20 Hmm, the issues you've posted is related to resizable BARs. I'm not familia= r with it but afaik, bhyve isn't able to emulate resizable BARs yet. Btw. resizable BARs are somehow supported by QEMU, so it might be worth giv= ing it a try: https://gitlab.com/qemu-project/qemu/-/commit/b5048a4cbfa0362abc720b5198fe9= a35441bf5fe > Peter. > --=20 > Peter Wood > peter@alastria.net >=20 --=20 Kind regards, Corvin --=-ROMEs70aYfmzhmJO/I+c Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEgvRSla3m2t/H2U9G2FTaVjFeAmoFAmd763sACgkQ2FTaVjFe Amri8w/8CBq7bGS2fMHQw3P7AEzJLV7nSslm4m6mIdMwwSWHHC7fF5kwPIBw4nUH pQbxhRyns0hlmvJul93La8mpnTCQ1EPozxS/PtK/r36d/j1zPa44JoLWAYrYupkR 8OYEQiBJvU9eE/yT/59v5ebq/s4cV547HhsbZ15ubBA3ZT9Is2kadE376WC7GgD9 xxY19pQM9B3EgW0GUC7AXLj57KCFaPnl15uJ4XssofDetyrOrbTl8TbIu/LmDSsh 7k+pmX8KII9oOsDIdLjL/RFNvNVeapz3TiDjFgSUKLnDfmE2eU5FksgORVu5JH/M fWhHemantla3bJouZ7MqBijRQcjCKeC+FzcMPBBVWH0Q91/PpjMLlyvqHxmu8xHB XT6b7pv1EQ2JfbvkVvhlb42AmvN4pjEq09zsIME445OI2iQV8t2AAMKNMbSY4jbD pr74494PR5u4I+BvNmdJRCxS/rml9oAg50y1sKftBeUf5/rUlkmLUIz5QhjxIU1d kLIPVwn++GILRlw4QXxkj+nc66sgv9f9nFoo3eQ7Wgp/I/9RMsYqspHIdQhWGYgt 9Rj92ZSxLZJk29MJpQaOXLKSwSs9jn0/asVtG0NDq5HyMK87gGjivZTO91Cj4lXw h4X/NCV3uh4GbG68uB79Ojs0fEHW66NcvhbSoV96iAbwVmjZsi4= =eZg9 -----END PGP SIGNATURE----- --=-ROMEs70aYfmzhmJO/I+c-- From nobody Mon Jan 6 15:05:08 2025 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YRctz2Gs0z5kgcm for ; Mon, 06 Jan 2025 15:05:47 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YRctz00mMz4qRb; Mon, 6 Jan 2025 15:05:47 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2ef89dbd8eeso15474838a91.0; Mon, 06 Jan 2025 07:05:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736175945; x=1736780745; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=agIpc47/nvAXs/rNmE58ZWV5Y3c6Ydu2+JuC/N5oF0k=; b=dzvJs+BrmtYpKdRDCGfNE4Ei3u89OUKr5Hq0JMciHq2iuZJPjusxnl7ypRNj0xz2JL h44pudySyIyvfunT4yY1WBaajf2LCMl/UAWOpOgUsY6LAT74s62sPfy7kYitbbIQ4ibm 8a/X1y2NSTLd/VXv4amFqjEzucIeR+vWxNxtnE/KmWKS8MDlzQ4jh4tJFcGz4boPAl7I HlqlJkhE3zk+pTZtg6jsozYzgJtqYCy4rGNJY7/9uhigUrY//Mu5iHSopGStkDz6BydU SpD33nMv4kb8vNqHtbT0qemMpUp2IEcLWRneelFV7naQaCZJuFXe8/Xbe4EFeMHpMYQG 0AQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736175945; x=1736780745; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=agIpc47/nvAXs/rNmE58ZWV5Y3c6Ydu2+JuC/N5oF0k=; b=R1E+PcdRfAP8Pce7cjpI15LQ5X2V8kavP1d4be+CLhoh5MRtiPPu0rk5cFpn7c3t6L NuSBuHEjjzrjPT2O54/zkWgccVrZKbblvpILKFo5MKU1jBpZg2izIRDYdI4eao/uR+B1 bLT+DJp836+thMCLFV9JdRvGNuWF/dzNh9dbpEwKtrNOQhsRpLqr5I/+BBzMuou5MDt8 /dPyHm3WOUc2VRF9UWiMtJeQ3xLQkLZywVODnzwmylIxAC7F58aCFvypBRS9L0rJljhR edSF6yknkIZoCmblk4tvM1uN+FdWs/F9IzTOs7V97zTz91qhy70CG/6uasrf/Yf2B7Mo S3FQ== X-Forwarded-Encrypted: i=1; AJvYcCVcuIVSjOKdMhh8yOlIjRiB+35zF2cNjqqDXRR8PQ5E8rbXE77lvT0A6GpjZTuU43sowuJVv+f+EsSXQptK0DTcMJaCaId5@freebsd.org X-Gm-Message-State: AOJu0YzCqdIr3el8b7bz45NJSkYnLdofw6xlAi4LYL1/MJvpslDuKrtd CD4Yuo+3fSXvElnjJv23MkRSpZsr7XD06aZIiRENIZqNuteW9s21Ag4fwuHa34lT8bknpL0reE6 iW5d6AWJ30J81WbEqk4/TqJUrITBuJzso X-Gm-Gg: ASbGncuk4dqQt5CqYF/JNHKk+OSjf/MOhxPLSHKDUxWJGWKZ2PCK71Q15q7Z/hBs4Ph go6BkcaIvFWp/BtZ//hZtEIY1gqF6L1Cph77HkNk= X-Google-Smtp-Source: AGHT+IENrIRdSA0dO+KMjnso1K7tbVWyPppkWiMeqGJHon89GYsLFEW5S+99LR+C/+kr1E+bWS380+dVd0mOjSBa274= X-Received: by 2002:a17:90b:2807:b0:2ee:599e:f411 with SMTP id 98e67ed59e1d1-2f452eeb5a3mr76662150a91.34.1736175944832; Mon, 06 Jan 2025 07:05:44 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Mon, 6 Jan 2025 16:05:08 +0100 Message-ID: Subject: Re: bhyve/passthru for Intel dGPU (ARC A380)? To: =?UTF-8?Q?Corvin_K=C3=B6hne?= Cc: Peter Wood , freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000759f67062b0af68e" X-Rspamd-Queue-Id: 4YRctz00mMz4qRb X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --000000000000759f67062b0af68e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I want here to reiterate the importance of creating a version of Bhyve that is able to accelerate qemu. Please can some developer "hear" me ? you know that this idea is good and fundamental for bhyve and for FreeBSD,for every user who loves this system. On Mon, Jan 6, 2025 at 3:41=E2=80=AFPM Corvin K=C3=B6hne wrote: > On Mon, 2025-01-06 at 13:56 +0000, Peter Wood wrote: > > Thanks for the feedback Corvin, and thank you for the hard work you've > been > > putting into GPU passthru. > > > > I'm reaching the end of my limited knowledge here, and I have no > expectation > > of any further assistance - but as a status update, BIOS in CSM (with > legacy > > video op rom, so I see the console). > > > > > If you want to pass the option rom to the guest, you can use the rom > option > > > of > > > passthru devices: > > > > > > -s 1/2/3,passthru,1/2/3,rom=3D/path/to/rom > > > > > > > > > I extracted the option ROM using linux, I was able to use the > > /sys/devices/pci*/rom route to extract it, it seems valid at a glance > (768k > > dump) - but no idea how to really tell. > > > > Using the patched bhyve executable to bypass gvt-d: -s > > 4/0/0,passthru,4/0/0,rom=3D/mnt/vm/intel-arc-a380.bin -s > 5/0/0,passthru,5/0/0 > > (5/0/0 is a separate audio device exposing the audio channels of the HD= MI > > ports). > > > > Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux > 6.8.0) VM > > still fails: > > [ 2.508656] i915 0000:00:04.0: enabling device (0000 -> 0002) > > [ 2.520226] i915 0000:00:04.0: [drm] Local memory IO size: > > 0x000000017c800000 > > [ 2.520232] i915 0000:00:04.0: [drm] Local memory available: > > 0x000000017c800000 > > [ 2.540148] i915 0000:00:04.0: vgaarb: VGA decodes changed: > > olddecodes=3Dio+mem,decodes=3Dnone:owns=3Dnone > > [ 2.550829] i915 0000:00:04.0: [drm] Finished loading DMC firmware > > i915/dg2_dmc_ver2_08.bin (v2.8) > > [ 2.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc siz= e > > changed from 32768 to 36864 > > [ 2.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware > i915/dg2_guc_70.bin > > version 70.20.0 > > [ 2.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware > i915/dg2_huc_gsc.bin > > version 7.10.3 > > [ 2.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS capture alloc siz= e > > changed from 32768 to 36864 > > [ 2.567001] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status = =3D > > 0x40000056, time =3D 0ms, freq =3D 2300MHz, ret =3D 0 > > [ 2.567006] i915 0000:00:04.0: [drm] GT0: GUC: load failed: status: > Reset =3D > > 0, BootROM =3D 0x2B, UKernel =3D 0x00, MIA =3D 0x00, Auth =3D 0x01 > > [ 2.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware production > part > > check failure > > [ 2.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC initialization > failed > > -ENOEXEC > > [ 2.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enabling uc failed > (-5) > > [ 2.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed to > initialize GPU, > > declaring it wedged! > > [ 2.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i915]] CI > tainted:0x9 > > by intel_gt_set_wedged_on_init+0x34/0x50 [i915] > > [ 2.587048] [drm] Initialized i915 1.6.0 20230929 for 0000:00:04.0 o= n > minor > > 1 > > > > If possible, it might be a good idea to check if it's running on a Linux > host > with QEMU properly. If yes, we may be able to check if QEMU has some > special > quirks for those devices (don't see one yet). > > > Interestingly intel_gpu_top will interact with the card to a degree, it > shows > > a render utilization (of 0%), but none of the other card capabilities. > There > > are some very similar errors in Google which may suggest it's may not b= e > a > > bhyve/passthru issue, though it could be. I need to spin up a new VM > with more > > bleeding edge linux (or maybe even Win11) to see if it can talk to the > card. > > > > https://github.com/intel-analytics/ipex-llm/issues/12122 > > > > I'll post if I get any further, but I suspect this is the end for now. > > > > Hmm, the issues you've posted is related to resizable BARs. I'm not > familiar > with it but afaik, bhyve isn't able to emulate resizable BARs yet. > > Btw. resizable BARs are somehow supported by QEMU, so it might be worth > giving > it a try: > > > https://gitlab.com/qemu-project/qemu/-/commit/b5048a4cbfa0362abc720b5198f= e9a35441bf5fe > > > Peter. > > -- > > Peter Wood > > peter@alastria.net > > > > -- > Kind regards, > Corvin > --=20 Mario. --000000000000759f67062b0af68e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I want here to reiterate the importance of creating a= version of Bhyve that is able to accelerate qemu.
Please ca= n some developer "hear" me ? you know that this idea is good and = fundamental for bhyve and for FreeBSD,for every user who loves this system.=

On Mon, Jan 6, 2025 at 3:41=E2=80=AFPM Corvin K=C3=B6hne <corvink@freebsd.org> wrote:
On = Mon, 2025-01-06 at 13:56 +0000, Peter Wood wrote:
> Thanks for the feedback Corvin, and thank you for the hard work you= 9;ve been
> putting into GPU passthru.
>
> I'm reaching the end of my limited knowledge here, and I have no e= xpectation
> of any further assistance - but as a status update, BIOS in CSM (with = legacy
> video op rom, so I see the console).
>
> > If you want to pass the option rom to the guest, you can use the = rom option
> > of
> > passthru devices:
> >
> > -s 1/2/3,passthru,1/2/3,rom=3D/path/to/rom
> >
>
>
> I extracted the option ROM using linux, I was able to use the
> /sys/devices/pci*/rom route to extract it, it seems valid at a glance = (768k
> dump) - but no idea how to really tell.
>
> Using the patched bhyve executable to bypass gvt-d: -s
> 4/0/0,passthru,4/0/0,rom=3D/mnt/vm/intel-arc-a380.bin -s 5/0/0,passthr= u,5/0/0
> (5/0/0 is a separate audio device exposing the audio channels of the H= DMI
> ports).
>
> Sadly initialization of the GPU in the linux (Ubuntu 24.04 / linux 6.8= .0) VM
> still fails:
> [ =C2=A0 =C2=A02.508656] i915 0000:00:04.0: enabling device (0000 ->= ; 0002)
> [ =C2=A0 =C2=A02.520226] i915 0000:00:04.0: [drm] Local memory IO size= :
> 0x000000017c800000
> [ =C2=A0 =C2=A02.520232] i915 0000:00:04.0: [drm] Local memory availab= le:
> 0x000000017c800000
> [ =C2=A0 =C2=A02.540148] i915 0000:00:04.0: vgaarb: VGA decodes change= d:
> olddecodes=3Dio+mem,decodes=3Dnone:owns=3Dnone
> [ =C2=A0 =C2=A02.550829] i915 0000:00:04.0: [drm] Finished loading DMC= firmware
> i915/dg2_dmc_ver2_08.bin (v2.8)
> [ =C2=A0 =C2=A02.564885] i915 0000:00:04.0: [drm] GT0: GUC: ADS captur= e alloc size
> changed from 32768 to 36864
> [ =C2=A0 =C2=A02.565855] i915 0000:00:04.0: [drm] GT0: GuC firmware i9= 15/dg2_guc_70.bin
> version 70.20.0
> [ =C2=A0 =C2=A02.565859] i915 0000:00:04.0: [drm] GT0: HuC firmware i9= 15/dg2_huc_gsc.bin
> version 7.10.3
> [ =C2=A0 =C2=A02.565979] i915 0000:00:04.0: [drm] GT0: GUC: ADS captur= e alloc size
> changed from 32768 to 36864
> [ =C2=A0 =C2=A02.567001] i915 0000:00:04.0: [drm] GT0: GUC: load faile= d: status =3D
> 0x40000056, time =3D 0ms, freq =3D 2300MHz, ret =3D 0
> [ =C2=A0 =C2=A02.567006] i915 0000:00:04.0: [drm] GT0: GUC: load faile= d: status: Reset =3D
> 0, BootROM =3D 0x2B, UKernel =3D 0x00, MIA =3D 0x00, Auth =3D 0x01
> [ =C2=A0 =C2=A02.567009] i915 0000:00:04.0: [drm] GT0: GUC: firmware p= roduction part
> check failure
> [ =C2=A0 =C2=A02.567077] i915 0000:00:04.0: [drm] *ERROR* GT0: GuC ini= tialization failed
> -ENOEXEC
> [ =C2=A0 =C2=A02.567610] i915 0000:00:04.0: [drm] *ERROR* GT0: Enablin= g uc failed (-5)
> [ =C2=A0 =C2=A02.567949] i915 0000:00:04.0: [drm] *ERROR* GT0: Failed = to initialize GPU,
> declaring it wedged!
> [ =C2=A0 =C2=A02.570106] i915 0000:00:04.0: [drm:add_taint_for_CI [i91= 5]] CI tainted:0x9
> by intel_gt_set_wedged_on_init+0x34/0x50 [i915]
> [ =C2=A0 =C2=A02.587048] [drm] Initialized i915 1.6.0 20230929 for 000= 0:00:04.0 on minor
> 1
>

If possible, it might be a good idea to check if it's running on a Linu= x host
with QEMU properly. If yes, we may be able to check if QEMU has some specia= l
quirks for those devices (don't see one yet).

> Interestingly intel_gpu_top will interact with the card to a degree, i= t shows
> a render utilization (of 0%), but none of the other card capabilities.= There
> are some very similar errors in Google which may suggest it's may = not be a
> bhyve/passthru issue, though it could be. I need to spin up a new VM w= ith more
> bleeding edge linux (or maybe even Win11) to see if it can talk to the= card.
>
> https://github.com/intel-analytics/ipex= -llm/issues/12122
>
> I'll post if I get any further, but I suspect this is the end for = now.
>

Hmm, the issues you've posted is related to resizable BARs. I'm not= familiar
with it but afaik, bhyve isn't able to emulate resizable BARs yet.

Btw. resizable BARs are somehow supported by QEMU, so it might be worth giv= ing
it a try:

https://gitlab= .com/qemu-project/qemu/-/commit/b5048a4cbfa0362abc720b5198fe9a35441bf5fe

> Peter.
> --
> Peter Wood
> peter@alastria= .net
>

--
Kind regards,
Corvin


--
Ma= rio.
--000000000000759f67062b0af68e-- From nobody Tue Jan 7 17:47:07 2025 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YSJQh4rSDz5ktj0 for ; Tue, 07 Jan 2025 17:47:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YSJQh1sVWz4fqh for ; Tue, 7 Jan 2025 17:47:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1736272028; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6WtbuwRO2DEh5nfymJA7ws7vsNTQcpkb2tD6lsJZmdI=; b=XKpmEUq70+bPsQqPeh2aqCJyd6ZY9UytFuzxgyj4dawj/zp4HTn2BsZSVyzQKa4q1bKon8 vwKtjbbQZmLBSeo6Owqz9LSlPFWp8xBh3SshHAm3QodBR7k+P9q9A3sLyTdVRqV7nIpr/7 842hjbC0qQTWKpx8A6OYnzqoGjfL3SnCT8u7YRPBlRej9cV7Jgqwj8UV4sLb4osXMMGkBj 625aabIisGrQzcjMU0ot27ICUlFJEcSD/+oX0PBECRwsqknpNrc/+T4I5O/ckTnnWQjJwX WbAK1UXtzuCENvJxiYe3olSNw98bCBt9rYr3vBGTk4DT5NSlie5fCklr4jAhZw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1736272028; a=rsa-sha256; cv=none; b=NU2HDpd+6CefX3BkMdIWKqpYCDwbRQAceIu8SEE2YxYmU4FjuhaKOe3ACucJ/SI886WJaC SWjhO4J8BxxbLX0c+gOa1jC8OlN37opk/+RV91GGkLYOsgAU1a7IBWXZo5j09QqhjR44wB NTz0YNPWiAMm0AtS2gqAL/9CZJNMrgnINOpqjur7u+lWjJVGAfJtPJ28d//vOWSA8yaXcD osMQH/WBKwG7EHXk+iZH4Lh7DmVmH1pM1uNvAzFqWSwTT7Kv86txm7wOA3uoWKfGw20VDr LVOCR4ETszeCiPXf0w8S1ZcqGdOs/1jjP5OEHkKiFO9clae6REqnE9yrBOZhng== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4YSJQh0xFMz53b for ; Tue, 07 Jan 2025 17:47:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 507Hl8CH064617 for ; Tue, 7 Jan 2025 17:47:08 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 507Hl8fe064616 for virtualization@FreeBSD.org; Tue, 7 Jan 2025 17:47:08 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 273372] SR-IOV Networking in Bhyve Causes Chelsio T520-SO-CR to Fail on Host, Kernel Panic if Reset Date: Tue, 07 Jan 2025 17:47:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-STABLE X-Bugzilla-Keywords: bhyve, crash X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: john@jnielsen.net X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D273372 John Nielsen changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |john@jnielsen.net --- Comment #11 from John Nielsen --- Kind of a shot in the dark but what temperature is your card reporting? I recently realized that when my T520 goes out to lunch it's probably running crazy hot (even by Chelsio's high (>100C) standards). sysctl dev.t5nex.0.temperature --=20 You are receiving this mail because: You are the assignee for the bug.=