From nobody Wed Jun 16 08:19:49 2021 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4D63411DF4EF for ; Wed, 16 Jun 2021 08:20:02 +0000 (UTC) (envelope-from SRS0=yD/m=LK=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from vito-vars.onthenet.com.au (vito-vars.OntheNet.com.au [203.13.68.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G4dQn73nGz4tDR for ; Wed, 16 Jun 2021 08:20:01 +0000 (UTC) (envelope-from SRS0=yD/m=LK=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by vito-vars.onthenet.com.au (Postfix) with ESMTPS id A368C20B49AA for ; Wed, 16 Jun 2021 18:19:50 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 8B1AA209C003 for ; Wed, 16 Jun 2021 18:19:50 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 8122B2104D2A for ; Wed, 16 Jun 2021 18:19:50 +1000 (AEST) Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id yfuI5ZMXqK_8 for ; Wed, 16 Jun 2021 18:19:50 +1000 (AEST) Received: from MacBook-Air-4.local (unknown [120.29.46.207]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 529B52104D27; Wed, 16 Jun 2021 18:19:49 +1000 (AEST) Subject: Re: [PATCH 0/4] Prepare bhyve's OVMF for GPU-Passthrough To: =?UTF-8?Q?Corvin_K=c3=b6hne?= Cc: p.bruenn@beckhoff.com, freebsd-virtualization@freebsd.org References: <20210615083000.1286-1-c.koehne@beckhoff.com> From: Peter Grehan Message-ID: Date: Wed, 16 Jun 2021 18:19:49 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 In-Reply-To: <20210615083000.1286-1-c.koehne@beckhoff.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=BZdDLol2 c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=azl8yinJeVLa8tjSA6Q3hg==:17 a=IkcTkHD0fZMA:10 a=r6YtysWOX24A:10 a=wEmkmCM3V6jHzAu2OP0A:9 a=QEXdDO2ut3YA:10 X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=B/4iLNlM c=1 sm=1 tr=0 a=mJOSnoNX3k71adV6TmU0eQ==:117 a=azl8yinJeVLa8tjSA6Q3hg==:17 a=IkcTkHD0fZMA:10 a=r6YtysWOX24A:10 a=wEmkmCM3V6jHzAu2OP0A:9 a=QEXdDO2ut3YA:10 X-Rspamd-Queue-Id: 4G4dQn73nGz4tDR X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi Corvin, Thanks for posting these here. > GPU-Passthrough for bhyve requires a few patches to work properly. > These patches will allow GPU-Passthrough for bhyve. > It will work for dedicated AMD GPUs and integrated Intel GPUs. > Nvidia GPUs are untested. I have no issue with patches 3/4 and 4/4. However, 1+2/4 switch over to having EFI do the PCI scan/address assignment rather than accepting bhyve's probe. This history here is that tychon@'s initial version of bhyve/EFI had the scan when it was mandatory in OVMF. That didn't work with passthru devices, requiring the "Duet" (EFI-over-BIOS) PCI code to be used. The recent upstreaming of Bhyve/EFI was done prior to the fix allowing dynamic relocation of pass-thru BARs. With Duet not being available, the flag allowing EFI to inherit PCI BAR assignment was set (PcdPciDisableBusEnumeration, as you've found). While it's now possible to allow EFI to scan/assign PCI BARs, I'd like to avoid it if possible for 2 reasons: - assignment policy can stay in bhyve, such as whether to locate 64-bit BARs in the 32-bit region which EFI didn't (doesn't?) allow. Bugs or corner-cases can be fixed in bhyve without requiring a modification to upstream EFI. - there is no need for EFI to perform a slow can via PCI bus operations, resulting in VM-exits, where bhyve can perform all this in memory, which can result in faster boot. Your patch description states: >For Linux guests, AMD GPUs require that their PCI ROM is processed by UEFI. Is it possible to fix this in bhyve ? Can pass-thru ROMs be mapped just like mmio BARs are ? later, Peter.