From nobody Fri Jul 29 13:42:21 2022 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LvTH84GXLz4XCrh for ; Fri, 29 Jul 2022 13:43:00 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LvTH75x7jz483R for ; Fri, 29 Jul 2022 13:42:59 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: by mail-ej1-x62e.google.com with SMTP id bp15so8597036ejb.6 for ; Fri, 29 Jul 2022 06:42:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=4xXdYa2wBatBuB85q7ytegNHRvbOAkokrsPdg1gB0NU=; b=iAp2RjT+9orRiGms6mmjTfqKTydZ3smH754ZIo6+SC9F1MT1p9W8RLvKDkIqCinHlh E1y9UUOqsjKrmG0NVpX+9JuxXbhy+jfoBgdRP63gGfICSNgWHn38+kpr2A06D7Sw5kVy s73fj5Dce1k3zjC5x0ZElAvft1egeC4iFJIBEG8qhS20PuazrT7cI+ahPGDQnpF7eWf+ qUhYPomWpYx3zlnTCxhSS5qUKaM128zdFWzqNBgwIiel46spOXBltc01niR9x/qiYSwq XMfy1nHYA/OU/48fPByNCu1XB5GuJTaBRcw8drJl57o1p/twg5dwf07TR0WiPZIyoyIu cJFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=4xXdYa2wBatBuB85q7ytegNHRvbOAkokrsPdg1gB0NU=; b=mHi545jnKy0T0hGHj3eYm3WmCWLYaV2cMoOUK1AfwCTwUTQQzZO0XWwB3cofLpqTWO LAHya+crf8fwFQagIVfZUhyuomM1KyKkK08+0zti6KKjxzrj8AvtwU2bl7l1aFMhpYGk cNiln5y/9O05BEWOz+Mdvc0Z3HpmRUoDQfCRoeL/fvKEfL5R8WNKP1W2Mu+10GoGTxml wcdsOlWKNjq4vDBcsWjqqAhtwxOlXimgJqepj8xTd9kaXvMQlLlHYj/ut5Y2s9XNQWt4 sYo2u2LPTwSPcOfs9Ga9YRpjWBgz/NY50metsin/ECKcryWUPgUBQpUj3pl6bhQVM2yF oAAA== X-Gm-Message-State: AJIora92K+xtWFbjZegELwUW4zmWHQRdsh9uCv9A1eOgp+hBWo78vFBU QtrNXHf79/ROMSMUSfDswTJY1QYkX5JJKLzr5dctv7+gy4yEZQ== X-Google-Smtp-Source: AGRyM1sEbKPT70imneUyJtUzw9r/LIpYNAouzxY5BgeGdHDfqpWADkhskxrmErpcdkE5o+1axVYooa/3AAPrrDneKx0= X-Received: by 2002:a17:907:7639:b0:730:22e4:cee3 with SMTP id jy25-20020a170907763900b0073022e4cee3mr2923322ejc.434.1659102178321; Fri, 29 Jul 2022 06:42:58 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Fri, 29 Jul 2022 15:42:21 +0200 Message-ID: Subject: Re: bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. GPU passthrough might not work properly. bhyve: gvt_d_setup_opregion: Unable to get OpRegion base and length bhyve: gvt_d_init: Unable to setup OpRegion device emulation initialization error: Operation not supported by device To: Jonathan Vasquez , FreeBSD virtualization , =?UTF-8?Q?Corvin_K=C3=B6hne?= Content-Type: multipart/alternative; boundary="000000000000fc2aff05e4f1d360" X-Rspamd-Queue-Id: 4LvTH75x7jz483R X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=iAp2RjT+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2a00:1450:4864:20::62e as permitted sender) smtp.mailfrom=marietto2008@gmail.com X-Spamd-Result: default: False [2.28 / 15.00]; LONG_SUBJ(2.23)[297]; HTTP_TO_IP(1.00)[]; URI_COUNT_ODD(1.00)[1]; NEURAL_HAM_SHORT(-0.99)[-0.993]; NEURAL_SPAM_MEDIUM(0.88)[0.882]; NEURAL_HAM_LONG(-0.84)[-0.840]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62e:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; FREEMAIL_FROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N --000000000000fc2aff05e4f1d360 Content-Type: text/plain; charset="UTF-8" It's not used by the host. FreeBSD is using the geforce 1060. And on the BIOS I have set up the 1060 as default. I know that I can't pass through a gpu that I'm using. And the third gpu is the geforce rtx 2080 ti that I can actually pass. Actually I have 3 gpus : ppt4@pci0:0:2:0: class=0x038000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x3e98 subvendor=0x1458 subdevice=0xd000 vendor = 'Intel Corporation' device = 'CoffeeLake-S GT2 [UHD Graphics 630]' class = display vgapci0@pci0:1:0:0: class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1c02 subvendor=0x19d a subdevice=0x2438 vendor = 'NVIDIA Corporation' device = 'GP106 [GeForce GTX 1060 3GB]' class = display subclass = VGA hdac0@pci0:1:0:1: class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f1 subvendor=0x19d a subdevice=0x2438 vendor = 'NVIDIA Corporation' device = 'GP106 High Definition Audio Controller' class = multimedia subclass = HDA ppt0@pci0:2:0:0: class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1e04 subvendor=0x19d a subdevice=0x2503 vendor = 'NVIDIA Corporation' device = 'TU102 [GeForce RTX 2080 Ti]' class = display subclass = VGA ppt1@pci0:2:0:1: class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f7 subvendor=0x19d a subdevice=0x2503 vendor = 'NVIDIA Corporation' device = 'TU102 High Definition Audio Controller' class = multimedia subclass = HDA ppt2@pci0:2:0:2: class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad6 subvendor=0x19d a subdevice=0x2503 vendor = 'NVIDIA Corporation' device = 'TU102 USB 3.1 Host Controller' class = serial bus subclass = USB ppt3@pci0:2:0:3: class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad7 subvendor=0x19d a subdevice=0x2503 vendor = 'NVIDIA Corporation' device = 'TU102 USB Type-C UCSI Controller' class = serial bus As you can see,I'm using the ppt driver on the gpus that I'm not using. Il giorno ven 29 lug 2022 alle ore 14:58 Jonathan Vasquez ha scritto: > I thought in general we couldn't pass the GPU if it was being used by the > host? (Thus you would need to have two GPUs, one to allow the host to > display its output and one to passthrough. Unless the host is completely > headless). > > > Sent from ProtonMail mobile > > > > -------- Original Message -------- > On Jul 29, 2022, 02:38, Mario Marietto < marietto2008@gmail.com> wrote: > > > Hello. > > I've tried to passthru my CoffeeLake-S GT2 [UHD Graphics 630] on a Linux > VM but it failed. This graphic card : > > ppt4@pci0:0:2:0: class=0x038000 rev=0x02 hdr=0x00 vendor=0x8086 > device=0x3e98 subvendor=0x1458 subdevice=0xd000 > vendor = 'Intel Corporation' > device = 'CoffeeLake-S GT2 [UHD Graphics 630]' > class = display > > below there are the bhyve parameters that I've used : > > bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \ > -s 0,hostbridge \ > -s 1,nvme,/dev/$vmdisk0,bootindex=1 \ > -s 2,ahci-hd,/dev/$vmdisk2 \ > -s 3,ahci-hd,/dev/$vmdisk4 \ > -s 4,passthru,0/2/0 \ > -s 7,virtio-net,tap4 \ > -s 10,hda,play=/dev/dsp,rec=/dev/dsp \ > -s 29,fbuf,tcp=0.0.0.0:5904,w=1500,h=950 \ > -s 30,xhci,tablet \ > -s 31,lpc \ > -l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \ > vm4 < /dev/null & sleep 2 && vncviewer 0:4 > > this is the error I get : > > bhyve: Warning: Unable to reuse host address of Graphics Stolen Memory. > GPU passthrough might not work properly. > bhyve: gvt_d_setup_opregion: Unable to get OpRegion base and length > bhyve: gvt_d_init: Unable to setup OpRegion > device emulation initialization error: Operation not supported by device > > Should I apply some patches ? I know for sure that I can pass through this > kind of gpu,using the Corvin patches,but I don't know if I have found a bug > that should be fixed. Let me know. > > -- > Mario. > > -- Mario. --000000000000fc2aff05e4f1d360 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It's not used by the host. FreeBSD is using the g= eforce 1060. And on the BIOS I have set up the 1060 as default. I know that= I can't pass through a gpu that I'm using. And the third gpu is th= e geforce rtx 2080 ti that I can actually pass.

Actually I have 3 gpus :

ppt4@pci0:0:2:0: =C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0class=3D0x038000 rev=3D0x02 hdr=3D0x00 = vendor=3D0x8086 device=3D0x3e98 subvendor=3D0x1458 subdevice=3D0xd00= 0
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'Intel Corpor= ation'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'CoffeeLake-S= GT2 [UHD Graphics 630]'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D display

=
vgapci= 0@pci0:1:0:0: =C2=A0=C2=A0=C2=A0=C2=A0class=3D0x030000 rev=3D0xa1 hdr=3D0x0= 0 vendor=3D0x10de device=3D0x1c02 subvendor=3D0x19d
a subdevice= =3D0x2438
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'NVIDIA Corpo= ration'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'GP106 [GeFor= ce GTX 1060 3GB]'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D display
=C2=A0=C2=A0=C2=A0subclass =C2=A0=C2=A0=3D VGA

hdac0@pci0:1:0:1: =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0class=3D0x040300 rev=3D0xa1 hdr=3D0x00 vendor=3D0x1= 0de device=3D0x10f1 subvendor=3D0x19d
a subdevice=3D0x2438
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'NVIDIA Corpo= ration'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'GP106 High D= efinition Audio Controller'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D multimedia
=C2=A0=C2=A0=C2=A0subclass =C2=A0=C2=A0=3D HDA

ppt0@pci0:2:0:0: =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0class=3D0x030000 rev=3D0xa1 hdr=3D0x00 vendor= =3D0x10de device=3D0x1e04 subvendor=3D0x19d
a subdevice=3D0x2503
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'NVIDIA Corpo= ration'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'TU102 [GeFor= ce RTX 2080 Ti]'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D display
=C2=A0=C2=A0=C2=A0subclass =C2=A0=C2=A0=3D VGA

ppt1@pci0:2:0:1: =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0class=3D0x040300 rev=3D0xa1 hdr=3D0x00 vendor= =3D0x10de device=3D0x10f7 subvendor=3D0x19d
a subdevice=3D0x2503
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'NVIDIA Corpo= ration'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'TU102 High D= efinition Audio Controller'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D multimedia
=C2=A0=C2=A0=C2=A0subclass =C2=A0=C2=A0=3D HDA

ppt2@pci0:2:0:2: =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0class=3D0x0c0330 rev=3D0xa1 hdr=3D0x00 vendor= =3D0x10de device=3D0x1ad6 subvendor=3D0x19d
a subdevice=3D0x2503
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'NVIDIA Corpo= ration'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'TU102 USB 3.= 1 Host Controller'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D serial bus
=C2=A0=C2=A0=C2=A0subclass =C2=A0=C2=A0=3D USB

ppt3@pci0:2:0:3: =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0class=3D0x0c8000 rev=3D0xa1 hdr=3D0x00 vendor= =3D0x10de device=3D0x1ad7 subvendor=3D0x19d
a subdevice=3D0x2503
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'NVIDIA Corpo= ration'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'TU102 USB Ty= pe-C UCSI Controller'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D serial bus
As you can see,= I'm using the ppt driver on the gpus that I'm not using.

Il giorno ven 29 lug 2022 alle ore 14:58 Jonathan Vasquez <jon@xyinn.org> ha scritto:
I thought in general we couldn= 't pass the GPU if it was being used by the host? (Thus you would need = to have two GPUs, one to allow the host to display its output and one to pa= ssthrough. Unless the host is completely headless).


Sent from Pr= otonMail mobile



-------- Original Message --------
On Jul= 29, 2022, 02:38, Mario Marietto < marietto2008@gmail.com> wrote:
Hello.

<= span style=3D"color:rgb(0,0,0);background-color:rgb(255,255,255)">I've = tried to passthru my CoffeeLake-S GT2 [UHD Graphics = 630] on a Linux VM but it failed. This graphic card :

=
ppt4@pc= i0:0:2:0: =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0class=3D0x038000 rev=3D= 0x02 hdr=3D0x00 vendor=3D0x8086 device=3D0x3e98 subvendor=3D0x1458 s= ubdevice=3D0xd000
=C2=A0=C2=A0=C2=A0vendor =C2=A0=C2=A0=C2=A0=C2=A0=3D 'Intel Corpor= ation'
=C2=A0=C2=A0=C2=A0device =C2=A0=C2=A0=C2=A0=C2=A0=3D 'CoffeeLake-S= GT2 [UHD Graphics 630]'
=C2=A0=C2=A0=C2=A0class =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=3D display

be= low there are the bhyve parameters that I've used :

<= /span>
bhyve -S -c sockets=3D1,cores=3D2,threads=3D2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,nvme,/dev/$vmdisk0,bootindex=3D1 \
-s 2,ahci-hd,/dev/$vmdisk2 \
-s 3,ahci-hd,/dev/$vmdisk4 \
-s 4,passthru,0/2/0 \
-s 7,virtio-net,tap4 \
-s 10,hda,play=3D/dev/dsp,rec=3D/dev/dsp \
-s 29,fbuf,tcp=3D0.0.= 0.0:5904,w=3D1500,h=3D950 \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm4 < /dev/null & sleep 2 && vncviewer 0:4=

this is the error I get :

bhyve: Warning: Unable to reuse host address of Graphics Stolen Memor= y.
GPU passthrou= gh might not work prope= rly.
bhyve: gvt_d_setup_opregion: Unable to get OpReg= ion base and length
bhyve: gvt_d_init: Unable to setup OpRegion
device emulation initialization error: Operation not supported by devic= e

Should I apply some patches ? I know for su= re that I can pass through this kind of gpu,using the Corvin patches,but I = don't know if I have found a bug that should be fixed. Let me know.

--
Mar= io.


--
Mario.
--000000000000fc2aff05e4f1d360--