From nobody Wed Jan 17 14:34:42 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TFT195n1wz57xfc for ; Wed, 17 Jan 2024 14:34:53 +0000 (UTC) (envelope-from michael@sloged.al) Received: from exchange.dots.no (vmi1067082.contaboserver.net [109.123.240.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TFT182t8tz44cC; Wed, 17 Jan 2024 14:34:49 +0000 (UTC) (envelope-from michael@sloged.al) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of michael@sloged.al designates 109.123.240.255 as permitted sender) smtp.mailfrom=michael@sloged.al Received: from exchange.dots.no (localhost6.localdomain6 [IPv6:::1]) by exchange.dots.no (Postfix) with ESMTP id EC9CB7062B93; Wed, 17 Jan 2024 15:34:44 +0100 (CET) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 From: "=?utf-8?B?TWljaGFlbCBTbMO4Z2VkYWw=?=" To: "=?utf-8?B?Q29ydmluIEvDtmhuZQ==?=" , "freebsd-virtualization@FreeBSD.org" Importance: Normal Sensitivity: Normal Date: Wed, 17 Jan 2024 15:34:42 +0100 Subject: RE: bhyve intel 530 integrated graphics error Thread-Topic: bhyve intel 530 integrated graphics error Thread-Index: ARVJ2gGG6AMMiOd950GC0AkYWiWDj/////9Rif6gHtCAAFATUA== Message-ID: References: In-Reply-To: Content-Language: en-us X-MS-Exchange-Organization-SCL: -1 Content-ID: <1b05ef391b029340bb0bdf44f74a5b9a@sloged.al> X-Mailer: gromox-emsmdb 2.9.69.fd54100 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_ABC2190A.687D538" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.12 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.97)[-0.971]; NEURAL_HAM_SHORT(-0.95)[-0.954]; R_SPF_ALLOW(-0.20)[+ip4:109.123.240.255]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_EQ_ADDR_SOME(0.00)[]; ASN(0.00)[asn:51167, ipnet:109.123.240.0/20, country:DE]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_ONE(0.00)[1]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-virtualization@FreeBSD.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[sloged.al]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4TFT182t8tz44cC This is a multi-part message in MIME format. ------=_NextPart_000_0005_ABC2190A.687D538 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The mmap_memseg=C2=A0 error was resolved by reducing vm memory to 4G. (No= t a viable=0Asolution, but at least it runs)=0A=0A =0A=0ABest regards,=0A= =0A =0A=0AMichael=0A=0A =0A=0AFrom: owner-freebsd-virtualization@freebsd.= org=0A On Behalf Of Michael Sl=C3= =B8gedal=0ASent: Wednesday, January 17, 2024 1:30 PM=0ATo: Corvin K=C3=B6= hne ; freebsd-virtualization@FreeBSD.org=0ASubject: = RE: bhyve intel 530 integrated graphics error=0A=0A =0A=0AThanks Corvin,=0A= =0A =0A=0AHowever it=E2=80=99s still not clear to me from D26209 what exa= ctly I need to do, and=0Awhat I=E2=80=99ve tried doesn=E2=80=99t work =F0= =9F=98=8A Any additional advice is appreciated.=0A=0A =0A=0A 1. Dropped t= he GOP rom for now. Unless I need it for windows loading screen /=0A w= indows diagnostics/repair ? GpuZ won=E2=80=99t extract it anyhow.=0A 2. I= =E2=80=99ve added the -A option, no visible change=0A 3. BHYVE_CODE.fd =E2= =80=93 not sure if I=E2=80=99m using it right. I changed the symlink for=0A= BHYVE_UEFI.fd to point to this file. No visible change =E2=80=93 exce= pt I now have=0A 4 non-working COM ports.=0A 4. I tried passthru on us= b, and get the same error that I did with the audio=0A card (see below= )=0A 5. The gfx card initially boots with windows Error 43. When I delete= it and=0A scan for hardware changes it pops back in, without the erro= r, but no=0A displays detected.=0A 6. I=E2=80=99m using vga/dvi ports = without success, haven=E2=80=99t tried displayport. No hdmi=0A ports. = In case that matters.=0A 7. Should I try to update freebsd to 14-STABLE b= ranch and see if that makes a=0A difference? (If I can figure out how)= =0A=0A =0A=0A# tail /vm/win10ltsc/bhyve.log=0A=0Apci_fbuf: mmap_memseg fa= iled=0A=0AAssertion failed: (mr->name =3D=3D memp->name), function unregi= ster_mem, file /usr/=0Asrc/usr.sbin/bhyve/mem.c, line 344.=0A=0A =0A=0A# = tail /vm/win10ltsc/vm-bhyve.log=0A=0AJan 17 13:08:48: [primary disk: dis= k0.img]=0A=0AJan 17 13:08:48: [primary disk dev: file]=0A=0AJan 17 13:08= :48: dynamically allocated port 5900 for vnc connections=0A=0AJan 17 13:0= 8:48: booting=0A=0AJan 17 13:08:48: [bhyve options: -c 6,sockets=3D1,cor= es=3D3,threads=3D2 -m 8G -Hwl=0Abootrom,/usr/local/share/uefi-firmware/BH= YVE_UEFI.fd -A -U=0A8dff5c4a-b083-11ee-b2cd-78d004259091 -S]=0A=0AJan 17 = 13:08:48: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,ahci,hd:/vm/=0A= win10ltsc/disk0.img -s 31:6,passthru,0/31/6 -s 2:0,passthru,0/2/0 -s=0A20= :0,passthru,0/20/0 -s 6:0,fbuf,tcp=3D0.0.0.0:5900,w=3D1920,h=3D1080 -s=0A= 7:0,xhci,tablet]=0A=0AJan 17 13:08:48: [bhyve console: -l com1,/dev/nmdm= -win10ltsc.1A]=0A=0AJan 17 13:08:48: starting bhyve (run 1)=0A=0AJan 17 1= 3:08:50: bhyve exited with status 134=0A=0AJan 17 13:08:50: stopped=0A=0A= =0A=0A# pciconf -lv |grep -EC 2 "(USB 3|Audio)"=0A=0Appt1@pci0:0:20:0: = class=3D0x0c0330 rev=3D0x31 hdr=3D0x00 vendor=3D0x8086 device=3D=0A0= xa12f subvendor=3D0x8086 subdevice=3D0xa12f=0A=0A vendor =3D 'Inte= l Corporation'=0A=0A device =3D '100 Series/C230 Series Chipset Fa= mily USB 3.0 xHCI=0AController'=0A=0A class =3D serial bus=0A=0A = subclass =3D USB=0A=0A--=0A=0Appt2@pci0:0:31:3: class=3D0x0403= 00 rev=3D0x31 hdr=3D0x00 vendor=3D0x8086 device=3D=0A0xa170 subvendor=3D0= x8086 subdevice=3D0xa170=0A=0A vendor =3D 'Intel Corporation'=0A=0A= device =3D '100 Series/C230 Series Chipset Family HD Audio Contro= ller'=0A=0A class =3D multimedia=0A=0A subclass =3D HDA=0A=0A= =0A=0A =0A=0AFrom: Corvin K=C3=B6hne =0ASent: Wedne= sday, January 17, 2024 8:19 AM=0ATo: Michael Sl=C3=B8gedal ; freebsd-virtualization@FreeBSD.org=0ASubject: Re: bhyve intel 530 = integrated graphics error=0A=0A =0A=0AHi,=0A=0A =0A=0Aunfortunately, GPU = passthrough is supported by bhyve but not OVMF. So,=0A=0Ayou have to use = a patched OVMF. The required patch stack is available=0A=0Aat https://rev= iews.freebsd.org/D41714. An older version and most=0A=0Aprobably still wo= rking prebuilt version is available at=0A=0Ahttps://reviews.freebsd.org/D= 26209.=0A=0A =0A=0AOn Wed, 2024-01-17 at 01:18 +0100, Michael Sl=C3=B8ged= al wrote:=0A=0A> Hi,=0A=0A> =0A=0A> I=E2=80=99ve been trying to collect = the necessary info online about how to do=0A=0A> gpu passthru with integr= ated intel graphics to win10, but can=E2=80=99t make=0A=0A> it work.=0A=0A= > =0A=0A> 1. First tried with just passthru, and after 3-4 reboots, t= he=0A=0A> display adapter shows up in win10 guest, but it won=E2=80=99t l= oad properly.=0A=0A =0A=0ADespite some configuration requirements (e.g. a= lways use slot 2) that's=0A=0Aall you need.=0A=0A =0A=0A> 2. Tried add= ing the hd audio device as well, get bluescreen with=0A=0A> =E2=80=9Cmemo= ry_management=E2=80=9D stop code in guest=0A=0A =0A=0AShouldn't happen bu= t seems like another issue.=0A=0A =0A=0A> 3. Tried adding a gop rom fr= om=0A=0A> https://github.com/patmagauran/i915ovmfPkg - host crashed and g= ot=0A=0A> error below=0A=0A =0A=0AA GOP is only required for pre OS graph= ics (e.g. grub menu).=0A=0A =0A=0AI highly recommend using the GOP from y= our host system to avoid damages=0A=0Aor strange issues due to incompatib= ilities. Unfortunately, it's not=0A=0Aalways possible to easily extract t= he GOP from your host system. Even=0A=0AIntel recommends to ask your boar= d manufacturer:=0A=0Ahttps://projectacrn.github.io/3.2/tutorials/gpu-pass= thru.html#enable-the-gvt-d-gop-driver=0A=0A =0A=0ASome alternative ways t= o extract the GOP:=0A=0A =0A=0AOn Windows try GPU-Z:=0A=0Ahttps://nvidia.= custhelp.com/app/answers/detail/a_id/4188/~/extracting-the-geforce-video-= bios-rom-file=0A=0AOn Linux try sysfs:=0A=0Ahttps://pve.proxmox.com/wiki/= PCI_Passthrough#The_.27romfile.27_option=0A=0A =0A=0A> =0A=0A> I=E2=80=99= m at a loss what to try next. Any ideas where I=E2=80=99m going wrong?=0A= =0A> =0A=0A> # tail /vm/win10ltsc/vm-bhyve.log =0A=0A> Jan 17 00:4= 9:02: [primary disk: disk0.img]=0A=0A> Jan 17 00:49:02: [primary disk d= ev: file]=0A=0A> Jan 17 00:49:02: dynamically allocated port 5900 for vnc= connections=0A=0A> Jan 17 00:49:02: booting=0A=0A> Jan 17 00:49:02: [bh= yve options: -c 6,sockets=3D1,cores=3D3,threads=3D2 -m=0A=0A> 8G -Hwl boo= trom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s=0A=0A> 2:0,passthru,= 0/2/0,rom=3D/vm/win10ltsc/i915ovmf.rom -U 8dff5c4a-b083-=0A=0A> 11ee-b2cd= -78d004259091 -S]=0A=0A> Jan 17 00:49:02: [bhyve devices: -s 0,hostbridg= e -s 31,lpc -s=0A=0A> 4:0,ahci,hd:/vm/win10ltsc/disk0.img -s 31:6,passthr= u,0/31/6 -s=0A=0A> 6:0,fbuf,tcp=3D0.0.0.0:5900,w=3D1920,h=3D1080 -s 7:0,x= hci,tablet]=0A=0A> Jan 17 00:49:02: [bhyve console: -l com1,/dev/nmdm-wi= n10ltsc.1A]=0A=0A =0A=0AYou have to use the `-A` flag. There's an open PR= for vm-bhyve:=0A=0Ahttps://github.com/churchers/vm-bhyve/pull/525/commit= s/653d01f5b53c0bd8ce707137f67617536de8b1c0=0A=0A =0A=0A> Jan 17 00:49:02:= starting bhyve (run 1)=0A=0A> Jan 17 00:49:04: bhyve exited with status = 134=0A=0A> Jan 17 00:49:04: stopped=0A=0A> =0A=0A> # tail /vm/win10ltsc/= bhyve.log =0A=0A> pci_fbuf: mmap_memseg failed=0A=0A> Assertion failed: = (mr->name =3D=3D memp->name), function unregister_mem,=0A=0A> file /usr/s= rc/usr.sbin/bhyve/mem.c, line 344.=0A=0A> =0A=0A> # uname -a=0A=0A> Free= BSD yellowfellow 14.0-RELEASE-p3 FreeBSD 14.0-RELEASE-p3 #0: Mon=0A=0A> D= ec 11 04:56:01 UTC 2023 =0A=0A> root@amd64-builder.daemonology.net:/us= r/obj/usr/src/amd64.amd64/sys/G=0A=0A> ENERIC amd64=0A=0A =0A=0A--=0A=0AK= ind regards,=0A=0ACorvin=0A=0A ------=_NextPart_000_0005_ABC2190A.687D538 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8"

The mmap_memseg=C2=A0 error was resolved by re= ducing vm memory to 4G. (Not a viable solution, but at least it runs)

 

Best regards,

 

Michael

 =

From: owner-freebsd= -virtualization@freebsd.org <owner-freebsd-virtualization@freebsd.org&= gt; On Behalf Of Michael Sl=C3=B8gedal
Sent: Wednesday, = January 17, 2024 1:30 PM
To: Corvin K=C3=B6hne <corvink@Free= BSD.org>; freebsd-virtualization@FreeBSD.org
Subject: RE: bh= yve intel 530 integrated graphics error

 

Thanks Corvin,

 

However it=E2=80=99s still not clear to me from D26209 what exactly I n= eed to do, and what I=E2=80=99ve tried doesn=E2=80=99t work 😊 Any additional= advice is appreciated.

 

  1. Dropped the = GOP rom for now. Unless I need it for windows loading screen / windows di= agnostics/repair ? GpuZ won=E2=80=99t extract it anyhow.
    2. <= li class=3DMsoListParagraph style=3D'margin-left:0in;mso-list:l1 level1 l= fo3'>I=E2=80=99ve added the -A option, no visible change<= li class=3DMsoListParagraph style=3D'margin-left:0in;mso-list:l1 level1 l= fo3'>BHYVE_CODE.fd =E2=80=93 not sure if I=E2=80=99m using it right. I ch= anged the symlink for BHYVE_UEFI.fd to point to this file. No visible cha= nge =E2=80=93 except I now have 4 non-working COM ports.<= li class=3DMsoListParagraph style=3D'margin-left:0in;mso-list:l1 level1 l= fo3'>I tried passthru on usb, and get the same error that I did with the = audio card (see below)
  2. The gfx card initially boots wi= th windows Error 43. When I delete it and scan for hardware changes it po= ps back in, without the error, but no displays detected.
    3. <= li class=3DMsoListParagraph style=3D'margin-left:0in;mso-list:l1 level1 l= fo3'>I=E2=80=99m using vga/dvi ports without success, haven=E2=80=99t tri= ed displayport. No hdmi ports. In case that matters.
  3. Should I try to update freebsd to 14-STABLE branch and see if that makes= a difference? (If I can figure out how)

 

# tail /vm/win10ltsc/b= hyve.log

pci_fbuf: mmap_memseg failed=

Assertion failed: (mr->name =3D=3D= memp->name), function unregister_mem, file /usr/src/usr.sbin/bhyve/me= m.c, line 344.

 

# tail /vm/win10ltsc/vm-bhyve.log

Jan 17 13:08:48:  [primary disk: disk0.img]

Jan 17 13:08:48:  [primary disk dev: file]=

Jan 17 13:08:48: dynamically allocate= d port 5900 for vnc connections

Jan 17= 13:08:48: booting

Jan 17 13:08:48:&nb= sp; [bhyve options: -c 6,sockets=3D1,cores=3D3,threads=3D2 -m 8G -Hwl boo= trom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -A -U 8dff5c4a-b083-11e= e-b2cd-78d004259091 -S]

Jan 17 13:08:4= 8:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,ahci,hd:/vm/win= 10ltsc/disk0.img -s 31:6,passthru,0/31/6 -s 2:0,passthru,0/2/0 -s 20:0,pa= ssthru,0/20/0 -s 6:0,fbuf,tcp=3D0.0.0.0:5900,w=3D1920,h=3D1080 -s 7:0,xhc= i,tablet]

Jan 17 13:08:48:  [bhyv= e console: -l com1,/dev/nmdm-win10ltsc.1A]

Jan 17 13:08:48: starting bhyve (run 1)

Jan 17 13:08:50: bhyve exited with status 134

Jan 17 13:08:50: stopped

 

# pciconf -lv |grep -EC 2 &qu= ot;(USB 3|Audio)"

ppt1@pci0:0:20:= 0:       class=3D0x0c0330 rev=3D0x31 hdr=3D= 0x00 vendor=3D0x8086 device=3D0xa12f subvendor=3D0x8086 subdevice=3D0xa12= f

    vendor  = ;   =3D 'Intel Corporation'

=     device     =3D '100 Series/C230 Se= ries Chipset Family USB 3.0 xHCI Controller'

    class      =3D serial = bus

    subclass &= nbsp; =3D USB

--

ppt2@pci0:0:31:3:       class= =3D0x040300 rev=3D0x31 hdr=3D0x00 vendor=3D0x8086 device=3D0xa170 subvend= or=3D0x8086 subdevice=3D0xa170

 &= nbsp;  vendor     =3D 'Intel Corporation'

    device   &= nbsp; =3D '100 Series/C230 Series Chipset Family HD Audio Controller'

    class   = ;   =3D multimedia

 &nb= sp;  subclass   =3D HDA

 

 

From: Corvin K=C3=B6hne <corvink@FreeBSD.org>
Sent:= Wednesday, January 17, 2024 8:19 AM
To: Michael Sl=C3=B8gedal = <michael@sloged.al>; freebsd-virtualization@F= reeBSD.org
Subject: Re: bhyve intel 530 integrated graphics= error

 

Hi,
 
unfortunately=
, GPU passthrough is supported by bhyve but not OVMF. So,
you have to use a patched OVMF. The required patch stack is availab=
le
at https://reviews.freebsd.org/D41714. An older version and most
probably still working prebuilt version is available at
https:/=
/reviews.freebsd.org/D26209.
 
On Wed, 2024-01-17 at 01:18 +0100, Michael Sl=C3=B8gedal wrote:<=
o:p>
> Hi,
>  =

> I=E2=80=99ve been trying to collect the necessary info on=
line about how to do
> gpu passthru with integrat=
ed intel graphics to win10, but can=E2=80=99t make
&=
gt; it work.
>  
> &=
nbsp;  1. First tried with just passthru, and after 3-4 reboots=
, the
> display adapter shows up in win10 guest, =
but it won=E2=80=99t load properly.
 
Despite some configuration requirements (e.g. always use slot=
 2) that's
all you need.
&=
nbsp;
>    2. Tried adding the hd audio=
 device as well, get bluescreen with
> =E2=80=9Cm=
emory_management=E2=80=9D stop code in guest
&n=
bsp;
Shouldn't happen but seems like another issue.<=
/o:p>
 
>    3. Tri=
ed adding a gop rom from
> https://github.com/patmagauran/i915ovmfP=
kg - host crashed and got
> error below<=
/o:p>
 
A GOP is only required for pr=
e OS graphics (e.g. grub menu).
 
I highly recommend using the GOP from your host system to avoid d=
amages
or strange issues due to incompatibilities. U=
nfortunately, it's not
always possible to easily ext=
ract the GOP from your host system. Even
Intel recom=
mends to ask your board manufacturer:
https://projectacrn.github.io/3.2/tutorials/gpu-passthru.h=
tml#enable-the-gvt-d-gop-driver
 
Some alternative ways to extract the GOP:
 
On Windows try GPU-Z:
=
https://nvidia.custhelp.com/app/=
answers/detail/a_id/4188/~/extracting-the-geforce-video-bios-rom-file=

On Linux try sysfs:
h=
ttps://pve.proxmox.com/wiki/PCI_Passthrough#The_.27romfile.27_option<=
o:p>
 
>  
> I=E2=80=99m at a loss what to try next. Any ideas where I=E2=
=80=99m going wrong?
>  
> # tail /vm/win10ltsc/vm-bhyve.log     &nb=
sp;  
> Jan 17 00:49:02:  [primary disk=
: disk0.img]
> Jan 17 00:49:02:  [primary di=
sk dev: file]
> Jan 17 00:49:02: dynamically allo=
cated port 5900 for vnc connections
> Jan 17 00:4=
9:02: booting
> Jan 17 00:49:02:  [bhyve opt=
ions: -c 6,sockets=3D1,cores=3D3,threads=3D2 -m
>=
 8G -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s
> 2:0,passthru,0/2/0,rom=3D/vm/win10ltsc/i915ovmf.rom -U =
8dff5c4a-b083-
> 11ee-b2cd-78d004259091 -S]<=
/o:p>
> Jan 17 00:49:02:  [bhyve devices: -s 0,hostbrid=
ge -s 31,lpc -s
> 4:0,ahci,hd:/vm/win10ltsc/disk0=
=2Eimg -s 31:6,passthru,0/31/6 -s
> 6:0,fbuf,tcp=3D=
0.0.0.0:5900,w=3D1920,h=3D1080 -s 7:0,xhci,tablet]
&=
gt; Jan 17 00:49:02:  [bhyve console: -l com1,/dev/nmdm-win10ltsc.1A=
]
 
You have to use the `-=
A` flag. There's an open PR for vm-bhyve:
https://github.com/churchers/vm-bhyve/pull/525/c=
ommits/653d01f5b53c0bd8ce707137f67617536de8b1c0
=
 
> Jan 17 00:49:02: starting bhyve (run 1)<=
o:p>
> Jan 17 00:49:04: bhyve exited with status 134
> Jan 17 00:49:04: stopped
&g=
t;  
> # tail /vm/win10ltsc/bhyve.log &=
nbsp; 
> pci_fbuf: mmap_memseg failed<=
/pre>
> Assertion failed: (mr->name =3D=3D memp->name), func=
tion unregister_mem,
> file /usr/src/usr.sbin/bhy=
ve/mem.c, line 344.
>  
> # uname -a
> FreeBSD yellowfellow 14.0-RELE=
ASE-p3 FreeBSD 14.0-RELEASE-p3 #0: Mon
> Dec 11 0=
4:56:01 UTC 2023    
> root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64=
/sys/G
> ENERIC amd64
 
-- 
Kind regards,
Corvin

------=_NextPart_000_0005_ABC2190A.687D538--