Date: Mon, 22 Sep 2025 08:48:17 +0200 From: Tomek CEDRO <tomek@cedro.info> To: Jonathan Vasquez <jon@xyinn.org> Cc: virtualization@freebsd.org, Nuno Teixeira <eduardo@freebsd.org>, =?UTF-8?Q?Corvin_K=C3=B6hne?= <corvink@freebsd.org>, Desktop Mailing List <freebsd-desktop@freebsd.org> Subject: Re: GPU Passthrough on FreeBSD 14.3 (AMD Radeon RX 6900 XT and Windows 10 Pro) Message-ID: <CAFYkXjmujHH4pWH%2BgEt%2BV9ch4Yu16M1%2BxMPcGeX7yw79xkBwiA@mail.gmail.com> In-Reply-To: <bGbmqb3-uNLrYxXAKocEtYchg6CbArf7eyc7zEMupPC03LPxirdlSwADHniBK6LFO-38h8y-MQb1YBB8VCXTk3GYNq4hSdgN751LRt0DkTk=@xyinn.org> References: <6CV-OY6BcErrWRit9jSpi6fWsYBG3E_Z3u6eTLPcz6foPAZV1gQpZYaZTR7JA_1ot5RQVqrWQaLxJFySXjspIhSbBJGxmckcDQyzxhALNus=@xyinn.org> <bnYK-9ZTxo-Xo7yaZPfiesz2zJHt6HVKDRN9pZHOSZt8bYRIAVok5rHT6fp7mFyXLJi1hRsvFpwN3dwTLw_skWH6nPXiVsEIWz4OOT16zms=@xyinn.org> <Ags2LfZGkLKZGcRQuyphkcg9wsf5q-HB1cIcphC2Eujdm0EtRkTJdHq-UHj6QLLsT53dLsfLeOib25LpGSIA-4IIqCBMMlXcfNsfCcobRJs=@xyinn.org> <MVEz5qPYM1pdpKQkFxy-FMucy9_MNUG5kn7cQGs0Czl6bU68vITWeEomJxHNqnK-qBrxo8qsdNYVD-B41qnVVz4D_QZP9E4QB75bkG5bu_s=@xyinn.org> <CAFDf7ULvsf0W8iGSYu-PWGWPgzrOC%2BoshOH1c2tDqLtkgbcw1w@mail.gmail.com> <mBnRBonXMqOnoy_P1WG4OnI3FcnsCHxwwKWGLG_6sfbycSh9_YMoSOlJbNjLqks0x8uBJtkYyrh9oqILGmdLyJPWaOLkJ3wuWC1SsoADChg=@xyinn.org> <z3tMiVD-k0F9eW7GbnDN0uNBCY9koZVE4p2kxuU2uO8_hGozxe8iyIEzI-x9DdoKiaWf1JniixpTJl2UCtw1SZc928-sue3MYxWGMjCt8HU=@xyinn.org> <ypPPSBUj6OAAe8IIfOoII9r-_91q__7wv4k6kHmvcnGaCouHMkgpfOK5IMFWsrMM3pEEbgqQM4xQz4VhMFhO_W00PjqZPSQQY9iK4wer0Hs=@xyinn.org> <fMxzypJTfezIkpHl_uCPeeoTyEmoIxTX1q2MbuBbSAG5BO0eWwbrLf4gLsgu1Sfo97ruSmhKnlNayeMTpqa0IpTXFNAf9GQswdM31hzIbog=@xyinn.org> <bGbmqb3-uNLrYxXAKocEtYchg6CbArf7eyc7zEMupPC03LPxirdlSwADHniBK6LFO-38h8y-MQb1YBB8VCXTk3GYNq4hSdgN751LRt0DkTk=@xyinn.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000fd962e063f5e34ad Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable what i noticed that the new pc have very complex bios setup options that may impact stability of freebsd. i encountered this recently after hw upgrade. pci-e gen 4 or 5, x16 or x8 mode, and split controller between single x16 for gpu or x8 for gpu and x8/x4 between other peripherals such as nvme controller usb additional card slots etc. in addition to that there are power modes and runtime frequency changes for cpu and all peripherals. not all new features are supported by freebsd. it took me some while to adjust optimal settings to get machine stable (at the cost of power savings). also there were irritating glitches in audio that are now considerably less noticable. long story short it is cpu, ram, and peripherals settings that may impact overall stability of the freebsd box. this work seems to be not only important for virtualization but also FreeBSD's LDWG project (Laptop Desktop Work Group) i am attaching freebsd-desktop@ mailing list :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info On Mon, Sep 22, 2025, 02:20 Jonathan Vasquez <jon@xyinn.org> wrote: > Hey all, > > Just wanted to report that I still haven't received one crash after my > adjustments yesterday. So far over 20+ hours of uptime~. I've also reboot= ed > the VM/host a few times just to ensure that it is deterministically stabl= e > and so far zero issues. So this is pointing in a positive direction. I'm > tempted to mark this as a stable set up but I'll need to play for longer = to > make that call. > > Jonathan Vasquez > PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279 > Sent with ProtonMail Secure Email > > > On Sunday, September 21st, 2025 at 08:23, Jonathan Vasquez <jon@xyinn.org= > > wrote: > > System has been up for about 9h40m. Most of it idle and no crashes yet. > I'm gonna reboot the host and see if the stability still continues. > > Jonathan Vasquez > PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279 > Sent with ProtonMail Secure Email > > > > Sent from Proton Mail <https://proton.me/mail/home>; for Android. > > > -------- Original Message -------- > On Saturday, 09/20/25 at 23:45 Jonathan Vasquez <jon@xyinn.org> wrote: > > Just finished beating the main story line in Cyberpunk 2077 in the VM. So > far no crashes and it's been running for an hour. So this is a positive > signal... we'll see what happens. I'll leave the VM running idle over nig= ht > and see if it's still running in the morning. Stay safe all! > > Jonathan Vasquez > PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279 > Sent with ProtonMail Secure Email > > > On Saturday, September 20th, 2025 at 22:28, Jonathan Vasquez < > jon@xyinn.org> wrote: > > I did a bunch of more experiments today and I was able to switch from > using the USB controller connected to the multi-functional bus that has t= he > integrated card, and switched to a separate USB controller that only has > USB ports. I'm hoping that this increases VM stability and that there may > have been some wires being crossed. This was the previous bus I was using > (I couldn't isolate just the USB functions, it had to be all or nothing f= or > it to function): > > vgapci0@pci0:18:0:0: class=3D0x030000 rev=3D0xc1 hdr=3D0x00 vendor=3D0x10= 02 > device=3D0x164e subvendor=3D0x1043 subdevice=3D0x8877 > vendor =3D 'Advanced Micro Devices, Inc. [AMD/ATI]' > device =3D 'Raphael' > class =3D display > subclass =3D VGA > hdac0@pci0:18:0:1: class=3D0x040300 rev=3D0x00 hdr=3D0x00 vendor=3D0x1002 > device=3D0x1640 subvendor=3D0x1043 subdevice=3D0x8877 > vendor =3D 'Advanced Micro Devices, Inc. [AMD/ATI]' > device =3D 'Rembrandt Radeon High Definition Audio Controller' > class =3D multimedia > subclass =3D HDA > none1@pci0:18:0:2: class=3D0x108000 rev=3D0x00 hdr=3D0x00 vendor=3D0x1022 > device=3D0x1649 subvendor=3D0x1043 subdevice=3D0x8877 > vendor =3D 'Advanced Micro Devices, Inc. [AMD]' > device =3D 'Family 19h PSP/CCP' > class =3D encrypt/decrypt > xhci1@pci0:18:0:3: class=3D0x0c0330 rev=3D0x00 hdr=3D0x00 vendor=3D0x1022 > device=3D0x15b6 subvendor=3D0x1043 subdevice=3D0x8877 > vendor =3D 'Advanced Micro Devices, Inc. [AMD]' > device =3D 'Raphael/Granite Ridge USB 3.1 xHCI' > class =3D serial bus > subclass =3D USB > xhci2@pci0:18:0:4: class=3D0x0c0330 rev=3D0x00 hdr=3D0x00 vendor=3D0x1022 > device=3D0x15b7 subvendor=3D0x1043 subdevice=3D0x8877 > vendor =3D 'Advanced Micro Devices, Inc. [AMD]' > device =3D 'Raphael/Granite Ridge USB 3.1 xHCI' > class =3D serial bus > subclass =3D USB > > Although now I'm longer using this at all for USB functionality in the VM= . > I've switched to: > > ppt2@pci0:13:0:0: class=3D0x0c0330 rev=3D0x01 hdr=3D0x00 vendor=3D0x1022 > device=3D0x43f7 subvendor=3D0x1b21 subdevice=3D0x1142 > vendor =3D 'Advanced Micro Devices, Inc. [AMD]' > device =3D '600 Series Chipset USB 3.2 Controller' > class =3D serial bus > subclass =3D USB > > Much cleaner and still contains a variety of high speed ports. Although > ultimately I'm just using one of them because I have a KVM connected to i= t > that allows me to plug in keyboard/mouse/3.5mm jack (audio)/ethernet all > through one USB wire. Although to remind everyone, I'm gaming in an offli= ne > capacity to I intentionally mess up the IPv4 gateway so my VM can only > communicate internally (and I can sideload apps/games) but prevent > Microsoft from reaching my machine and collecting telemetry info, and doi= ng > forced updates. Plus, I have no intention of moving to Windows 11 with al= l > of their new requirements and forced online activation (I know of the > workarounds, but I'm against on mandating a MSA on principle). People > should be able to fully use their machines even during installation/OOBE > without having to authenticate to a central server. Anyways, small rant > over lol. > > I'll update everyone on my progress with this re-configuration. > > Jonathan Vasquez > PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279 > Sent with ProtonMail Secure Email > > > On Saturday, September 20th, 2025 at 20:02, Jonathan Vasquez < > jon@xyinn.org> wrote: > > Hey Nuno, > > > 1. I'm not. I plugged in my monitor's HDMI cable directly into the > GPU's HDMI port. Once bhyve starts up (if your passthrough is working > correctly and your GPU's drivers are installed), it will output to you= r > monitor. You would only need VNC for initial installation of Windows. > 2. I'm not sure of the NVIDIA case. I do know there is someone who > blogged about how they got theirs working: > https://dflund.se/~getz/Notes/2024/freebsd-gpu/ > > > Atm I'm currently trying to collect more info on why my bhyve VM randomly > crashes sometimes. So far I've collected that the exit reason bhyve is > showing is 0x60, which points to the following entries in the amd freebsd > kernel sources: > > 0x60 > > https://github.com/freebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe136= 5978a282178/sys/amd64/vmm/amd/vmcb.h#L195 > > https://github.com/freebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe136= 5978a282178/sys/amd64/vmm/amd/vmcb.h#L128C1-L128C30 > > XSETBV > > https://github.com/freebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe136= 5978a282178/sys/amd64/vmm/amd/vmcb.h#L94 > > I also asked Proton's Lumo+ AI what is 0x60 to see what it comes up with, > it said the following: > > exit_reason[0] =3D 0x60 is the VM=E2=80=91exit code that bhyve reports wh= en the > guest triggers the XSETBV instruction. > Why it happens > > XSETBV writes to the extended control register XCR0. > In Intel=E2=80=AFVMX (the virtualization technology bhyve uses on x86= ), > execution of XSETBV is not allowed to run directly in the guest; the CPU > forces a VM=E2=80=91exit so the hypervisor can validate the operation. > bhyve records the exit reason as 0x60, which maps to the VMX exit > reason =E2=80=9CXSETBV=E2=80=9D in the Intel specification. > > What it means for you > > The guest tried to enable or change AVX=E2=80=91512 / XSAVE feature b= its via > XSETBV. > Because bhyve does not currently emulate that instruction, it aborts > the guest=E2=80=99s instruction stream and reports the exit code. > In most cases the guest will continue after bhyve handles the exit, > but if the hypervisor isn=E2=80=99t set up to emulate XSETBV you=E2=80=99= ll see the VM stop > or crash. > > Typical ways to deal with it > > Update bhyve =E2=80=93 newer releases include basic XSETBV handling f= or common > feature sets. > Adjust the guest=E2=80=99s CPU flags =E2=80=93 start the VM with a CP= U model that > doesn=E2=80=99t expose the XSAVE/XGETBV features the guest expects (e.g.,= -c host > vs. a specific model). > Patch bhyve =E2=80=93 if you=E2=80=99re comfortable with C, you can a= dd a handler for > VMX_EXIT_REASON_XSETBV that either emulates the instruction or simply > ignores it when the requested bits are already enabled. > > If you keep seeing the same exit repeatedly, it usually points to a > mismatch between the guest=E2=80=99s expectations (often a modern Linux k= ernel or a > Windows build that wants AVX=E2=80=91512) and the capabilities bhyve is p= resenting. > Aligning those expectations=E2=80=94either by tweaking the VM=E2=80=99s C= PU configuration > or by upgrading bhyve=E2=80=94generally resolves the issue. > > > Jonathan Vasquez > PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279 > Sent with ProtonMail Secure Email > > > On Saturday, September 20th, 2025 at 19:47, Nuno Teixeira < > eduardo@freebsd.org> wrote: > > Hello Jonathan! > > Thanks for excellent quality videos on yt, I already subscribed it :) > > Just a quick questions: > > 1- Are you using freerdp3 to connect to win11pro? (Asking this because vn= c > is very slow) > 2- Do you know how is nvidia status compared to your use case? > > Thanks! > > Jonathan Vasquez <jon@xyinn.org> escreveu (s=C3=A1bado, 20/09/2025 =C3=A0= (s) 02:42): > >> Hey all, I've spent some time today doing a gaming demo. I've uploaded >> the video in 2K so you can get the maximum effect. It's been an interest= ing >> experience so far, and I'm going to try to make it my primary gaming >> machine. We'll see how it goes. It's definitely qwirky, and sometimes I >> need to reboot the VM 4-5 times before it "stabilizes" (which means the = VM >> won't just randomly crash or have massive lag in game). I'll need to >> collect more data through play testing. >> >> You can take a look at the video here (I'll post timestamps in the video >> soon): >> >> https://youtu.be/_cz0RUAw5p8 >> >> Jonathan Vasquez >> PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279 >> Sent with ProtonMail Secure Email >> >> >> >> On Wednesday, September 17th, 2025 at 22:33, Jonathan Vasquez < >> jon@xyinn.org> wrote: >> >> > Alright! The YouTube Video is up. Let me know what you think. I'm >> really excited about this. >> > >> > https://youtu.be/Ob4-v7dTJGs >> >> > > -- > Nuno Teixeira > FreeBSD UNIX: <eduardo@FreeBSD.org> Web: https://FreeBSD.org > <https://freebsd.org/>; > > > > > > --000000000000fd962e063f5e34ad Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div>what i noticed that the new pc have very complex bio= s setup options that may impact stability of freebsd. i encountered this re= cently after hw upgrade. pci-e gen 4 or 5, x16 or x8 mode, and split contro= ller between single x16 for gpu or x8 for gpu and x8/x4 between other perip= herals such as nvme controller usb additional card slots etc. in addition t= o that there are power modes and runtime frequency changes for cpu and all = peripherals. not all new features are supported by freebsd. it took me some= while to adjust optimal settings to get machine stable (at the cost of pow= er savings). also there were irritating glitches in audio that are now cons= iderably less noticable.</div><div dir=3D"auto"><br></div><div dir=3D"auto"= >long story short it is cpu, ram, and peripherals settings that may impact = overall stability of the freebsd box.</div><div dir=3D"auto"><br></div><div= dir=3D"auto">this work seems to be not only important for virtualization b= ut also FreeBSD's LDWG project (Laptop Desktop Work Group) i am attachi= ng freebsd-desktop@ mailing list :-)</div><div dir=3D"auto"><br></div><div>= <br></div><div data-smartmail=3D"gmail_signature">--<br>CeDeROM, SQ7MHZ, <a= href=3D"http://www.tomek.cedro.info">http://www.tomek.cedro.info</a></div>= </div><br><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr"= class=3D"gmail_attr">On Mon, Sep 22, 2025, 02:20 Jonathan Vasquez <<a h= ref=3D"mailto:jon@xyinn.org">jon@xyinn.org</a>> wrote:<br></div><blockqu= ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px= solid rgb(204,204,204);padding-left:1ex"><div style=3D"font-family:Arial,s= ans-serif;font-size:14px">Hey all,</div><div style=3D"font-family:Arial,san= s-serif;font-size:14px"><br></div><div style=3D"font-family:Arial,sans-seri= f;font-size:14px">Just wanted to report that I still haven't received o= ne crash after my adjustments yesterday. So far over 20+ hours of uptime~. = I've also rebooted the VM/host a few times just to ensure that it is de= terministically stable and so far zero issues. So this is pointing in a pos= itive direction. I'm tempted to mark this as a stable set up but I'= ll need to play for longer to make that call.</div><div style=3D"font-famil= y:Arial,sans-serif;font-size:14px"><br></div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"> <div> <div>Jonathan Vasquez<br></div><div>PGP: 34DA 858C 1447 509E C77A = D49F FB85 90B7 C4CA 5279<br></div><div>Sent with ProtonMail Secure Email<br= ></div><div><br></div> </div> =20 <div> =20 </div> </div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"><br></div><div> On Sunday, September 21st, 2025 at 08:23, Jonathan Vasquez <<a h= ref=3D"mailto:jon@xyinn.org" target=3D"_blank" rel=3D"noreferrer">jon@xyinn= .org</a>> wrote:<br> <blockquote type=3D"cite"> System has been up for about 9h40m. Most of it idle and no cras= hes yet. I'm gonna reboot the host and see if the stability still conti= nues.<br><br><div><div>Jonathan Vasquez<br></div><div>PGP: 34DA 858C 1447 5= 09E C77A D49F FB85 90B7 C4CA 5279<br></div><div>Sent with ProtonMail Secur= e Email<br></div><div><br></div></div><br><br>Sent from <a href=3D"https://= proton.me/mail/home" rel=3D"noreferrer nofollow noopener noreferrer" target= =3D"_blank">Proton Mail</a> for Android.<div><br><br>-------- Original Mess= age --------<br>On Saturday, 09/20/25 at 23:45 Jonathan Vasquez <<a href= =3D"mailto:jon@xyinn.org" target=3D"_blank" rel=3D"noreferrer">jon@xyinn.or= g</a>> wrote:<br><blockquote><div style=3D"font-family:Arial,sans-serif;= font-size:14px">Just finished beating the main story line in Cyberpunk 2077= in the VM. So far no crashes and it's been running for an hour. So thi= s is a positive signal... we'll see what happens. I'll leave the VM= running idle over night and see if it's still running in the morning. = Stay safe all!</div><div style=3D"font-family:Arial,sans-serif;font-size:14= px"><br></div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"> <div> <div>Jonathan Vasquez<br></div><div>PGP: 34DA 858C 1447 509E C77A = D49F FB85 90B7 C4CA 5279<br></div><div>Sent with ProtonMail Secure Email<br= ></div><div><br></div> </div> <div> </div> </div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"><br></div><div> On Saturday, September 20th, 2025 at 22:28, Jonathan Vasquez <<a= href=3D"mailto:jon@xyinn.org" target=3D"_blank" rel=3D"noreferrer">jon@xyi= nn.org</a>> wrote:<br> <blockquote type=3D"cite"> <div style=3D"font-family:Arial,sans-serif;font-size:14px">I di= d a bunch of more experiments today and I was able to switch from using the= USB controller connected to the multi-functional bus that has the integrat= ed card, and switched to a separate USB controller that only has USB ports.= I'm hoping that this increases VM stability and that there may have be= en some wires being crossed. This was the previous bus I was using (I could= n't isolate just the USB functions, it had to be all or nothing for it = to function):</div><div style=3D"font-family:Arial,sans-serif;font-size:14p= x"><br></div><div style=3D"font-family:Arial,sans-serif;font-size:14px"><sp= an>vgapci0@pci0:18:0:0: class=3D0x030000 rev=3D0xc1 hdr=3D0x00 vendor=3D0x1= 002 device=3D0x164e subvendor=3D0x1043 subdevice=3D0x8877</span><div><span>= =C2=A0 =C2=A0 vendor =C2=A0 =C2=A0 =3D 'Advanced Micro Devices, Inc. [A= MD/ATI]'</span></div><div><span>=C2=A0 =C2=A0 device =C2=A0 =C2=A0 =3D = 'Raphael'</span></div><div><span>=C2=A0 =C2=A0 class =C2=A0 =C2=A0 = =C2=A0=3D display</span></div><div><span>=C2=A0 =C2=A0 subclass =C2=A0 =3D = VGA</span></div><div><span>hdac0@pci0:18:0:1: class=3D0x040300 rev=3D0x00 h= dr=3D0x00 vendor=3D0x1002 device=3D0x1640 subvendor=3D0x1043 subdevice=3D0x= 8877</span></div><div><span>=C2=A0 =C2=A0 vendor =C2=A0 =C2=A0 =3D 'Adv= anced Micro Devices, Inc. [AMD/ATI]'</span></div><div><span>=C2=A0 =C2= =A0 device =C2=A0 =C2=A0 =3D 'Rembrandt Radeon High Definition Audio Co= ntroller'</span></div><div><span>=C2=A0 =C2=A0 class =C2=A0 =C2=A0 =C2= =A0=3D multimedia</span></div><div><span>=C2=A0 =C2=A0 subclass =C2=A0 =3D = HDA</span></div><div><span>none1@pci0:18:0:2: class=3D0x108000 rev=3D0x00 h= dr=3D0x00 vendor=3D0x1022 device=3D0x1649 subvendor=3D0x1043 subdevice=3D0x= 8877</span></div><div><span>=C2=A0 =C2=A0 vendor =C2=A0 =C2=A0 =3D 'Adv= anced Micro Devices, Inc. [AMD]'</span></div><div><span>=C2=A0 =C2=A0 d= evice =C2=A0 =C2=A0 =3D 'Family 19h PSP/CCP'</span></div><div><span= >=C2=A0 =C2=A0 class =C2=A0 =C2=A0 =C2=A0=3D encrypt/decrypt</span></div><d= iv><span>xhci1@pci0:18:0:3: class=3D0x0c0330 rev=3D0x00 hdr=3D0x00 vendor= =3D0x1022 device=3D0x15b6 subvendor=3D0x1043 subdevice=3D0x8877</span></div= ><div><span>=C2=A0 =C2=A0 vendor =C2=A0 =C2=A0 =3D 'Advanced Micro Devi= ces, Inc. [AMD]'</span></div><div><span>=C2=A0 =C2=A0 device =C2=A0 =C2= =A0 =3D 'Raphael/Granite Ridge USB 3.1 xHCI'</span></div><div><span= >=C2=A0 =C2=A0 class =C2=A0 =C2=A0 =C2=A0=3D serial bus</span></div><div><s= pan>=C2=A0 =C2=A0 subclass =C2=A0 =3D USB</span></div><div><span>xhci2@pci0= :18:0:4: class=3D0x0c0330 rev=3D0x00 hdr=3D0x00 vendor=3D0x1022 device=3D0x= 15b7 subvendor=3D0x1043 subdevice=3D0x8877</span></div><div><span>=C2=A0 = =C2=A0 vendor =C2=A0 =C2=A0 =3D 'Advanced Micro Devices, Inc. [AMD]'= ;</span></div><div><span>=C2=A0 =C2=A0 device =C2=A0 =C2=A0 =3D 'Raphae= l/Granite Ridge USB 3.1 xHCI'</span></div><div><span>=C2=A0 =C2=A0 clas= s =C2=A0 =C2=A0 =C2=A0=3D serial bus</span></div><span>=C2=A0 =C2=A0 subcla= ss =C2=A0 =3D USB</span></div><div style=3D"font-family:Arial,sans-serif;fo= nt-size:14px"><br></div><div style=3D"font-family:Arial,sans-serif;font-siz= e:14px">Although now I'm longer using this at all for USB functionality= in the VM. I've switched to:</div><div style=3D"font-family:Arial,sans= -serif;font-size:14px"><br></div><div style=3D"font-family:Arial,sans-serif= ;font-size:14px"><span>ppt2@pci0:13:0:0: class=3D0x0c0330 rev=3D0x01 hdr=3D= 0x00 vendor=3D0x1022 device=3D0x43f7 subvendor=3D0x1b21 subdevice=3D0x1142<= /span><div><span>=C2=A0 =C2=A0 vendor =C2=A0 =C2=A0 =3D 'Advanced Micro= Devices, Inc. [AMD]'</span></div><div><span>=C2=A0 =C2=A0 device =C2= =A0 =C2=A0 =3D '600 Series Chipset USB 3.2 Controller'</span></div>= <div><span>=C2=A0 =C2=A0 class =C2=A0 =C2=A0 =C2=A0=3D serial bus</span></d= iv><span>=C2=A0 =C2=A0 subclass =C2=A0 =3D USB</span></div><div style=3D"fo= nt-family:Arial,sans-serif;font-size:14px"><br></div><div style=3D"font-fam= ily:Arial,sans-serif;font-size:14px">Much cleaner and still contains a vari= ety of high speed ports. Although ultimately I'm just using one of them= because I have a KVM connected to it that allows me to plug in keyboard/mo= use/3.5mm jack (audio)/ethernet all through one USB wire. Although to remin= d everyone, I'm gaming in an offline capacity to I intentionally mess u= p the IPv4 gateway so my VM can only communicate internally (and I can side= load apps/games) but prevent Microsoft from reaching my machine and collect= ing telemetry info, and doing forced updates. Plus, I have no intention of = moving to Windows 11 with all of their new requirements and forced online a= ctivation (I know of the workarounds, but I'm against on mandating a MS= A on principle). People should be able to fully use their machines even dur= ing installation/OOBE without having to authenticate to a central server. A= nyways, small rant over lol.</div><div style=3D"font-family:Arial,sans-seri= f;font-size:14px"><br></div><div style=3D"font-family:Arial,sans-serif;font= -size:14px">I'll update everyone on my progress with this re-configurat= ion.</div><div style=3D"font-family:Arial,sans-serif;font-size:14px"><br></= div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"> <div> <div>Jonathan Vasquez<br></div><div>PGP: 34DA 858C 1447 509E C77A = D49F FB85 90B7 C4CA 5279<br></div><div>Sent with ProtonMail Secure Email<br= ></div><div><br></div> </div> <div> </div> </div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"><br></div><div> On Saturday, September 20th, 2025 at 20:02, Jonathan Vasquez <<a= href=3D"mailto:jon@xyinn.org" target=3D"_blank" rel=3D"noreferrer">jon@xyi= nn.org</a>> wrote:<br> <blockquote type=3D"cite"> <div style=3D"font-family:Arial,sans-serif;font-size:14px">Hey = Nuno,</div><div style=3D"font-family:Arial,sans-serif;font-size:14px"><br><= /div><div style=3D"font-family:Arial,sans-serif;font-size:14px"><ol style= =3D"margin-top:0px;margin-bottom:0px"><li style=3D"list-style-type:"1.= ""><span>I'm not. I plugged in my monitor's HDMI cable direct= ly into the GPU's HDMI port. Once bhyve starts up (if your passthrough = is working correctly and your GPU's drivers are installed), it will out= put to your monitor. You would only need VNC for initial installation of Wi= ndows.</span></li><li style=3D"list-style-type:"2. ""><span>I'= ;m not sure of the NVIDIA case. I do know there is someone who blogged abou= t how they got theirs working:=C2=A0<span><a rel=3D"noreferrer nofollow noo= pener noreferrer" href=3D"https://dflund.se/~getz/Notes/2024/freebsd-gpu/" = target=3D"_blank">https://dflund.se/~getz/Notes/2024/freebsd-gpu/</a></span= ></span></li></ol><div><br></div><div>Atm I'm currently trying to colle= ct more info on why my bhyve VM randomly crashes sometimes. So far I've= collected that the exit reason bhyve is showing is 0x60, which points to t= he following entries in the amd freebsd kernel sources:</div><div><br></div= ><div><span>0x60</span><div><span><a rel=3D"noreferrer nofollow noopener no= referrer" href=3D"https://github.com/freebsd/freebsd-src/blob/4c3a868d13c05= 3ef173268cdfe1365978a282178/sys/amd64/vmm/amd/vmcb.h#L195" target=3D"_blank= ">https://github.com/freebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe136= 5978a282178/sys/amd64/vmm/amd/vmcb.h#L195</a></span></div><div><span><a rel= =3D"noreferrer nofollow noopener noreferrer" href=3D"https://github.com/fre= ebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe1365978a282178/sys/amd64/vm= m/amd/vmcb.h#L128C1-L128C30" target=3D"_blank">https://github.com/freebsd/f= reebsd-src/blob/4c3a868d13c053ef173268cdfe1365978a282178/sys/amd64/vmm/amd/= vmcb.h#L128C1-L128C30</a></span></div><div><br></div><div><span>XSETBV</spa= n></div><div><span><a rel=3D"noreferrer nofollow noopener noreferrer" href= =3D"https://github.com/freebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe1= 365978a282178/sys/amd64/vmm/amd/vmcb.h#L94" target=3D"_blank">https://githu= b.com/freebsd/freebsd-src/blob/4c3a868d13c053ef173268cdfe1365978a282178/sys= /amd64/vmm/amd/vmcb.h#L94</a></span></div><span></span></div><div><br></div= ><div>I also asked Proton's Lumo+ AI what is 0x60 to see what it comes = up with, it said the following:</div><div><br></div><div><span>exit_reason[= 0] =3D 0x60 is the VM=E2=80=91exit code that bhyve reports when the guest t= riggers the XSETBV instruction.</span><div><span>Why it happens</span></div= ><div><br></div><div><span>=C2=A0 =C2=A0 XSETBV writes to the extended cont= rol register XCR0.</span></div><div><span>=C2=A0 =C2=A0 In Intel=E2=80=AFVM= X (the virtualization technology bhyve uses on x86), execution of XSETBV is= not allowed to run directly in the guest; the CPU forces a VM=E2=80=91exit= so the hypervisor can validate the operation.</span></div><div><span>=C2= =A0 =C2=A0 bhyve records the exit reason as 0x60, which maps to the VMX exi= t reason =E2=80=9CXSETBV=E2=80=9D in the Intel specification.</span></div><= div><br></div><div><span>What it means for you</span></div><div><br></div><= div><span>=C2=A0 =C2=A0 The guest tried to enable or change AVX=E2=80=91512= / XSAVE feature bits via XSETBV.</span></div><div><span>=C2=A0 =C2=A0 Beca= use bhyve does not currently emulate that instruction, it aborts the guest= =E2=80=99s instruction stream and reports the exit code.</span></div><div><= span>=C2=A0 =C2=A0 In most cases the guest will continue after bhyve handle= s the exit, but if the hypervisor isn=E2=80=99t set up to emulate XSETBV yo= u=E2=80=99ll see the VM stop or crash.</span></div><div><br></div><div><spa= n>Typical ways to deal with it</span></div><div><br></div><div><span>=C2=A0= =C2=A0 Update bhyve =E2=80=93 newer releases include basic XSETBV handling= for common feature sets.</span></div><div><span>=C2=A0 =C2=A0 Adjust the g= uest=E2=80=99s CPU flags =E2=80=93 start the VM with a CPU model that doesn= =E2=80=99t expose the XSAVE/XGETBV features the guest expects (e.g., -c hos= t vs. a specific model).</span></div><div><span>=C2=A0 =C2=A0 Patch bhyve = =E2=80=93 if you=E2=80=99re comfortable with C, you can add a handler for V= MX_EXIT_REASON_XSETBV that either emulates the instruction or simply ignore= s it when the requested bits are already enabled.</span></div><div><br></di= v><div><span>If you keep seeing the same exit repeatedly, it usually points= to a mismatch between the guest=E2=80=99s expectations (often a modern Lin= ux kernel or a Windows build that wants AVX=E2=80=91512) and the capabiliti= es bhyve is presenting. Aligning those expectations=E2=80=94either by tweak= ing the VM=E2=80=99s CPU configuration or by upgrading bhyve=E2=80=94genera= lly resolves the issue.</span></div><div><br></div><br></div></div><div sty= le=3D"font-family:Arial,sans-serif;font-size:14px"> <div> <div>Jonathan Vasquez<br></div><div>PGP: 34DA 858C 1447 509E C77A = D49F FB85 90B7 C4CA 5279<br></div><div>Sent with ProtonMail Secure Email<br= ></div><div><br></div> </div> <div> </div> </div> <div style=3D"font-family:Arial,sans-serif;font-size:14px"><br></div><div> On Saturday, September 20th, 2025 at 19:47, Nuno Teixeira <<a hr= ef=3D"mailto:eduardo@freebsd.org" target=3D"_blank" rel=3D"noreferrer">edua= rdo@freebsd.org</a>> wrote:<br> <blockquote type=3D"cite"> <div dir=3D"ltr"><div>Hello Jonathan!</div><div><br></div><div>= Thanks for excellent quality videos on yt, I already subscribed it :)</div>= <div><br></div><div>Just a quick questions:</div><div><br></div><div>1- Are= you using freerdp3 to connect to win11pro? (Asking this because vnc is ve= ry slow)</div><div>2- Do you know how is nvidia status compared to your use= case?</div><div><br></div><div>Thanks!</div></div><br><div class=3D"gmail_= quote"><div dir=3D"ltr" class=3D"gmail_attr">Jonathan Vasquez <<a href= =3D"mailto:jon@xyinn.org" rel=3D"noreferrer nofollow noopener noreferrer" t= arget=3D"_blank">jon@xyinn.org</a>> escreveu (s=C3=A1bado, 20/09/2025 = =C3=A0(s) 02:42):<br></div><blockquote class=3D"gmail_quote" style=3D"margi= n:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex= ">Hey all, I've spent some time today doing a gaming demo. I've upl= oaded the video in 2K so you can get the maximum effect. It's been an i= nteresting experience so far, and I'm going to try to make it my primar= y gaming machine. We'll see how it goes. It's definitely qwirky, an= d sometimes I need to reboot the VM 4-5 times before it "stabilizes&qu= ot; (which means the VM won't just randomly crash or have massive lag i= n game). I'll need to collect more data through play testing.<br> <br> You can take a look at the video here (I'll post timestamps in the vide= o soon):<br> <br> <a href=3D"https://youtu.be/_cz0RUAw5p8" rel=3D"noreferrer nofollow noopene= r noreferrer" target=3D"_blank">https://youtu.be/_cz0RUAw5p8</a><br>; <br> Jonathan Vasquez<br> PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279<br> Sent with ProtonMail Secure Email<br> <br> <br> <br> On Wednesday, September 17th, 2025 at 22:33, Jonathan Vasquez <<a href= =3D"mailto:jon@xyinn.org" rel=3D"noreferrer nofollow noopener noreferrer" t= arget=3D"_blank">jon@xyinn.org</a>> wrote:<br> <br> > Alright! The YouTube Video is up. Let me know what you think. I'm = really excited about this.<br> > <br> > <a href=3D"https://youtu.be/Ob4-v7dTJGs" rel=3D"noreferrer nofollow no= opener noreferrer" target=3D"_blank">https://youtu.be/Ob4-v7dTJGs</a><br>; <br> </blockquote></div><div><br clear=3D"all"></div><br><span class=3D"gmail_si= gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d= iv dir=3D"ltr"><div><font color=3D"#888888">Nuno Teixeira</font></div><div>= <div><font color=3D"#888888"> FreeBSD UNIX: <eduardo@FreeBSD.org> Web: <a href=3D"https://freeb= sd.org/" rel=3D"noreferrer nofollow noopener noreferrer" target=3D"_blank">= https://FreeBSD.org</a><br></font></div></div></div></div>; </blockquote><br> </div> </blockquote><br> </div> </blockquote><br> </div></blockquote></div> =20 </blockquote><br> </div></blockquote></div> --000000000000fd962e063f5e34ad--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFYkXjmujHH4pWH%2BgEt%2BV9ch4Yu16M1%2BxMPcGeX7yw79xkBwiA>