Date: Sat, 03 Dec 2022 17:34:30 +0100 From: =?ISO-8859-1?Q?Corvin_K=F6hne?= <corvink@FreeBSD.org> To: virtualization@freebsd.org, Mario Marietto <marietto2008@gmail.com>, freebsd-x11+help@freebsd.org, FreeBSD virtualization <freebsd-virtualization@freebsd.org>, hackers@freebsd.org Subject: =?US-ASCII?Q?Re=3A_How_to_use_the_framebuffer_?= =?US-ASCII?Q?as_primary_video_device_instead?= =?US-ASCII?Q?_of_the_nvidia_passed-through_g?= =?US-ASCII?Q?raphic_card_in_a_bhyve/linux_vm?= Message-ID: <2E8B4BF7-A39E-4B93-820D-FC66F8C2F94B@FreeBSD.org> In-Reply-To: <CA%2B1FSigGi%2BY-pqS9ARwY%2BdpRS2HBc-wxc0cd5ixKTAw%2Ba6uNZw@mail.gmail.com> References: <CA%2B1FSigGi%2BY-pqS9ARwY%2BdpRS2HBc-wxc0cd5ixKTAw%2Ba6uNZw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------6U2JCXSDZKQSLVE05FUH4M8CDP1MWU
Content-Transfer-Encoding: 7bit
Content-Type: multipart/alternative;
boundary=----D9Z2G0TV1R1H3IMXZUCKA7G3Y7ATBL
------D9Z2G0TV1R1H3IMXZUCKA7G3Y7ATBL
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
On December 3, 2022 4:49:46 PM GMT+01:00, Mario Marietto <marietto2008@gmai=
l=2Ecom> wrote:
>Hello to everyone=2E
>
>what Im trying to do is to set the framebuffer video adapter as primary
>graphic card on my bhyve-ubuntu vm instead of the nvidia RTX 2080 ti card
>that I have passed through=2E What I want to do really is to use both the
>graphic adapters,but the primary should be the framebuffer and the
>secondary the nvidia=2E I tried different Xorg configurations,but what I'=
ve
>got is that Xorg failed to display some errors=2E So,the controller that =
you
>see below should be used as primary inside the ubuntu vm :
>
>00:1d=2E0 VGA compatible controller: Device fb5d:40fb
>
>while the ones you see below as secondary :
>
>08:00=2E0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce
>RTX 2080 Ti] (rev a1)
>08:00=2E1 Audio device: NVIDIA Corporation TU102 High Definition Audio
>Controller (rev a1)
>08:00=2E2 USB controller: NVIDIA Corporation TU102 USB 3=2E1 Host
>Controller (rev a1)
>08:00=2E3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C
>UCSI Controller (rev a1)
>
>
>The script that I use to launch the vm is the following :
>
>#!/bin/sh
>setxkbmap it
>vms=3D"$(ls /dev/vmm/*)"
>vncs=3D"$(ps ax | awk '/vncviewer [0]/{print $6}')"
>
>for vm in $vms; do
> session=3D"${vm##*/}"
> echo "bhyve session =3D $session"
> echo "vnc session =3D $vncs"
> if ! printf '%s\n' "${vncs}" | grep "${session}"; then
> printf 'VNC session not
>found,destroying ghost vms\n'
> bhyvectl --vm=3D$session --destroy
> else
> printf 'Found VNC session %s\n'
>"${session},no ghost vms found,not destroying them"
> fi
>done
>
>vmdisk1=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident:
>(2015020204055E)/ && d{print d}'`
>echo "TOSHIBA External USB 3=2E0 1=2E8 TB ; $vmdisk1"
>
>mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
>
>bhyve -S -c sockets=3D1,cores=3D2,threads=3D2 -m 4G -w -H -A \
>-s 0,hostbridge \
>-s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210=2Eimg,bootin=
dex=3D1 \
>-s 3,virtio-blk,/dev/$vmdisk4 \
>-s 4,virtio-blk,/dev/$vmdisk2 \
>-s 8:0,passthru,2/0/0 \
>-s 8:1,passthru,2/0/1 \
>-s 8:2,passthru,2/0/2 \
>-s 8:3,passthru,2/0/3 \
>-s 10,virtio-net,tap19 \
>-s 11,virtio-9p,sharename=3D/ \
>-s 29,fbuf,tcp=3D0=2E0=2E0=2E0:5919,w=3D1600,h=3D950,wait \
>-s 30,xhci,tablet \
>-s 31,lpc \
>-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE=2Efd \
>vm0:19 < /dev/null & sleep 2 && vncviewer 0:19
>
>For sure ,on /boot/loader=2Econf I've added :
>
>/boot/loader=2Econf
>
>pptdevs=3D"2/0/0 2/0/1 2/0/2 2/0/3"
>
>As I said before,I tried two xorg conf files to achieve the goal=2E On th=
e
>first one I tried to add only the framebuffer,like this :
>
>
>Section "Files"
> ModulePath "/usr/lib/xorg/modules"
> FontPath "/usr/share/fonts/X11/misc"
> FontPath "/usr/share/fonts/X11/cyrillic"
> FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
> FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
> FontPath "/usr/share/fonts/X11/Type1"
> FontPath "/usr/share/fonts/X11/100dpi"
> FontPath "/usr/share/fonts/X11/75dpi"
> FontPath "built-ins"
>EndSection
>
>Section "Module"
> Load "vnc"
> Load "glx"
>EndSection
>
>
>Section "InputDevice"
> Identifier "Keyboard0"
> Driver "kbd"
>EndSection
>
>Section "InputDevice"
> Identifier "Mouse0"
> Driver "mouse"
> Option "Protocol" "auto"
> Option "Device" "/dev/input/mice"
> Option "ZAxisMapping" "4 5 6 7"
>
>EndSection
>
>Section "Monitor"
> Identifier "Monitor0"
> VendorName "Monitor Vendor"
> ModelName "Monitor Model"
>EndSection
>
>Section "Device"
> Identifier "Card0"
> Driver "modesetting"
> BusID "PCI:0:29:0"
>
>EndSection
>
>Section "Screen"
> Identifier "Screen0"
> Device "Card0"
> Monitor "Monitor0"
> SubSection "Display"
> Viewport 0 0
> Depth 1
> EndSubSection
> SubSection "Display"
> Viewport 0 0
> Depth 4
> EndSubSection
> SubSection "Display"
> Viewport 0 0
> Depth 8
> EndSubSection
> SubSection "Display"
> Viewport 0 0
> Depth 15
> EndSubSection
> SubSection "Display"
> Viewport 0 0
> Depth 16
> EndSubSection
> SubSection "Display"
> Viewport 0 0
> Depth 24
> EndSubSection
>EndSection
>
>
>but it didn't work=2E This is the log file that shows the errors
>reported : https://pastebin=2Eubuntu=2Ecom/p/Gv7wgsDR4K/
>I have also removed the xorg=2Econf file,but it didn't work either=2E Thi=
s
>is the log file that shows the errors reported :
>
>
>https://pastebin=2Eubuntu=2Ecom/p/wNcfQTByQm/
>
>Can someone give me some suggestions that can help me to understand
>where the mistake is,please,thanks=2E
>
>--=20
>Mario=2E
Try to assign a lower pci slot number to the framebuffer device than to th=
e nvidia gpu in your bhyve command=2E
--=20
Best regards,
Corvin
------D9Z2G0TV1R1H3IMXZUCKA7G3Y7ATBL
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div class=3D"gmail_quote">On December 3, 2022 4:4=
9:46 PM GMT+01:00, Mario Marietto <marietto2008@gmail=2Ecom> wrote:<b=
lockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt 0=2E8ex; borde=
r-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir=3D"ltr">Hello to everyone=2E<br>
<br>
what Im trying to do is to set the framebuffer video adapter as primary=20
graphic card on my bhyve-ubuntu vm instead of the nvidia RTX 2080 ti=20
card that I have passed through=2E What I want to do really is to use both
the graphic adapters,but the primary should be the framebuffer and the=20
secondary the nvidia=2E I tried different Xorg configurations,but what I'v=
e
got is that Xorg failed to display some errors=2E So,the controller
that you see below should be used as primary inside the ubuntu vm :<br><d=
iv><br></div><div><div class=3D"gmail-bbCodeBlock-content" dir=3D"ltr">
<pre class=3D"gmail-bbCodeCode" dir=3D"ltr"><code>00:1d=2E0 VGA compatib=
le controller: Device fb5d:40fb<br><br></code></pre>
</div><div class=3D"gmail-bbCodeBlock-content" dir=3D"ltr">
while the ones you see below as secondary :</div><div class=3D"gmail-bbCod=
eBlock-content" dir=3D"ltr"><br></div></div><div><pre class=3D"gmail-bbCode=
Code" dir=3D"ltr"><code>08:00=2E0 VGA compatible controller: NVIDIA Corpora=
tion TU102 [GeForce RTX 2080 Ti] (rev a1)
08:00=2E1 Audio device: NVIDIA Corporation TU102 High Definition Audio Con=
troller (rev a1)
08:00=2E2 USB controller: NVIDIA Corporation TU102 USB 3=2E1 Host Controll=
er (rev a1)
08:00=2E3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI =
Controller (rev a1)</code></pre></div><div><br></div><div>
The script that I use to launch the vm is the following :</div><div><br></=
div><div><pre class=3D"gmail-bbCodeCode" dir=3D"ltr"><code>#!/bin/sh
setxkbmap it
vms=3D"$(ls /dev/vmm/*)"
vncs=3D"$(ps ax | awk '/vncviewer [0]/{print $6}')"
for vm in $vms; do
session=3D"${vm##*/}"
echo "bhyve session =3D $session"
echo "vnc session =3D $vncs"
if ! printf '%s\n' "${vncs}" | grep "${session}"; then
printf 'VNC session not found,destroying g=
host vms\n'
bhyvectl --vm=3D$session --destroy =
=20
else
printf 'Found VNC session %s\n' "${session=
},no ghost vms found,not destroying them"
fi
done
vmdisk1=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (20150=
20204055E)/ && d{print d}'`
echo "TOSHIBA External USB 3=2E0 1=2E8 TB ; $vmdisk1"
mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
bhyve -S -c sockets=3D1,cores=3D2,threads=3D2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210=2Eimg,bootind=
ex=3D1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2 \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=3D/ \
-s 29,fbuf,tcp=3D<a href=3D"http://0=2E0=2E0=2E0:5919">0=2E0=2E0=2E0:5919<=
/a>,w=3D1600,h=3D950,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE=2Efd \
vm0:19 < /dev/null & sleep 2 && vncviewer 0:19<br><br>
<span style=3D"font-family:arial black,sans-serif">For sure ,on /boot/load=
er=2Econf I've added :<br><br></span></code></pre></div><div><pre class=3D"=
gmail-bbCodeCode" dir=3D"ltr"><code>/boot/loader=2Econf
pptdevs=3D"2/0/0 2/0/1 2/0/2 2/0/3"<br><br>
<span style=3D"font-family:arial black,sans-serif">As I said before,I trie=
d two xorg conf files to achieve the goal=2E On the
first one I tried to add only the framebuffer,like this :</span></code></p=
re></div><div><pre class=3D"gmail-bbCodeCode" dir=3D"ltr"><code><br>Section=
"Files"
ModulePath "/usr/lib/xorg/modules"
FontPath "/usr/share/fonts/X11/misc"
FontPath "/usr/share/fonts/X11/cyrillic"
FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
FontPath "/usr/share/fonts/X11/Type1"
FontPath "/usr/share/fonts/X11/100dpi"
FontPath "/usr/share/fonts/X11/75dpi"
FontPath "built-ins"
EndSection
Section "Module"
Load "vnc"
Load "glx"
EndSection
Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
EndSection
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection
Section "Monitor"
Identifier "Monitor0"
VendorName "Monitor Vendor"
ModelName "Monitor Model"
EndSection
Section "Device"
Identifier "Card0"
Driver "modesetting"
BusID "PCI:0:29:0"
EndSection
Section "Screen"
Identifier "Screen0"
Device "Card0"
Monitor "Monitor0"
SubSection "Display"
Viewport 0 0
Depth 1
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 4
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 8
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 15
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 16
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 24
EndSubSection
EndSection<br><br><br><span style=3D"font-family:arial black,sans-serif">b=
ut it didn't work=2E This is the log file that shows the errors reported : =
<a href=3D"https://pastebin=2Eubuntu=2Ecom/p/Gv7wgsDR4K/" target=3D"_blank"=
class=3D"gmail-link gmail-link--external" rel=3D"nofollow ugc noopener">ht=
tps://pastebin=2Eubuntu=2Ecom/p/Gv7wgsDR4K/</a></span><br><span style=3D"fo=
nt-family:arial black,sans-serif">I have also removed the xorg=2Econf file,=
but it didn't work either=2E This is the log file that shows the errors rep=
orted :</span><br><br><br><a href=3D"https://pastebin=2Eubuntu=2Ecom/p/wNcf=
QTByQm/">https://pastebin=2Eubuntu=2Ecom/p/wNcfQTByQm/</a><br><br>;
<span style=3D"font-family:arial black,sans-serif">Can someone give me som=
e suggestions that can help me to understand where the mistake is,please,th=
anks=2E</span>
</code></pre></div><div>-- </div><div dir=3D"ltr" class=3D"gmail_signature=
" data-smartmail=3D"gmail_signature">Mario=2E<br></div></div>
</blockquote></div><br clear=3D"all">Try to assign a lower pci slot number=
to the framebuffer device than to the nvidia gpu in your bhyve command=2E<=
br><div style=3D'white-space: pre-wrap'><div class=3D'k9mail-signature'>-- =
<br>Best regards,<br>Corvin</div></div></body></html>
------D9Z2G0TV1R1H3IMXZUCKA7G3Y7ATBL--
------6U2JCXSDZKQSLVE05FUH4M8CDP1MWU
Content-Type: application/pgp-signature; name="signature.asc"
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
iQJBBAABCgArJBxDb3J2aW4gS8O2aG5lIDxjb3J2aW5rQEZyZWVCU0Qub3JnPgUC
Y4t6lgAKCRDYVNpWMV4Cau6XEADAvqdR3fbxTsQ+eGtggydEmtwwWdtd75Sltyiv
e5ZMdCiYFzqm+runAOECOSy2+PGiOQqShYOaSgA5bfKm/Gz7IyWZeocq6Pw78RvB
9PRpbsoAxji4Wlxp4By9Bqps99niT30a7h8oi4PB+dmsKY3HKz4tpiB8SP/hzU0h
WTeKBFqgDNxAMDuzNp9lxK6rTqvleVvhavwahkNfg8fFuSU3agmYz//qwQ6SpCUS
psOx9967ypAXfAm8KTiL1JhxJrqOmTI28Bj3Vc7ekZkg8uh5hvaKpdAN6N3DRGbr
LJ5SSjADItcFNpZvEum/3rCY5nYoYCHQ5xuzdJ7IdPlWnamczNKtqcPiQZwM15ya
Nk8Xq+duIUPbnPUlKuf2+Z/cuLwlFXqN+x+UTTRxMhwmzyKJkoLadc24CACmBN5a
0Iifs5CRxGjcEIamgAgzMEAiEkEvT4Id067687DjDeVpUPLXKtWMBzvJYqduaRIE
NSj6rdIBcLFMm9YTxTyAyp54Y+Tw9z7mKQnGiCOoqijsChAq29SQ61p6+8PAZNh2
tQXzJ2z6lWSaT8wCkjUCub5XL6ManPJy2YdN8UsKiWlT8hDjJtyw3SWfTudMQ53r
0kLH8mKxDpERsk8hwB/TSbk0hWwM4l0HiLV6Hw3stpqzNE+750DSHPV/5C7eUbIZ
xfXiVg==
=0LcV
-----END PGP SIGNATURE-----
------6U2JCXSDZKQSLVE05FUH4M8CDP1MWU--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2E8B4BF7-A39E-4B93-820D-FC66F8C2F94B>