Date: Sat, 3 Dec 2022 19:14:29 +0100 From: Mario Marietto <marietto2008@gmail.com> To: =?UTF-8?Q?Corvin_K=C3=B6hne?= <corvink@freebsd.org> Cc: virtualization@freebsd.org, freebsd-x11+help@freebsd.org, FreeBSD virtualization <freebsd-virtualization@freebsd.org>, hackers@freebsd.org Subject: Re: How to use the framebuffer as primary video device instead of the nvidia passed-through graphic card in a bhyve/linux vm Message-ID: <CA%2B1FSig_TNz6iVaVAEAJtUuXu6QuCGM_-oatDEje4yUsE4yRkw@mail.gmail.com> In-Reply-To: <2E8B4BF7-A39E-4B93-820D-FC66F8C2F94B@FreeBSD.org> References: <CA%2B1FSigGi%2BY-pqS9ARwY%2BdpRS2HBc-wxc0cd5ixKTAw%2Ba6uNZw@mail.gmail.com> <2E8B4BF7-A39E-4B93-820D-FC66F8C2F94B@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
ok. I tried like this,but it didn't work :
bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2 \-s 7,fbuf,tcp=0.0.0.0:5919,w=1600,h=950,wait \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:19 < /dev/null & sleep 2 && vncviewer 0:19
I tried specifying the bus ID of the framebuffer and I have used this
xorg.conf file :
Section "Files"
ModulePath "/usr/lib/xorg/modules"
FontPath "/usr/share/fonts/X11/misc"
FontPath "/usr/share/fonts/X11/cyrillic"
FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
FontPath "/usr/share/fonts/X11/Type1"
FontPath "/usr/share/fonts/X11/100dpi"
FontPath "/usr/share/fonts/X11/75dpi"
FontPath "built-ins"
EndSection
Section "Module"
Load "vnc"
Load "glx"
EndSection
Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
EndSection
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection
Section "Monitor"
Identifier "Monitor0"
VendorName "Monitor Vendor"
ModelName "Monitor Model"
EndSection
Section "Device"
Identifier "Card0"
Driver "modesetting"
BusID "PCI:0:7:0"
EndSection
Section "Screen"
Identifier "Screen0"
Device "Card0"
Monitor "Monitor0"
SubSection "Display"
Viewport 0 0
Depth 1
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 4
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 8
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 15
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 16
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 24
EndSubSection
EndSection
The error reported has been :
https://ibb.co/1KX2h26
https://ibb.co/Cv5FffB
thanks.
Il giorno sab 3 dic 2022 alle ore 17:34 Corvin Köhne <corvink@freebsd.org>
ha scritto:
> On December 3, 2022 4:49:46 PM GMT+01:00, Mario Marietto <
> marietto2008@gmail.com> wrote:
>>
>> Hello to everyone.
>>
>> what Im trying to do is to set the framebuffer video adapter as primary
>> graphic card on my bhyve-ubuntu vm instead of the nvidia RTX 2080 ti card
>> that I have passed through. What I want to do really is to use both the
>> graphic adapters,but the primary should be the framebuffer and the
>> secondary the nvidia. I tried different Xorg configurations,but what I've
>> got is that Xorg failed to display some errors. So,the controller that you
>> see below should be used as primary inside the ubuntu vm :
>>
>> 00:1d.0 VGA compatible controller: Device fb5d:40fb
>>
>> while the ones you see below as secondary :
>>
>> 08:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
>> 08:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
>> 08:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
>> 08:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)
>>
>>
>> The script that I use to launch the vm is the following :
>>
>> #!/bin/sh
>> setxkbmap it
>> vms="$(ls /dev/vmm/*)"
>> vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
>>
>> for vm in $vms; do
>> session="${vm##*/}"
>> echo "bhyve session = $session"
>> echo "vnc session = $vncs"
>> if ! printf '%s\n' "${vncs}" | grep "${session}"; then
>> printf 'VNC session not found,destroying ghost vms\n'
>> bhyvectl --vm=$session --destroy
>> else
>> printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them"
>> fi
>> done
>>
>> vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2015020204055E)/ && d{print d}'`
>> echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"
>>
>> mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
>>
>> bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
>> -s 0,hostbridge \
>> -s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
>> -s 3,virtio-blk,/dev/$vmdisk4 \
>> -s 4,virtio-blk,/dev/$vmdisk2 \
>> -s 8:0,passthru,2/0/0 \
>> -s 8:1,passthru,2/0/1 \
>> -s 8:2,passthru,2/0/2 \
>> -s 8:3,passthru,2/0/3 \
>> -s 10,virtio-net,tap19 \
>> -s 11,virtio-9p,sharename=/ \
>> -s 29,fbuf,tcp=0.0.0.0:5919,w=1600,h=950,wait \
>> -s 30,xhci,tablet \
>> -s 31,lpc \
>> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
>> vm0:19 < /dev/null & sleep 2 && vncviewer 0:19
>>
>> For sure ,on /boot/loader.conf I've added :
>>
>> /boot/loader.conf
>>
>> pptdevs="2/0/0 2/0/1 2/0/2 2/0/3"
>>
>> As I said before,I tried two xorg conf files to achieve the goal. On the
>> first one I tried to add only the framebuffer,like this :
>>
>>
>> Section "Files"
>> ModulePath "/usr/lib/xorg/modules"
>> FontPath "/usr/share/fonts/X11/misc"
>> FontPath "/usr/share/fonts/X11/cyrillic"
>> FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
>> FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
>> FontPath "/usr/share/fonts/X11/Type1"
>> FontPath "/usr/share/fonts/X11/100dpi"
>> FontPath "/usr/share/fonts/X11/75dpi"
>> FontPath "built-ins"
>> EndSection
>>
>> Section "Module"
>> Load "vnc"
>> Load "glx"
>> EndSection
>>
>>
>> Section "InputDevice"
>> Identifier "Keyboard0"
>> Driver "kbd"
>> EndSection
>>
>> Section "InputDevice"
>> Identifier "Mouse0"
>> Driver "mouse"
>> Option "Protocol" "auto"
>> Option "Device" "/dev/input/mice"
>> Option "ZAxisMapping" "4 5 6 7"
>>
>> EndSection
>>
>> Section "Monitor"
>> Identifier "Monitor0"
>> VendorName "Monitor Vendor"
>> ModelName "Monitor Model"
>> EndSection
>>
>> Section "Device"
>> Identifier "Card0"
>> Driver "modesetting"
>> BusID "PCI:0:29:0"
>>
>> EndSection
>>
>> Section "Screen"
>> Identifier "Screen0"
>> Device "Card0"
>> Monitor "Monitor0"
>> SubSection "Display"
>> Viewport 0 0
>> Depth 1
>> EndSubSection
>> SubSection "Display"
>> Viewport 0 0
>> Depth 4
>> EndSubSection
>> SubSection "Display"
>> Viewport 0 0
>> Depth 8
>> EndSubSection
>> SubSection "Display"
>> Viewport 0 0
>> Depth 15
>> EndSubSection
>> SubSection "Display"
>> Viewport 0 0
>> Depth 16
>> EndSubSection
>> SubSection "Display"
>> Viewport 0 0
>> Depth 24
>> EndSubSection
>> EndSection
>>
>>
>> but it didn't work. This is the log file that shows the errors reported : https://pastebin.ubuntu.com/p/Gv7wgsDR4K/
>> I have also removed the xorg.conf file,but it didn't work either. This is the log file that shows the errors reported :
>>
>>
>> https://pastebin.ubuntu.com/p/wNcfQTByQm/
>>
>> Can someone give me some suggestions that can help me to understand where the mistake is,please,thanks.
>>
>> --
>> Mario.
>>
>
> Try to assign a lower pci slot number to the framebuffer device than to
> the nvidia gpu in your bhyve command.
> --
> Best regards,
> Corvin
>
--
Mario.
[-- Attachment #2 --]
<div dir="ltr"><div>ok. I tried like this,but it didn't work :</div><div><pre dir="ltr"><code>bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2 \
<code>-s 7,fbuf,tcp=<a href="http://0.0.0.0:5919" target="_blank">0.0.0.0:5919</a>,w=1600,h=950,wait \<br></code>-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:19 < /dev/null & sleep 2 && vncviewer 0:19<br><br></code></pre></div><div>I tried specifying the bus ID of the framebuffer and I have used this xorg.conf file :</div><div><br></div><div><pre dir="ltr"><code>Section "Files"
ModulePath "/usr/lib/xorg/modules"
FontPath "/usr/share/fonts/X11/misc"
FontPath "/usr/share/fonts/X11/cyrillic"
FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
FontPath "/usr/share/fonts/X11/Type1"
FontPath "/usr/share/fonts/X11/100dpi"
FontPath "/usr/share/fonts/X11/75dpi"
FontPath "built-ins"
EndSection
Section "Module"
Load "vnc"
Load "glx"
EndSection
Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
EndSection
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection
Section "Monitor"
Identifier "Monitor0"
VendorName "Monitor Vendor"
ModelName "Monitor Model"
EndSection
Section "Device"
Identifier "Card0"
Driver "modesetting"
BusID "PCI:0:7:0"
EndSection
Section "Screen"
Identifier "Screen0"
Device "Card0"
Monitor "Monitor0"
SubSection "Display"
Viewport 0 0
Depth 1
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 4
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 8
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 15
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 16
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 24
EndSubSection
EndSection<br></code></pre>The error reported has been :</div><div><br></div><div><a href="https://ibb.co/1KX2h26">https://ibb.co/1KX2h26</a><br><a href="https://ibb.co/Cv5FffB">https://ibb.co/Cv5FffB</a></div><div><br></div><div>thanks. <br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno sab 3 dic 2022 alle ore 17:34 Corvin Köhne <<a href="mailto:corvink@freebsd.org">corvink@freebsd.org</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail_quote">On December 3, 2022 4:49:46 PM GMT+01:00, Mario Marietto <<a href="mailto:marietto2008@gmail.com" target="_blank">marietto2008@gmail.com</a>> wrote:<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hello to everyone.<br>
<br>
what Im trying to do is to set the framebuffer video adapter as primary
graphic card on my bhyve-ubuntu vm instead of the nvidia RTX 2080 ti
card that I have passed through. What I want to do really is to use both
the graphic adapters,but the primary should be the framebuffer and the
secondary the nvidia. I tried different Xorg configurations,but what I've
got is that Xorg failed to display some errors. So,the controller
that you see below should be used as primary inside the ubuntu vm :<br><div><br></div><div><div dir="ltr">
<pre dir="ltr"><code>00:1d.0 VGA compatible controller: Device fb5d:40fb<br><br></code></pre>
</div><div dir="ltr">
while the ones you see below as secondary :</div><div dir="ltr"><br></div></div><div><pre dir="ltr"><code>08:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
08:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
08:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
08:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)</code></pre></div><div><br></div><div>
The script that I use to launch the vm is the following :</div><div><br></div><div><pre dir="ltr"><code>#!/bin/sh
setxkbmap it
vms="$(ls /dev/vmm/*)"
vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
for vm in $vms; do
session="${vm##*/}"
echo "bhyve session = $session"
echo "vnc session = $vncs"
if ! printf '%s\n' "${vncs}" | grep "${session}"; then
printf 'VNC session not found,destroying ghost vms\n'
bhyvectl --vm=$session --destroy
else
printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them"
fi
done
vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2015020204055E)/ && d{print d}'`
echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"
mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2 \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=<a href="http://0.0.0.0:5919" target="_blank">0.0.0.0:5919</a>,w=1600,h=950,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:19 < /dev/null & sleep 2 && vncviewer 0:19<br><br>
<span style="font-family:arial black,sans-serif">For sure ,on /boot/loader.conf I've added :<br><br></span></code></pre></div><div><pre dir="ltr"><code>/boot/loader.conf
pptdevs="2/0/0 2/0/1 2/0/2 2/0/3"<br><br>
<span style="font-family:arial black,sans-serif">As I said before,I tried two xorg conf files to achieve the goal. On the
first one I tried to add only the framebuffer,like this :</span></code></pre></div><div><pre dir="ltr"><code><br>Section "Files"
ModulePath "/usr/lib/xorg/modules"
FontPath "/usr/share/fonts/X11/misc"
FontPath "/usr/share/fonts/X11/cyrillic"
FontPath "/usr/share/fonts/X11/100dpi/:unscaled"
FontPath "/usr/share/fonts/X11/75dpi/:unscaled"
FontPath "/usr/share/fonts/X11/Type1"
FontPath "/usr/share/fonts/X11/100dpi"
FontPath "/usr/share/fonts/X11/75dpi"
FontPath "built-ins"
EndSection
Section "Module"
Load "vnc"
Load "glx"
EndSection
Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
EndSection
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "ZAxisMapping" "4 5 6 7"
EndSection
Section "Monitor"
Identifier "Monitor0"
VendorName "Monitor Vendor"
ModelName "Monitor Model"
EndSection
Section "Device"
Identifier "Card0"
Driver "modesetting"
BusID "PCI:0:29:0"
EndSection
Section "Screen"
Identifier "Screen0"
Device "Card0"
Monitor "Monitor0"
SubSection "Display"
Viewport 0 0
Depth 1
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 4
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 8
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 15
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 16
EndSubSection
SubSection "Display"
Viewport 0 0
Depth 24
EndSubSection
EndSection<br><br><br><span style="font-family:arial black,sans-serif">but it didn't work. This is the log file that shows the errors reported : <a href="https://pastebin.ubuntu.com/p/Gv7wgsDR4K/" rel="nofollow ugc noopener" target="_blank">https://pastebin.ubuntu.com/p/Gv7wgsDR4K/</a></span><br><span style="font-family:arial black,sans-serif">I have also removed the xorg.conf file,but it didn't work either. This is the log file that shows the errors reported :</span><br><br><br><a href="https://pastebin.ubuntu.com/p/wNcfQTByQm/" target="_blank">https://pastebin.ubuntu.com/p/wNcfQTByQm/</a><br><br>;
<span style="font-family:arial black,sans-serif">Can someone give me some suggestions that can help me to understand where the mistake is,please,thanks.</span>
</code></pre></div><div>-- </div><div dir="ltr">Mario.<br></div></div>
</blockquote></div><br clear="all">Try to assign a lower pci slot number to the framebuffer device than to the nvidia gpu in your bhyve command.<br><div style="white-space:pre-wrap"><div>-- <br>Best regards,<br>Corvin</div></div></div></blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature">Mario.<br></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSig_TNz6iVaVAEAJtUuXu6QuCGM_-oatDEje4yUsE4yRkw>