Date: Tue, 27 Sep 2022 14:21:18 +0200 From: Mario Marietto <marietto2008@gmail.com> To: FreeBSD virtualization <freebsd-virtualization@freebsd.org> Subject: My nvidia gpu can't be passed through from Linux (qemu+kvm) to the 2. guest OS (Puppy Linux) that I tried to virtualize on the 1. (FreeBSD 13.1) Message-ID: <CA%2B1FSijOAPCJ6ry2cb_6Vf2UuOmH5r8aoWNSdHZ=ewfSwGWcVg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hello.
I've configured Xubuntu 22.04 so that it can allow me to pass thru my
nvidia gpu on the guest os vms. This time I've chosen freebsd 13.1 as a
guest,because I was curious to see if bhyve supports the passthrough of my
nvidia gpu within another guest os (Puppy Linux) that I have virtualized
with bhyve. This is how I have configured the FreeBSD 13.1 guest vm :
/boot/loader.conf
vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"
bridgestp_load="YES"
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
kern.racct.enable=1
aio_load="YES"
cryptodev_load="YES"
zfs_load="YES"
verbose_loading="YES"
pptdevs="8/0/0 9/0/0 10/0/0 11/0/0"
pciconf -vl says that pptdevs reserved correctly the addresses 8/0/0 9/0/0
10/0/0 11/0/0 which belong to the nvidia GPU :
ppt0@pci0:8:0:0: class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1e04 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 [GeForce RTX 2080 Ti]'
class = display
subclass = VGA
ppt1@pci0:9:0:0: class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x10f7 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 High Definition Audio Controller'
class = multimedia
subclass = HDA
ppt2@pci0:10:0:0: class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1ad6 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 USB 3.1 Host Controller'
class = serial bus
subclass = USB
ppt3@pci0:11:0:0: class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1ad7 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 USB Type-C UCSI Controller'
At this point,inside FreeBSD 13.1 guest OS,I tried to virtualize another
OS,the puppy Linux :
bhyve -S -c sockets=1,cores=1,threads=1 -m 2G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Desktop/bhyve/Files/fossapup64-9.5.iso,bootindex=1 \
-s 2,virtio-blk,/home/marietto/Desktop/bhyve/Files/puppy.img,bootindex=2 \
-s 8:0,passthru,8/0/0 \
-s 8:1,passthru,9/0/0 \
-s 8:2,passthru,10/0/0 \
-s 8:3,passthru,11/0/0 \
-s 10,virtio-net,tap18 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=0.0.0.0:5918,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18
unfortunately it gives this error :
*bhyve: PCI device at 8/0/0 is not using the ppt(4) driverdevice emulation
initialization error: Device busy*
I tried to remove the slots related to the passthru of the gpu and puppy
booted like a charm,like this one :
bhyve -S -c sockets=1,cores=1,threads=1 -m 2G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Desktop/bhyve/Files/fossapup64-9.5.iso,bootindex=1 \
-s 2,virtio-blk,/home/marietto/Desktop/bhyve/Files/puppy.img,bootindex=2 \
-s 10,virtio-net,tap18 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=0.0.0.0:5918,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18
So,where could be the error in this specific scenario ? Nested vm works,but
I can't pass through the gpu,even if the host OS (xubuntu) makes it
available to the guest.
--
Mario.
[-- Attachment #2 --]
<div dir="ltr"><div class="gmail-_3xX726aBn29LDbsDtzr_6E gmail-_1Ap4F5maDtT1E1YuCiaO0r gmail-D3IL3FD0RFy_mkKLPwL4" style="max-width:800px"><div class="gmail-_292iotee39Lmt0MkQZ2hPV gmail-RichTextJSON-root"><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">Hello.</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">I've
configured Xubuntu 22.04 so that it can allow me to pass thru my
nvidia gpu on the guest os vms. This time I've chosen freebsd 13.1 as a
guest,because I was curious to see if bhyve supports the passthrough of
my nvidia gpu within another guest os (Puppy Linux) that I have
virtualized with bhyve. This is how I have configured the FreeBSD 13.1
guest vm :</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">/boot/loader.conf</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><pre class="gmail-_3GnarIQX9tD_qsgXkfSDz1"><code class="gmail-_34q3PgLsx9zIU5BiSOjFoM">vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"
bridgestp_load="YES"
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
kern.racct.enable=1
aio_load="YES"
cryptodev_load="YES"
zfs_load="YES"
verbose_loading="YES"
pptdevs="8/0/0 9/0/0 10/0/0 11/0/0"
</code></pre><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">pciconf -vl says that pptdevs reserved correctly the addresses 8/0/0 9/0/0 10/0/0 11/0/0 which belong to the nvidia GPU :</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><pre class="gmail-_3GnarIQX9tD_qsgXkfSDz1"><code class="gmail-_34q3PgLsx9zIU5BiSOjFoM">ppt0@pci0:8:0:0: class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1e04 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 [GeForce RTX 2080 Ti]'
class = display
subclass = VGA
ppt1@pci0:9:0:0: class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f7 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 High Definition Audio Controller'
class = multimedia
subclass = HDA
ppt2@pci0:10:0:0: class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad6 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 USB 3.1 Host Controller'
class = serial bus
subclass = USB
ppt3@pci0:11:0:0: class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad7 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 USB Type-C UCSI Controller'
</code></pre><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">At this point,inside FreeBSD 13.1 guest OS,I tried to virtualize another OS,the puppy Linux :</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><pre class="gmail-_3GnarIQX9tD_qsgXkfSDz1"><code class="gmail-_34q3PgLsx9zIU5BiSOjFoM">bhyve -S -c sockets=1,cores=1,threads=1 -m 2G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Desktop/bhyve/Files/fossapup64-9.5.iso,bootindex=1 \
-s 2,virtio-blk,/home/marietto/Desktop/bhyve/Files/puppy.img,bootindex=2 \
-s 8:0,passthru,8/0/0 \
-s 8:1,passthru,9/0/0 \
-s 8:2,passthru,10/0/0 \
-s 8:3,passthru,11/0/0 \
-s 10,virtio-net,tap18 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=<a href="http://0.0.0.0:5918">0.0.0.0:5918</a>,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18
</code></pre><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">unfortunately it gives this error :</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><strong class="gmail-_12FoOEddL7j_RgMQN0SNeU">bhyve: PCI device at 8/0/0 is not using the ppt(4) driverdevice emulation initialization error: Device busy</strong></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><strong class="gmail-_12FoOEddL7j_RgMQN0SNeU"><br></strong></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">I tried to remove the slots related to the passthru of the gpu and puppy booted like a charm,like this one :</p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><pre class="gmail-_3GnarIQX9tD_qsgXkfSDz1"><code class="gmail-_34q3PgLsx9zIU5BiSOjFoM">bhyve -S -c sockets=1,cores=1,threads=1 -m 2G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Desktop/bhyve/Files/fossapup64-9.5.iso,bootindex=1 \
-s 2,virtio-blk,/home/marietto/Desktop/bhyve/Files/puppy.img,bootindex=2 \
-s 10,virtio-net,tap18 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=<a href="http://0.0.0.0:5918">0.0.0.0:5918</a>,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18
</code></pre><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM"><br></p><p class="gmail-_1qeIAgB0cPwnLhDF9XSiJM">So,where
could be the error in this specific scenario ? Nested vm works,but I
can't pass through the gpu,even if the host OS (xubuntu) makes it
available to the guest.</p></div></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Mario.<br></div></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSijOAPCJ6ry2cb_6Vf2UuOmH5r8aoWNSdHZ=ewfSwGWcVg>