Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 1 Feb 2023 12:11:18 +0100
From:      Mario Marietto <marietto2008@gmail.com>
To:        "Daniel O'Connor" <darius@dons.net.au>
Cc:        freebsd-hackers <freebsd-hackers@freebsd.org>
Subject:   Re: devctl: Failed to detach pci0:1:0:0: Device busy / devctl: Failed to set pci0:1:0:0 driver to ppt: Device busy
Message-ID:  <CA%2B1FSiiA3vn7saY4Xnhh1AwfO54f_pKxpPzB1KoECB_SG7=p6A@mail.gmail.com>
In-Reply-To: <09A3D05B-E269-437F-8ACB-A150EBCACF4E@dons.net.au>
References:  <CA%2B1FSigT376-VQoKM-D96fpUZi5MWDbdNjjeAgdhnWs0vVzN-A@mail.gmail.com> <09A3D05B-E269-437F-8ACB-A150EBCACF4E@dons.net.au>

next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000001b03b505f3a184aa
Content-Type: text/plain; charset="UTF-8"

---> Do you have 'vmm_load="YES"' in loader.conf ?

yes.

---> If you want to pass through the video card (as evidenced by devctl and
pptdevs) then why have the nvidia driver installed?

because I have 3 gpus on my PC. I use the intel gpu on the host ; I would
like to use the nvidia gtx 1060 within a bhyve Linux vm and the 2080 ti
with the Linuxulator to run stable diffusion,or vice versa. Stable
diffusion needs pytorch + cuda and they need the nVidia driver installed on
FreeBSD and the modules loaded.

My goal is to run stable diffusion and a bhyve / linux vm +my nvidia gpus
attached.

Thanks.

Il giorno mer 1 feb 2023 alle ore 07:24 Daniel O'Connor <darius@dons.net.au>
ha scritto:

>
>
> > On 30 Jan 2023, at 05:37, Mario Marietto <marietto2008@gmail.com> wrote:
> > In some FreeBSD 13.1 machines I have the problem below,in some others I
> don't have it. I would like to know what the causes could be and how to fix
> it.
> >
> > root@marietto:/usr/home/marietto # devctl detach pci0:1:0:0
> >
> > devctl: Failed to detach pci0:1:0:0: Device busy
> >
> > root@marietto:/usr/home/marietto # devctl detach pci0:2:0:0
> >
> > devctl: Failed to detach pci0:2:0:0: Device busy
> >
> > Not even it works if instead of detach them, I try to attach them
> directly to the ppt driver :
> >
> > root@marietto:/usr/home/marietto/bhyve/Files # devctl set driver
> pci0:2:0:0 ppt
> > devctl: Failed to set pci0:2:0:0 driver to ppt: Device busy
> >
> > root@marietto:/usr/home/marietto/bhyve/Files # devctl detach pci0:2:0:0
> > devctl: Failed to detach pci0:2:0:0: Device busy
> >
> > Pci addresses 1:0:0 and 2:0:0 belong to the two GPUs that I have on my
> PC :
> >
> > vgapci0@pci0:1:0:0:     class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de
> device=0x1c02 subvendor=0x19da subdevice=0x2438
> > vgapci1@pci0:2:0:0:     class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de
> device=0x1e04 subvendor=0x19da subdevice=0x2503
> >
> > Actually I have commented this line on /boot/loader.conf,because it
> makes no difference if I keep it uncommented or not. It is totally ignored.
> >
> > #pptdevs="1/0/0 1/0/1 2/0/0 2/0/1 2/0/2 2/0/3"
>
> Do you have 'vmm_load="YES"' in loader.conf?
>
> If not I am not sure it will have an effect - vmm must be loaded early
> otherwise the card will be grabbed by the vgapci driver rather than be put
> under ppt control.
>
> > In every FreeBSD machine I load the nvidia kernel modules adding the
> line below to /etc/rc.conf :
> >
> > kld_list="nvidia nvidia-modeset"
> >
> > and I have installed the nvidia-driver package on every machine. But as
> I said,in some of them I see the error above,in some others I don't see it.
> I'm not able to isolate the dysfunctional pattern. The xorg.conf file is
> the same for each machine. Can someone help me to troubleshoot the error ?
> thanks.
>
> If you want to pass through the video card (as evidenced by devctl and
> pptdevs) then why have the nvidia driver installed?
>
> Perhaps explain what you are trying to achieve rather than asking for help
> on how to sold what you think the problem is.
>
> PS I have trimmed the CCs as that seems impolite.
>
> --
> Daniel O'Connor
> "The nice thing about standards is that there
> are so many of them to choose from."
> -- Andrew Tanenbaum
>
>

-- 
Mario.

--0000000000001b03b505f3a184aa
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div>---&gt; Do you have &#39;vmm_load=3D=
&quot;YES&quot;&#39; in loader.conf ?</div><div><br></div><div>yes.<br></di=
v></div><div><span><br>
---&gt; </span>If you want to pass through the video card (as evidenced by =
devctl and pptdevs) then why have the nvidia driver installed?</div><div><b=
r></div><div>because I have 3 gpus on my PC. I use the intel gpu on the hos=
t ; I would like to use the nvidia gtx 1060 within a bhyve Linux vm and the=
 2080 ti with the Linuxulator to run stable diffusion,or vice versa. Stable=
 diffusion needs pytorch + cuda and they need the nVidia driver installed o=
n FreeBSD and the modules loaded.<br></div><div><br></div><div>My goal is t=
o run stable diffusion and a bhyve / linux vm +my nvidia gpus attached.<br>=
</div><div><br></div><div>Thanks.</div><div><br></div><div class=3D"gmail_q=
uote"><div dir=3D"ltr" class=3D"gmail_attr">Il giorno mer 1 feb 2023 alle o=
re 07:24 Daniel O&#39;Connor &lt;<a href=3D"mailto:darius@dons.net.au" targ=
et=3D"_blank">darius@dons.net.au</a>&gt; ha scritto:<br></div><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px soli=
d rgb(204,204,204);padding-left:1ex"><br>
<br>
&gt; On 30 Jan 2023, at 05:37, Mario Marietto &lt;<a href=3D"mailto:mariett=
o2008@gmail.com" target=3D"_blank">marietto2008@gmail.com</a>&gt; wrote:<br=
>
&gt; In some FreeBSD 13.1 machines I have the problem below,in some others =
I don&#39;t have it. I would like to know what the causes could be and how =
to fix it. <br>
&gt; <br>
&gt; root@marietto:/usr/home/marietto # devctl detach pci0:1:0:0 <br>
&gt; <br>
&gt; devctl: Failed to detach pci0:1:0:0: Device busy<br>
&gt; <br>
&gt; root@marietto:/usr/home/marietto # devctl detach pci0:2:0:0 <br>
&gt; <br>
&gt; devctl: Failed to detach pci0:2:0:0: Device busy <br>
&gt; <br>
&gt; Not even it works if instead of detach them, I try to attach them dire=
ctly to the ppt driver :<br>
&gt; <br>
&gt; root@marietto:/usr/home/marietto/bhyve/Files # devctl set driver pci0:=
2:0:0 ppt <br>
&gt; devctl: Failed to set pci0:2:0:0 driver to ppt: Device busy<br>
&gt;=C2=A0 <br>
&gt; root@marietto:/usr/home/marietto/bhyve/Files # devctl detach pci0:2:0:=
0 <br>
&gt; devctl: Failed to detach pci0:2:0:0: Device busy<br>
&gt; <br>
&gt; Pci addresses 1:0:0 and 2:0:0 belong to the two GPUs that I have on my=
 PC :<br>
&gt; <br>
&gt; vgapci0@pci0:1:0:0:=C2=A0 =C2=A0 =C2=A0class=3D0x030000 rev=3D0xa1 hdr=
=3D0x00 vendor=3D0x10de device=3D0x1c02 subvendor=3D0x19da subdevice=3D0x24=
38 <br>
&gt; vgapci1@pci0:2:0:0:=C2=A0 =C2=A0 =C2=A0class=3D0x030000 rev=3D0xa1 hdr=
=3D0x00 vendor=3D0x10de device=3D0x1e04 subvendor=3D0x19da subdevice=3D0x25=
03<br>
&gt; <br>
&gt; Actually I have commented this line on /boot/loader.conf,because it ma=
kes no difference if I keep it uncommented or not. It is totally ignored.<b=
r>
&gt; <br>
&gt; #pptdevs=3D&quot;1/0/0 1/0/1 2/0/0 2/0/1 2/0/2 2/0/3&quot; <br>
<br>
Do you have &#39;vmm_load=3D&quot;YES&quot;&#39; in loader.conf?<br>
<br>
If not I am not sure it will have an effect - vmm must be loaded early othe=
rwise the card will be grabbed by the vgapci driver rather than be put unde=
r ppt control.<br>
<br>
&gt; In every FreeBSD machine I load the nvidia kernel modules adding the l=
ine below to /etc/rc.conf :<br>
&gt; <br>
&gt; kld_list=3D&quot;nvidia nvidia-modeset&quot;<br>
&gt; <br>
&gt; and I have installed the nvidia-driver package on every machine. But a=
s I said,in some of them I see the error above,in some others I don&#39;t s=
ee it. I&#39;m not able to isolate the dysfunctional pattern. The xorg.conf=
 file is the same for each machine. Can someone help me to troubleshoot the=
 error ? thanks. <br>
<br>
If you want to pass through the video card (as evidenced by devctl and pptd=
evs) then why have the nvidia driver installed?<br>
<br>
Perhaps explain what you are trying to achieve rather than asking for help =
on how to sold what you think the problem is.<br>
<br>
PS I have trimmed the CCs as that seems impolite.<br>
<br>
--<br>
Daniel O&#39;Connor<br>
&quot;The nice thing about standards is that there<br>
are so many of them to choose from.&quot;<br>
-- Andrew Tanenbaum<br>
<br>
</blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr">Mario.<br>=
</div></div>

--0000000000001b03b505f3a184aa--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSiiA3vn7saY4Xnhh1AwfO54f_pKxpPzB1KoECB_SG7=p6A>