Date: Fri, 3 Feb 2023 14:07:42 +0100 From: Mario Marietto <marietto2008@gmail.com> To: "Daniel O'Connor" <darius@dons.net.au> Cc: freebsd-hackers <freebsd-hackers@freebsd.org> Subject: Re: devctl: Failed to detach pci0:1:0:0: Device busy / devctl: Failed to set pci0:1:0:0 driver to ppt: Device busy Message-ID: <CA%2B1FSigVQr=B0FNvgWXHgTgSRWnc8cfo2zJf6azt2P1uLT%2BasQ@mail.gmail.com> In-Reply-To: <128F6D81-D71B-4BC7-9143-8205E826088F@dons.net.au> References: <CA%2B1FSigT376-VQoKM-D96fpUZi5MWDbdNjjeAgdhnWs0vVzN-A@mail.gmail.com> <09A3D05B-E269-437F-8ACB-A150EBCACF4E@dons.net.au> <CA%2B1FSiiA3vn7saY4Xnhh1AwfO54f_pKxpPzB1KoECB_SG7=p6A@mail.gmail.com> <128F6D81-D71B-4BC7-9143-8205E826088F@dons.net.au>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000000fc01705f3cb609f Content-Type: text/plain; charset="UTF-8" Hello Daniel and everyone. ---> If you want to pass through the video card (as evidenced by devctl and pptdevs) then why have the nvidia driver installed? Why did you say this ? Are you aware of some bug that prevents a pci device from being passed if its driver is installed ? Maybe there is. Infact I made some progress. I have commented this line on /etc/rc.conf : #kld_list="nvidia nvidia-modeset" and I've been able to detach and reattach the 4 slots of the 2080 ti. I think that if I load the nvidia modules on the rc.conf,they interfere with a later attaching / detaching of 1 or 2 slots of the gpu (hdac and vgapci). Do you think that it could be caused by a bug on the nvidia driver or in the bhyve source code ? What do you suggest I do ? Il giorno mer 1 feb 2023 alle ore 13:49 Daniel O'Connor <darius@dons.net.au> ha scritto: > > > > On 1 Feb 2023, at 21:41, Mario Marietto <marietto2008@gmail.com> wrote: > > ---> Do you have 'vmm_load="YES"' in loader.conf ? > > > > yes. > > > > ---> If you want to pass through the video card (as evidenced by devctl > and pptdevs) then why have the nvidia driver installed? > > > > because I have 3 gpus on my PC. I use the intel gpu on the host ; I > would like to use the nvidia gtx 1060 within a bhyve Linux vm and the 2080 > ti with the Linuxulator to run stable diffusion,or vice versa. Stable > diffusion needs pytorch + cuda and they need the nVidia driver installed on > FreeBSD and the modules loaded. > > > > My goal is to run stable diffusion and a bhyve / linux vm +my nvidia > gpus attached. > > I note that https://wiki.freebsd.org/bhyve/pci_passthru says "Note: VGA / > GPU pass-through devices are not currently supported" so perhaps that is > why it is ignoring your pptdevs line. > > Although that said some googling suggests that is not the case so I don't > know, unfortunately I haven't tried it myself. > > There are some threads about it on the forums, eg: > https://forums.freebsd.org/threads/bhyve-gpu-pass-through.83152/ > > Those suggest that modifications to bhyve are required before it will work > properly but none of them are for vmm that I can see. > > -- > Daniel O'Connor > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > > -- Mario. --0000000000000fc01705f3cb609f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Hello Daniel and everyone.</div><div><span class=3D"g= mail-im"> <br></span> ---> If you want to pass through the video card (as evidenced by devctl = and pptdevs) then why have the nvidia driver installed?</div><div><br></div= ><div>Why did you say this ? Are you aware of some bug that prevents a pci = device from being passed if its driver is installed ? <br></div><div><br></= div><div>Maybe there is. Infact I made some progress. I have commented this= line on /etc/rc.conf :</div><div><br></div><div><span style=3D"font-family= :arial,sans-serif"><span style=3D"color:rgb(0,0,0);background-color:rgb(255= ,255,255)">#kld_list=3D"nvidia nvidia-modeset"</span></span></div= ><div><span style=3D"font-family:arial,sans-serif"><span style=3D"color:rgb= (0,0,0);background-color:rgb(255,255,255)"><br></span></span></div><div><sp= an style=3D"font-family:arial,sans-serif"><span style=3D"color:rgb(0,0,0);b= ackground-color:rgb(255,255,255)">and I've been able to detach and reattach the 4 slots of the 2080 ti. I=20 think that if I load the nvidia modules on the rc.conf,they interfere=20 with a later attaching / detaching of 1 or 2 slots of the gpu (hdac and=20 vgapci). Do you think that it could be caused by a bug on the nvidia driver= or in the bhyve source code ? What do you suggest I do ?<br></span></span>= </div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_= attr">Il giorno mer 1 feb 2023 alle ore 13:49 Daniel O'Connor <<a hr= ef=3D"mailto:darius@dons.net.au">darius@dons.net.au</a>> ha scritto:<br>= </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left:1px solid rgb(204,204,204);padding-left:1ex"><br> <br> > On 1 Feb 2023, at 21:41, Mario Marietto <<a href=3D"mailto:marietto= 2008@gmail.com" target=3D"_blank">marietto2008@gmail.com</a>> wrote:<br> > ---> Do you have 'vmm_load=3D"YES"' in loader.con= f ?<br> > <br> > yes.<br> > <br> > ---> If you want to pass through the video card (as evidenced by de= vctl and pptdevs) then why have the nvidia driver installed?<br> > <br> > because I have 3 gpus on my PC. I use the intel gpu on the host ; I wo= uld like to use the nvidia gtx 1060 within a bhyve Linux vm and the 2080 ti= with the Linuxulator to run stable diffusion,or vice versa. Stable diffusi= on needs pytorch + cuda and they need the nVidia driver installed on FreeBS= D and the modules loaded.<br> > <br> > My goal is to run stable diffusion and a bhyve / linux vm +my nvidia g= pus attached.<br> <br> I note that <a href=3D"https://wiki.freebsd.org/bhyve/pci_passthru" rel=3D"= noreferrer" target=3D"_blank">https://wiki.freebsd.org/bhyve/pci_passthru</= a> says "Note: VGA / GPU pass-through devices are not currently suppor= ted" so perhaps that is why it is ignoring your pptdevs line.<br> <br> Although that said some googling suggests that is not the case so I don'= ;t know, unfortunately I haven't tried it myself.<br> <br> There are some threads about it on the forums, eg:<br> <a href=3D"https://forums.freebsd.org/threads/bhyve-gpu-pass-through.83152/= " rel=3D"noreferrer" target=3D"_blank">https://forums.freebsd.org/threads/b= hyve-gpu-pass-through.83152/</a><br> <br> Those suggest that modifications to bhyve are required before it will work = properly but none of them are for vmm that I can see.<br> <br> --<br> Daniel O'Connor<br> "The nice thing about standards is that there<br> are so many of them to choose from."<br> -- Andrew Tanenbaum<br> <br> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g= mail_signature">Mario.<br></div> --0000000000000fc01705f3cb609f--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSigVQr=B0FNvgWXHgTgSRWnc8cfo2zJf6azt2P1uLT%2BasQ>