Date: Sat, 12 Aug 2023 12:22:33 +0200 From: Mario Marietto <marietto2008@gmail.com> To: Odhiambo Washington <odhiambo@gmail.com> Cc: freebsd-virtualization@freebsd.org Subject: Re: Starting the bhyve journey Message-ID: <CA%2B1FSijNC_Cvd3%2B=BvQRZqxVh2mVqmzdpg4zdcbb-OTJ3Tip9g@mail.gmail.com> In-Reply-To: <CAAdA2WOsT%2Bb20F=eoXQ4Xw9FBvTMxsEHCWYsv38QSwSvNRE2gg@mail.gmail.com> References: <CAAdA2WPwjJV=qDyx_NEi_CFYgvfKz6FjySBb9PrfMYYa%2BBpFCw@mail.gmail.com> <CA%2B1FSij=nJ7CE3R4MJ5-kRqcZL3-DqLixrbr%2BRPm7Sm7ZrmOMw@mail.gmail.com> <CA%2B1FSijyg=t4rwjQq%2B1pR7FBYe7Y8p0BbBZDNYnM7hOaq4w2WA@mail.gmail.com> <CAAdA2WOsT%2Bb20F=eoXQ4Xw9FBvTMxsEHCWYsv38QSwSvNRE2gg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000438f440602b73757 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable As I had suspected,the first tutorial wants you learn vm-bhyve and iohyve (the wrapper for zfs and bhyve),as he says : *I=E2=80=99m not saying this is the way to go*. Really, really, If you are = planning on setting things up from scratch to have a nice FreeBSD virtual machines server, look into all the cool projects out there like vm-bhyve <https://github.com/churchers/vm-bhyve>; or iohyve <https://github.com/pr1ntf/iohyve>; (to name two of them, I=E2=80=99m sure t= here are more). That's not good. If you want to set up from scratch you should not focus too much on studying the vm-bhyve or iohyve,it makes things even more complicated because these are tools that change themself during the time so you should keep updated with more knowledge. Take a look at this tutorial : https://www.cyberciti.biz/faq/how-to-install-linux-vm-on-freebsd-using-bhyv= e-and-zfs/ On Sat, Aug 12, 2023 at 12:09=E2=80=AFPM Odhiambo Washington <odhiambo@gmai= l.com> wrote: > I have read https://wu.e-shell.org/articles/hw-to-bhyve.html as well as > https://www.netscylla.com/blog/2021/06/11/FreeBSD-and-Bhyve.html and many > more. > I will just master it with time. > > On Sat, Aug 12, 2023 at 1:01=E2=80=AFPM Mario Marietto <marietto2008@gmai= l.com> > wrote: > >> Try like this : >> >> bhyve -S -c sockets=3D2,cores=3D2,threads=3D2 -m 4G -w -H -A \ >> -s 0,hostbridge \ >> -s 1,ahci-hd,/mnt/zroot2/zroot2/bhyve/img/Windows/windows11.img,bootinde= x=3D1 \ >> -s 13,virtio-net,tap1 \ >> -s 29,fbuf,tcp=3D0.0.0.0:5918,w=3D1600,h=3D950,wait \ >> -s 30,xhci,tablet \ >> -s 31,lpc \ >> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ >> vm0:1 < /dev/null & sleep 2 && vncviewer 0:1 >> >> Doing this way you will learn better how to compose the bhyve parameters= and how they should be formatted naturally. >> As you can see,the vm-bhyve wrapper "muddies the waters". And I think th= at you should also learn how to create a bridge on FreeBSD. >> >> >> On Sat, Aug 12, 2023 at 11:46=E2=80=AFAM Mario Marietto <marietto2008@gm= ail.com> >> wrote: >> >>> In my opinion,the usage of the wrapper creates some difficulties to >>> understand how bhyve and the networking works. You should start from th= e >>> very beginning,studying bhyve and the freebsd networking,instead of >>> learning how to correctly use the wrapper. >>> >>> On Sat, Aug 12, 2023 at 11:25=E2=80=AFAM Odhiambo Washington <odhiambo@= gmail.com> >>> wrote: >>> >>>> Hello all. >>>> >>>> I am new to bhyve and have read a lot about it, so I decided to try >>>> creating a few VMs. >>>> I found a HOWTO that is detailed and which appears to be the easiest t= o >>>> follow: >>>> https://klarasystems.com/articles/from-0-to-bhyve-on-freebsd-13-1/ >>>> The only differences are that I did not install a fresh FreeBSD-13 >>>> server as I already had one, and I am not using ZFS on my server (host= ). >>>> I have gotten stuck midway and cannot figure out why: I cannot boot th= e >>>> VMs I create. >>>> I have tried creating a Windows2019 as well as a Debian12 VM, but both >>>> don't boot. >>>> >>>> vm-public: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric= 0 >>>> mtu 1492 >>>> ether ce:6f:18:51:d4:51 >>>> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >>>> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 >>>> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >>>> member: em1 flags=3D143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >>>> ifmaxaddr 0 port 2 priority 128 path cost 20000 >>>> groups: bridge vm-switch viid-4c918@ >>>> nd6 options=3D9<PERFORMNUD,IFDISABLED> >>>> tap0: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> met= ric >>>> 0 mtu 1492 >>>> description: vmnet/debian12/0/public >>>> options=3D80000<LINKSTATE> >>>> ether 58:9c:fc:10:ff:ed >>>> inet6 fe80::5a9c:fcff:fe10:ffed%tap0 prefixlen 64 scopeid 0x6 >>>> groups: tap vm-port >>>> media: Ethernet autoselect >>>> status: no carrier >>>> nd6 options=3D21<PERFORMNUD,AUTO_LINKLOCAL> >>>> >>>> 1. Windows2019 VM: >>>> vm create -t uefi -s 100G -m 8G -c8 windows2019 >>>> vm install windows2019 /usr/local/bhyve-vms/.iso/server2019.iso >>>> >>>> 2. Debian12 VM: >>>> vm create -s 60G -m 8G -c 8 debian12 >>>> vm install debian12 .iso/debian-12.0.0-amd64-DVD-1.iso >>>> >>>> root@gw:/usr/local/bhyve-vms # vm list >>>> NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE >>>> debian12 default bhyveload 8 8G >>>> - Yes[2] Bootloader (91866) >>>> windows2019 default uefi 8 8G >>>> - Yes [1] Stopped >>>> >>>> From the above output of `vm list`, you can see that none of them is >>>> booting up. >>>> >>>> What is it that I could be missing? >>>> >>>> >>>> -- >>>> Best regards, >>>> Odhiambo WASHINGTON, >>>> Nairobi,KE >>>> +254 7 3200 0004/+254 7 2274 3223 >>>> "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) >>>> [How to ask smart questions: >>>> http://www.catb.org/~esr/faqs/smart-questions.html] >>>> >>> >>> >>> -- >>> Mario. >>> >> >> >> -- >> Mario. >> > > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) > [How to ask smart questions: > http://www.catb.org/~esr/faqs/smart-questions.html] > --=20 Mario. --000000000000438f440602b73757 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>As I had suspected,the first tutorial wants you learn= vm-bhyve and iohyve (the wrapper for zfs and bhyve),as he says : <br></div= ><div><br></div><div><p><b>I=E2=80=99m not saying this is the way to go</b>= . Really, really, If you are planning on setting things up from scratch to have a nice FreeBSD virtual machines server, look into all the cool projects out there like <a href=3D"= https://github.com/churchers/vm-bhyve" target=3D"_blank">vm-bhyve</a> or <a href=3D"https://github.com/pr1ntf/iohyve" target=3D"_blank">iohyve</a= > (to name two of them, I=E2=80=99m sure there are more).</p><p><br></p><p>= That's not good. If you want to set up from scratch you should not focu= s too much on studying the vm-bhyve or iohyve,it makes things even more com= plicated because these are tools that change themself during the time so yo= u should keep updated with more knowledge. <br></p><p></p><p>Take a look at= this tutorial :</p><p></p><p><a href=3D"https://www.cyberciti.biz/faq/how-= to-install-linux-vm-on-freebsd-using-bhyve-and-zfs/" target=3D"_blank">http= s://www.cyberciti.biz/faq/how-to-install-linux-vm-on-freebsd-using-bhyve-an= d-zfs/</a></p></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" c= lass=3D"gmail_attr">On Sat, Aug 12, 2023 at 12:09=E2=80=AFPM Odhiambo Washi= ngton <<a href=3D"mailto:odhiambo@gmail.com" target=3D"_blank">odhiambo@= gmail.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"><div dir=3D"ltr"><div>I have read=C2=A0<a href=3D"https://wu.e-s= hell.org/articles/hw-to-bhyve.html" target=3D"_blank">https://wu.e-shell.or= g/articles/hw-to-bhyve.html</a> as well as=C2=A0<a href=3D"https://www.nets= cylla.com/blog/2021/06/11/FreeBSD-and-Bhyve.html" target=3D"_blank">https:/= /www.netscylla.com/blog/2021/06/11/FreeBSD-and-Bhyve.html</a> and many more= .</div><div>I will just master it with time.</div><br><div class=3D"gmail_q= uote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Aug 12, 2023 at 1:01=E2= =80=AFPM Mario Marietto <<a href=3D"mailto:marietto2008@gmail.com" targe= t=3D"_blank">marietto2008@gmail.com</a>> wrote:<br></div><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><pre dir=3D"ltr"><code>= Try like this :<br><br>bhyve -S -c sockets=3D2,cores=3D2,threads=3D2 -m 4G = -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/mnt/zroot2/zroot2/bhyve/img/Windows/windows11.img,bootindex= =3D1 \ -s 13,virtio-net,tap1 \ -s 29,fbuf,tcp=3D<a href=3D"http://0.0.0.0:5918" target=3D"_blank">0.0.0.0:= 5918</a>,w=3D1600,h=3D950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ vm0:1 < /dev/null & sleep 2 && vncviewer 0:1<br><br></code><= /pre><pre><code>Doing this way you will learn better how to compose the bhy= ve parameters and how they should be formatted naturally. <br>As you can se= e,the vm-bhyve wrapper "</code><span lang=3D"en">muddies the waters&qu= ot;. And I think that you should also learn how to create a bridge on FreeB= SD.</span></pre></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class= =3D"gmail_attr">On Sat, Aug 12, 2023 at 11:46=E2=80=AFAM Mario Marietto <= ;<a href=3D"mailto:marietto2008@gmail.com" target=3D"_blank">marietto2008@g= mail.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D= "margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-le= ft:1ex"><div dir=3D"ltr">In my opinion,the usage of the wrapper creates som= e difficulties to understand how bhyve and the networking works. You should= start from the very beginning,studying bhyve and the freebsd networking,in= stead of learning how to correctly use the wrapper.<br></div><br><div class= =3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Aug 12, 2023= at 11:25=E2=80=AFAM Odhiambo Washington <<a href=3D"mailto:odhiambo@gma= il.com" target=3D"_blank">odhiambo@gmail.com</a>> wrote:<br></div><block= quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1= px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Hello all.<div= ><br></div><div>I am new to bhyve and have read a lot about it, so I decide= d to try creating a few VMs.</div><div>I found a HOWTO that is detailed and= which appears to be the easiest to follow:=C2=A0<a href=3D"https://klarasy= stems.com/articles/from-0-to-bhyve-on-freebsd-13-1/" target=3D"_blank">http= s://klarasystems.com/articles/from-0-to-bhyve-on-freebsd-13-1/</a></div><di= v>The only differences are that I did not install a fresh FreeBSD-13 server= as I already had one, and I am not using ZFS on my server (host).</div><di= v>I have gotten stuck midway and cannot figure out why: I cannot boot the V= Ms I create.</div><div>I have tried creating a Windows2019 as well as a Deb= ian12 VM, but both don't boot.</div><div><br></div><div>vm-public: flag= s=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1492<br= >=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether ce:6f:18:51:d4:51<br>=C2=A0 =C2=A0 =C2= =A0 =C2=A0 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15<br>= =C2=A0 =C2=A0 =C2=A0 =C2=A0 maxage 20 holdcnt 6 proto rstp maxaddr 2000 tim= eout 1200<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 root id 00:00:00:00:00:00 priority= 32768 ifcost 0 port 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 member: em1 flags=3D1= 43<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP><br>=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 ifmaxaddr 0 port 2 priority 128 path cost 2000= 0<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: bridge vm-switch viid-4c918@<br>= =C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D9<PERFORMNUD,IFDISABLED><br= ></div><div>tap0: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULT= ICAST> metric 0 mtu 1492<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 description: vmn= et/debian12/0/public<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D80000<LINK= STATE><br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 58:9c:fc:10:ff:ed<br>=C2=A0 = =C2=A0 =C2=A0 =C2=A0 inet6 fe80::5a9c:fcff:fe10:ffed%tap0 prefixlen 64 scop= eid 0x6<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: tap vm-port<br>=C2=A0 =C2=A0= =C2=A0 =C2=A0 media: Ethernet autoselect<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 st= atus: no carrier<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D21<PERFORM= NUD,AUTO_LINKLOCAL><br></div><div><br></div><div>1.=C2=A0 Windows2019 VM= :</div><div>=C2=A0 =C2=A0 vm create -t uefi -s 100G -m 8G -c8 windows2019</= div><div>=C2=A0 =C2=A0=C2=A0vm install windows2019 /usr/local/bhyve-vms/.is= o/server2019.iso</div><div><br></div><div>2. Debian12 VM:</div><div>=C2=A0 = =C2=A0=C2=A0vm create -s 60G -m 8G -c 8 debian12</div><div>=C2=A0 =C2=A0=C2= =A0vm install debian12 .iso/debian-12.0.0-amd64-DVD-1.iso</div><div><br></d= iv><div>root@gw:/usr/local/bhyve-vms # vm list<br>NAME =C2=A0 =C2=A0 =C2=A0= =C2=A0 DATASTORE =C2=A0LOADER =C2=A0 =C2=A0 CPU =C2=A0MEMORY =C2=A0VNC =C2= =A0AUTO =C2=A0 =C2=A0 STATE<br>debian12 =C2=A0 =C2=A0 default=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 bhyveload=C2=A0 =C2=A0 8=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A08G=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2= =A0 =C2=A0 =C2=A0Yes[2]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Bootloader (91866)= <br>windows2019 =C2=A0default=C2=A0 =C2=A0 =C2=A0 =C2=A0 uefi=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A08=C2=A0 =C2=A0 =C2=A0 =C2=A0 8G=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0Ye= s [1]=C2=A0 =C2=A0 =C2=A0Stopped<br></div><div><br></div><div>From the abov= e output of `vm list`, you can see that none of them is booting up.</div><d= iv><br></div><div>What is it that I could be missing?</div><div><br></div><= div><br></div><div><span class=3D"gmail_signature_prefix">-- </span><br><di= v dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div dir=3D"ltr"><= div>Best regards,<br>Odhiambo WASHINGTON,<br>Nairobi,KE<br>+254 7 3200 0004= /+254 7 2274 3223<br>"<span style=3D"font-size:12.8px">Oh, the cruft.<= /span><span style=3D"font-size:12.8px">",=C2=A0</span><span style=3D"f= ont-size:12.8px">egrep -v '^$|^.*#'=C2=A0</span><span style=3D"back= ground-color:rgb(34,34,34);color:rgb(238,238,238);font-family:"Lucida = Console",Consolas,"Courier New",monospace;font-size:13.6px">= =C2=AF\_(=E3=83=84)_/=C2=AF</span><span style=3D"font-size:12.8px">=C2=A0:-= )</span></div><div><span style=3D"font-size:12.8px">[How to ask smart quest= ions:=C2=A0</span><span style=3D"font-size:12.8px"><a href=3D"http://www.ca= tb.org/~esr/faqs/smart-questions.html" target=3D"_blank">http://www.catb.or= g/~esr/faqs/smart-questions.html</a>]</span></div></div></div></div></div><= /div> </blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signature_pre= fix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Mario.<br></d= iv> </blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signature_pre= fix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Mario.<br></d= iv> </blockquote></div><br clear=3D"all"><div><br></div><span class=3D"gmail_si= gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d= iv dir=3D"ltr"><div dir=3D"ltr"><div>Best regards,<br>Odhiambo WASHINGTON,<= br>Nairobi,KE<br>+254 7 3200 0004/+254 7 2274 3223<br>"<span style=3D"= font-size:12.8px">Oh, the cruft.</span><span style=3D"font-size:12.8px">&qu= ot;,=C2=A0</span><span style=3D"font-size:12.8px">egrep -v '^$|^.*#'= ;=C2=A0</span><span style=3D"background-color:rgb(34,34,34);color:rgb(238,2= 38,238);font-family:"Lucida Console",Consolas,"Courier New&q= uot;,monospace;font-size:13.6px">=C2=AF\_(=E3=83=84)_/=C2=AF</span><span st= yle=3D"font-size:12.8px">=C2=A0:-)</span></div><div><span style=3D"font-siz= e:12.8px">[How to ask smart questions:=C2=A0</span><span style=3D"font-size= :12.8px"><a href=3D"http://www.catb.org/~esr/faqs/smart-questions.html" tar= get=3D"_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a>]</spa= n></div></div></div></div></div> </blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signature_pre= fix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Mario.<br></d= iv> --000000000000438f440602b73757--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSijNC_Cvd3%2B=BvQRZqxVh2mVqmzdpg4zdcbb-OTJ3Tip9g>