From nobody Sat Aug 12 10:22:33 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNGvh1jnNz4mK0h for ; Sat, 12 Aug 2023 10:23:12 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNGvg52mxz4W5X for ; Sat, 12 Aug 2023 10:23:11 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-99bf1f632b8so389093466b.1 for ; Sat, 12 Aug 2023 03:23:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691835790; x=1692440590; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=J/KWaPN/mLpSRTn9lDDY5tLuSGSrCqr5CgOoZN7CjzU=; b=IppHi7VLJkxwM80j5NZRrcFqkTjQoGKrUEpcsdjLbzgEDP/1lF7ggJdMEJDUh2YBxc 1XUKjlQs42lQMCpqWhEcsR24dgJrBhFNAD9/T8YovVMrUioxmyURddKfm/vUMQsvsUvJ wZ6Utce5uf2lK1MfpAGo+uK9Al0R8rc/2FZMJV7Ev4++6WIbL0/K90zXhbrfEUmtQBTN EslKCbCmZUqLU1byncHCFlmHu5jIU1xV1i9t6nrSd6YvaNUX08rM3yrcaadhuB9ZGX2K m2hn0/ppNwdJ2+dE6+zeRuFM0dwIjae+hL/ZZVcmWfSDxxYNSNTo5xuXFcHV4uu1EO6p wKww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691835790; x=1692440590; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=J/KWaPN/mLpSRTn9lDDY5tLuSGSrCqr5CgOoZN7CjzU=; b=ja72wQ+M8QF+RNAPQTDbI/tnGvrkuRo4MvxbiZ3mqUu4tNEoh1JTviTT8R6Nppsfap Ha9ukgRIns0iiunsE6Gu9L6mSP+a1AvYeO/zRdNVkJAGUxW90SxS4vLUzN8Es28mMhyu CKQP22xo9DZiV4lFOsIwYQBWJYppOFMEwtqYhJxY+E5wH8Qr2fPwrm6OVyOrd2ZXurlA CPKPAyizK1GAMksXYXdJG904TefmNH9MYh33hnAoGv9YyP5mj72h/xJl0If2BNy0Ty5P e3X87vgbszUKbaFS2zlwp5L/PlfRkGkrNqhU/LynwMnH/ROQsfXVY515Y8C/Ul+SlkUq +Omg== X-Gm-Message-State: AOJu0Yw6Dr820RAh8NeswvWNmr0FZBwhmU0pEyC1FtRMI4XqwWYsXWit NNxnR5NiIoGR5iV0Rj4lq288DbATU+WsnYCrLdGv0VqJabI= X-Google-Smtp-Source: AGHT+IHbOZEENObbNHt3p2x3nfksbEsTMC9qF43XwIrbLqiORHkcRfm9wD+eTvsH0rksJgVsFlI0JkKmAStEWZAoyRE= X-Received: by 2002:a17:906:cc0d:b0:997:caf0:9945 with SMTP id ml13-20020a170906cc0d00b00997caf09945mr3340428ejb.12.1691835789707; Sat, 12 Aug 2023 03:23:09 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Sat, 12 Aug 2023 12:22:33 +0200 Message-ID: Subject: Re: Starting the bhyve journey To: Odhiambo Washington Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000438f440602b73757" X-Rspamd-Queue-Id: 4RNGvg52mxz4W5X X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000438f440602b73757 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable As I had suspected,the first tutorial wants you learn vm-bhyve and iohyve (the wrapper for zfs and bhyve),as he says : *I=E2=80=99m not saying this is the way to go*. Really, really, If you are = planning on setting things up from scratch to have a nice FreeBSD virtual machines server, look into all the cool projects out there like vm-bhyve or iohyve (to name two of them, I=E2=80=99m sure t= here are more). That's not good. If you want to set up from scratch you should not focus too much on studying the vm-bhyve or iohyve,it makes things even more complicated because these are tools that change themself during the time so you should keep updated with more knowledge. Take a look at this tutorial : https://www.cyberciti.biz/faq/how-to-install-linux-vm-on-freebsd-using-bhyv= e-and-zfs/ On Sat, Aug 12, 2023 at 12:09=E2=80=AFPM Odhiambo Washington wrote: > I have read https://wu.e-shell.org/articles/hw-to-bhyve.html as well as > https://www.netscylla.com/blog/2021/06/11/FreeBSD-and-Bhyve.html and many > more. > I will just master it with time. > > On Sat, Aug 12, 2023 at 1:01=E2=80=AFPM Mario Marietto > wrote: > >> Try like this : >> >> bhyve -S -c sockets=3D2,cores=3D2,threads=3D2 -m 4G -w -H -A \ >> -s 0,hostbridge \ >> -s 1,ahci-hd,/mnt/zroot2/zroot2/bhyve/img/Windows/windows11.img,bootinde= x=3D1 \ >> -s 13,virtio-net,tap1 \ >> -s 29,fbuf,tcp=3D0.0.0.0:5918,w=3D1600,h=3D950,wait \ >> -s 30,xhci,tablet \ >> -s 31,lpc \ >> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ >> vm0:1 < /dev/null & sleep 2 && vncviewer 0:1 >> >> Doing this way you will learn better how to compose the bhyve parameters= and how they should be formatted naturally. >> As you can see,the vm-bhyve wrapper "muddies the waters". And I think th= at you should also learn how to create a bridge on FreeBSD. >> >> >> On Sat, Aug 12, 2023 at 11:46=E2=80=AFAM Mario Marietto >> wrote: >> >>> In my opinion,the usage of the wrapper creates some difficulties to >>> understand how bhyve and the networking works. You should start from th= e >>> very beginning,studying bhyve and the freebsd networking,instead of >>> learning how to correctly use the wrapper. >>> >>> On Sat, Aug 12, 2023 at 11:25=E2=80=AFAM Odhiambo Washington >>> wrote: >>> >>>> Hello all. >>>> >>>> I am new to bhyve and have read a lot about it, so I decided to try >>>> creating a few VMs. >>>> I found a HOWTO that is detailed and which appears to be the easiest t= o >>>> follow: >>>> https://klarasystems.com/articles/from-0-to-bhyve-on-freebsd-13-1/ >>>> The only differences are that I did not install a fresh FreeBSD-13 >>>> server as I already had one, and I am not using ZFS on my server (host= ). >>>> I have gotten stuck midway and cannot figure out why: I cannot boot th= e >>>> VMs I create. >>>> I have tried creating a Windows2019 as well as a Debian12 VM, but both >>>> don't boot. >>>> >>>> vm-public: flags=3D8843 metric= 0 >>>> mtu 1492 >>>> ether ce:6f:18:51:d4:51 >>>> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >>>> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 >>>> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >>>> member: em1 flags=3D143 >>>> ifmaxaddr 0 port 2 priority 128 path cost 20000 >>>> groups: bridge vm-switch viid-4c918@ >>>> nd6 options=3D9 >>>> tap0: flags=3D8943 met= ric >>>> 0 mtu 1492 >>>> description: vmnet/debian12/0/public >>>> options=3D80000 >>>> ether 58:9c:fc:10:ff:ed >>>> inet6 fe80::5a9c:fcff:fe10:ffed%tap0 prefixlen 64 scopeid 0x6 >>>> groups: tap vm-port >>>> media: Ethernet autoselect >>>> status: no carrier >>>> nd6 options=3D21 >>>> >>>> 1. Windows2019 VM: >>>> vm create -t uefi -s 100G -m 8G -c8 windows2019 >>>> vm install windows2019 /usr/local/bhyve-vms/.iso/server2019.iso >>>> >>>> 2. Debian12 VM: >>>> vm create -s 60G -m 8G -c 8 debian12 >>>> vm install debian12 .iso/debian-12.0.0-amd64-DVD-1.iso >>>> >>>> root@gw:/usr/local/bhyve-vms # vm list >>>> NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE >>>> debian12 default bhyveload 8 8G >>>> - Yes[2] Bootloader (91866) >>>> windows2019 default uefi 8 8G >>>> - Yes [1] Stopped >>>> >>>> From the above output of `vm list`, you can see that none of them is >>>> booting up. >>>> >>>> What is it that I could be missing? >>>> >>>> >>>> -- >>>> Best regards, >>>> Odhiambo WASHINGTON, >>>> Nairobi,KE >>>> +254 7 3200 0004/+254 7 2274 3223 >>>> "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) >>>> [How to ask smart questions: >>>> http://www.catb.org/~esr/faqs/smart-questions.html] >>>> >>> >>> >>> -- >>> Mario. >>> >> >> >> -- >> Mario. >> > > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) > [How to ask smart questions: > http://www.catb.org/~esr/faqs/smart-questions.html] > --=20 Mario. --000000000000438f440602b73757 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
As I had suspected,the first tutorial wants you learn= vm-bhyve and iohyve (the wrapper for zfs and bhyve),as he says :


On Sat, Aug 12, 2023 at 12:09=E2=80=AFPM Odhiambo Washi= ngton <odhiambo@= gmail.com> wrote:
I will just master it with time.

On Sat, Aug 12, 2023 at 1:01=E2= =80=AFPM Mario Marietto <marietto2008@gmail.com> wrote:
=
Try like this :

bhyve -S -c sockets=3D2,cores=3D2,threads=3D2 -m 4G =
-w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/zroot2/zroot2/bhyve/img/Windows/windows11.img,bootindex=
=3D1 \
-s 13,virtio-net,tap1 \
-s 29,fbuf,tcp=3D0.0.0.0:=
5918,w=3D1600,h=3D950,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:1 < /dev/null & sleep 2 && vncviewer 0:1

<=
/pre>
Doing this way you will learn better how to compose the bhy=
ve parameters and how they should be formatted naturally. 
As you can se=
e,the vm-bhyve wrapper "muddies the waters&qu=
ot;. And I think that you should also learn how to create a bridge on FreeB=
SD.

On Sat, Aug 12, 2023 at 11:46=E2=80=AFAM Mario Marietto <= ;marietto2008@g= mail.com> wrote:
In my opinion,the usage of the wrapper creates som= e difficulties to understand how bhyve and the networking works. You should= start from the very beginning,studying bhyve and the freebsd networking,in= stead of learning how to correctly use the wrapper.

On Sat, Aug 12, 2023= at 11:25=E2=80=AFAM Odhiambo Washington <odhiambo@gmail.com> wrote:
Hello all.
I am new to bhyve and have read a lot about it, so I decide= d to try creating a few VMs.
I found a HOWTO that is detailed and= which appears to be the easiest to follow:=C2=A0http= s://klarasystems.com/articles/from-0-to-bhyve-on-freebsd-13-1/
The only differences are that I did not install a fresh FreeBSD-13 server= as I already had one, and I am not using ZFS on my server (host).
I have gotten stuck midway and cannot figure out why: I cannot boot the V= Ms I create.
I have tried creating a Windows2019 as well as a Deb= ian12 VM, but both don't boot.

vm-public: flag= s=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1492=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether ce:6f:18:51:d4:51
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 maxage 20 holdcnt 6 proto rstp maxaddr 2000 tim= eout 1200
=C2=A0 =C2=A0 =C2=A0 =C2=A0 root id 00:00:00:00:00:00 priority= 32768 ifcost 0 port 0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 member: em1 flags=3D1= 43<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 ifmaxaddr 0 port 2 priority 128 path cost 2000= 0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: bridge vm-switch viid-4c918@
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D9<PERFORMNUD,IFDISABLED>
tap0: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULT= ICAST> metric 0 mtu 1492
=C2=A0 =C2=A0 =C2=A0 =C2=A0 description: vmn= et/debian12/0/public
=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D80000<LINK= STATE>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 58:9c:fc:10:ff:ed
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 inet6 fe80::5a9c:fcff:fe10:ffed%tap0 prefixlen 64 scop= eid 0x6
=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: tap vm-port
=C2=A0 =C2=A0= =C2=A0 =C2=A0 media: Ethernet autoselect
=C2=A0 =C2=A0 =C2=A0 =C2=A0 st= atus: no carrier
=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D21<PERFORM= NUD,AUTO_LINKLOCAL>

1.=C2=A0 Windows2019 VM= :
=C2=A0 =C2=A0 vm create -t uefi -s 100G -m 8G -c8 windows2019
=C2=A0 =C2=A0=C2=A0vm install windows2019 /usr/local/bhyve-vms/.is= o/server2019.iso

2. Debian12 VM:
=C2=A0 = =C2=A0=C2=A0vm create -s 60G -m 8G -c 8 debian12
=C2=A0 =C2=A0=C2= =A0vm install debian12 .iso/debian-12.0.0-amd64-DVD-1.iso

root@gw:/usr/local/bhyve-vms # vm list
NAME =C2=A0 =C2=A0 =C2=A0= =C2=A0 DATASTORE =C2=A0LOADER =C2=A0 =C2=A0 CPU =C2=A0MEMORY =C2=A0VNC =C2= =A0AUTO =C2=A0 =C2=A0 STATE
debian12 =C2=A0 =C2=A0 default=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 bhyveload=C2=A0 =C2=A0 8=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A08G=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2= =A0 =C2=A0 =C2=A0Yes[2]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Bootloader (91866)=
windows2019 =C2=A0default=C2=A0 =C2=A0 =C2=A0 =C2=A0 uefi=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A08=C2=A0 =C2=A0 =C2=A0 =C2=A0 8G=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0Ye= s [1]=C2=A0 =C2=A0 =C2=A0Stopped

From the abov= e output of `vm list`, you can see that none of them is booting up.

What is it that I could be missing?

<= div>
--
<= div>Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004= /+254 7 2274 3223
"Oh, the cruft.<= /span>",=C2=A0egrep -v '^$|^.*#'=C2=A0= =C2=AF\_(=E3=83=84)_/=C2=AF=C2=A0:-= )
[How to ask smart quest= ions:=C2=A0http://www.catb.or= g/~esr/faqs/smart-questions.html]
<= /div>


--
Mario.


--
Mario.


--
Best regards,
Odhiambo WASHINGTON,<= br>Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.&qu= ot;,=C2=A0egrep -v '^$|^.*#'= ;=C2=A0=C2=AF\_(=E3=83=84)_/=C2=AF=C2=A0:-)
[How to ask smart questions:=C2=A0http://www.catb.org/~esr/faqs/smart-questions.html]


--
Mario.
--000000000000438f440602b73757--