Date: Tue, 17 Aug 2021 15:55:23 +0200 From: Mario Marietto <marietto2008@gmail.com> To: freebsd-net@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Confirm subscription to freebsd-net@FreeBSD.org Message-ID: <CA%2B1FSigbnNg08Q3GTLG59xtepzg-uQB=QPQoAt_u2DiMkU_avg@mail.gmail.com> In-Reply-To: <CA%2B1FSii8M2E4ghOEenPkTw3HzyWj7a4Dm6=c7A7KruoMf%2BKg7w@mail.gmail.com> References: <1629141950-13176-mlmmj-4abd3c89@FreeBSD.org> <CA%2B1FSii8M2E4ghOEenPkTw3HzyWj7a4Dm6=c7A7KruoMf%2BKg7w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000074620f05c9c1ad86
Content-Type: text/plain; charset="UTF-8"
Hello.
I'm trying to configure the bridging and the tap interface on freebsd 13
because I want to boot and use a bhyve VM based on Ubuntu 21.04 and Windows
10,but unfortunately,after several readings and help requests on :
0.
https://www.reddit.com/r/Ubuntu/comments/p38ru1/ubuntu_2104_cant_connect_to_internet_when_it_is/
1.
https://forums.freebsd.org/threads/how-to-duplicate-the-tap0-interface-in-a-new-tap1-network-interface-that-it-is-not-used.81671/page-2#post-527355
2.
https://forums.freebsd.org/threads/ubuntu-budgie-emulated-with-bhyve-doesnt-connect-to-internet-because-the-error-network-unreachable-n-2.81340/#post-524303
3.
https://forums.freebsd.org/threads/im-tryng-to-emulate-ubuntu-budgie-with-bhyve-but-it-doesnt-connect-to-internet-because-the-error-network-unreachable.80916/#post-517845
4.
https://askubuntu.com/questions/1357670/ubuntu-21-04-cant-connect-to-internet-when-it-is-emulated-with-bhyve-on-my-free
5.
https://unix.stackexchange.com/questions/664466/ubuntu-21-04-cant-connect-to-internet-when-it-is-emulated-with-bhyve-on-my-free?noredirect=1#comment1250440_664466
6.
https://serverfault.com/questions/1074745/my-bridging-configuration-on-freebsd-13-inst-properly-configured-because-any-bh
7.
https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/my-bridging-configuration-on-freebsd-13-ins%27t-properly-configured-because-any-bhyve-vm-based-on-ubuntu-21-04-and-windows-10-can%27t-connect-4175699240/
I haven't been able to understand why Ubuntu and Windows 10 VM can't
connect to the internet,while if I use vm-bhyve,it configures the network
in some different way and the vm can connect to the internet. Since I'm not
a coder,I'm not able to look inside the vm-bhyve code and I'm not able to
understand what's the secret key. Below I want to give you some useful
information for understanding how I have configured the bridge and the tap
interface on FreeBSD and on Ubuntu.
/etc/resolv.conf
search homenet.telecomitalia.it
nameserver 127.0.0.1
options edns0
my /etc/rc.conf is like this :
hostname="marietto"
keymap="it.kbd"
ifconfig_em0="DHCP"
local_unbound_enable="YES"
sshd_enable="YES"
powerd_enable="YES"
ntpdate_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
dumpdev="NO"
dbus_enable="YES"
slim_enable="NO"
libvirt_enable="YES"
linux_enable="YES"
linux_mounts_enable="YES"
gateway_enable="YES"
kld_list="fusefs"
lightdm_enable="NO"
sendmail_enable="NO"
microcode_update_enable="YES"
fsck_y_enable="YES"
tcp_drop_synfin="NO"
telnet_enable="NO"
cupsd_enable="NO"
portmap_enable="NO"
rlogin_enable="NO"
inetd_enable="NO"
webcamd_enable="NO"
samba_enable="NO"
lpd_enable="NO"
winbindd_enable="NO"
nfs_server_enable="NO"
nfs_client_enable="NO"
vm_enable="YES"
vm_dir="/mnt/da1p1/vms/os/"
my /boot/loader.conf is like this :
vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"
fusefs_load="YES"
linprocfs_load="YES"
linsysfs_load="YES"
tmpfs_load="YES"
verbose_loading="YES"
vboxdrv_load="YES"
console=vga,com1
boot_multicons="YES"
boot_serial="YES"
console="comconsole,vidconsole"
my file /etc/sysctl.conf is like this :
net.link.tap.up_on_open=1
net.inet.ip.forwarding=1
I've read here to get some orientation about what to do :
https://wiki.freebsd.org/bhyve/UEFI
<https://wiki.freebsd.org/bhyve/UEFI,specially>; specially in this section :
kldload vmm
ifconfig tap0 create up
ifconfig bridge0 create up
ifconfig bridge0 addm em0
As a first attempt,with the suggestions of Vasily Postnicov,I tried to
configure the network interfaces of freebsd and of linux in a different
way,to be able to give the internet connection to ubuntu emulated with
bhyve. This is what I did :
root@marietto:/home/marietto # ifconfig em0 delete
root@marietto:/home/marietto # dhclient bridge0
DHCPDISCOVER on bridge0 to 255.255.255.255 port 67 interval 4
DHCPOFFER from 192.168.1.1
DHCPREQUEST on bridge0 to 255.255.255.255 port 67
DHCPACK from 192.168.1.1
bound to 192.168.1.2 -- renewal in 10800 seconds.
root@marietto:/home/marietto # ifconfig tap0 create up
root@marietto:/home/marietto # ifconfig bridge0 addm tap0
root@marietto:/usr/home/marietto # ifconfig
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
options=4810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d5:5e:e2:1f:22
inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:ff:b7
inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 2000000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether b6:74:ff:a8:4d:4e
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
groups: bridge vm-switch viid-4c918@
nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 58:9c:fc:10:ff:8e
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Unfortunately as soon as the vm boots,the em0 interface on freebsd gets
again the old IP number : 192.168.1.6 and I lose the connection to the net
on FreeBSD.
As a further try this is what I did : (take in consideration that I'm
trying to make a parallel network configuration on Freebsd,since I want to
keep vm-bhyve installed. Anyway,I've removed vm-bhyve several times and I
tried to configure the bridge network for bhyve,but even in this case it
didn't work. In short terms,it worked only the first time,but I haven't
been able to repeat succesfully the steps anymore. It's working only if I
let vm-bhyve configure the network by itself. This is the initial state of
the network / bridge configuration on my FreeBSD with vm-bhyve installed
but without starting a virtual machine.
marietto@marietto:~ $ ifconfig
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
options=481009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d5:5e:e2:1f:22
inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 0a:77:93:94:d5:77
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge vm-switch viid-4c918@
nd6 options=9<PERFORMNUD,IFDISABLED>
Since I already have a bridged interface (vm-public) it does not make any
sense to create another one (bridge0),so on the terminal I do :
root@marietto:/home/marietto # kldload vmm
kldload: can't load vmm: module already loaded or in kernel
root@marietto:/home/marietto # ifconfig tap0 create up
root@marietto:/home/marietto # ifconfig vm-public addm tap0
so,this is how appear the network configuration on FreeBSD after having
configured it like above :
root@marietto:/home/marietto # ifconfig
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
options=4810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d5:5e:e2:1f:22
inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 0a:77:93:94:d5:77
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge vm-switch viid-4c918@
nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 58:9c:fc:10:ff:8e
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Then,I launch the bhyve VM with this commands :
bhyve -c 4 -m 8G -w -H \
-s 0,hostbridge \
-s 4,virtio-blk,/mnt/da1p1/vms/os/ubuntu-budgie-gpu/ubuntu-2104-gpu.img \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
-s 20,hda,play=/dev/dsp8,rec=/dev/dsp8 \
vm0
As I said Ubuntu VM can't connect to the internet. Below I want to show you
how I have configured Ubuntu,instead. On Ubuntu my /etc/resolv.conf file is
like this :
nameserver 127.0.0.53
options edns0 trust-ad
search homenet.telecomitalia.it
In addition I've created a netplan file inside the folder /etc/netplan,with
this content inside :
network
version: 2
renderer: NetworkManager
ethernets:
enp0s5:
addresses:
- 192.168.1.7/24
gateway4: 192.168.1.1
nameservers:
search : [homenet.telecomitalia.it]
addresses: 192.168.1.1
netplan apply and this is what happens :
the network interface enp0s5 gets the IP number 192.168.1.7 ping
192.168.1.7 : OK ping 192.168.1.6 (FreeBSD IP) = destination host
unreachable ping 192.168.1.1 : destination host unreachable PING google.it
(172.217.21.67) : host unreachable
tcpdump :
listening on enp0s5,link-type EN10MB (Ethernet),capture size 262144 bytes
20:58:04:688913 ARP, request who-has 192.168.1.1 tell 192.168.1.7,length 28
route -n
destination gateway genmask flags metric ref use iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100
0 0 enp0s5
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0
0 enp0s5
192.168.1.0 0.0.0.0 255.255.255.0 U 100 0
0 enp0s5
ip route list
default via 192.168.1.1 dev enp0s5 proto static metric
100169.254.0.0/16 dev enp0s5 scope link metric 100192.168.1.0/24 dev
enp0s5 proto kernel scope link src 192.168.1.7 metric 100
ping google.it = temporary error in name resolution
firewall on ubuntu = not installed firewall on router = down
instead,from freebsd to ubuntu :
ping 192.168.1.7 : host is down
netstar -r
routing tables
internet:
destination gateway flags Netif Expire
default modemtim UGS em0
localhost link#2 UH lo0192.168.1.0/24
link#1 U em0
192.168.1.6 link#1 UHS lo0
I forgot. This is what happens if I remove the netplan conf file and I let
ubuntu choose the IP via DHCP :
/etc/netplan.conf
network:
version: 2
renderer: networkd
ethernets:
enp3s0:
dhcp4: true
ping PING google.it (172.217.21.67) : network unreachable. What else ?
PS : Since I'm not a real system administrator,but I do it as a hobby,don't
give me generic suggestions because probably I will not understant what to
do. If you want to be a kind person,try to be exhaustive and precise. You
will help me a lot to understand how to fix the problem. thanks.
PS : more failed attempt to fix the problem are listed here :
https://forums.freebsd.org/threads/how-to-duplicate-the-tap0-interface-in-a-new-tap1-network-interface-that-it-is-not-used.81671/page-3
--00000000000074620f05c9c1ad86--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSigbnNg08Q3GTLG59xtepzg-uQB=QPQoAt_u2DiMkU_avg>