Date: Sun, 11 Jan 2015 23:46:49 +0100 From: Willem Jan Withagen <wjw@digiware.nl> To: williamecowell@hush.ai, freebsd-net@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: ipv4 routing from bhyve Message-ID: <54B2FD59.9000407@digiware.nl> In-Reply-To: <20150111213203.60018E00B1@smtp.hushmail.com> References: <20150111213203.60018E00B1@smtp.hushmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11-1-2015 22:32, williamecowell@hush.ai wrote:
> Hello, I hope I can have some assistance.
>
> I am trying to get networking via wlan0 but without NAT or bridging (doesn't work on wifi unless WDS).
>
> say my my main network is 10.10.2.0/24, gateway/internet is 10.10.2.1, my ip is 10.10.2.252.
>
> I started to config my bhyve network on 172.16.32.0/24
>
> I added a bridge interface with an ip of 172.16.32.1
>
> enable forwarding and fastforwarding. from my understanding of the handbook chapter things should work when I type:
>
> # route add -net 172.16.32.0/24 10.10.2.252
> route: writing to routing socket: File exists
> add net 172.16.32.0: gateway 10.10.2.252 fib 0: route already in table
> #
>
> # netstat -4nr
> Routing tables
>
> Internet:
> Destination Gateway Flags Netif Expire
> default 10.10.2.1 UGS lagg0
> 127.0.0.1 link#3 UH lo0
> 10.10.2.0/24 link#5 U lagg0
> 10.10.2.252 link#5 UHS lo0
> 172.16.32.0/24 link#4 U bridge0
> 172.16.32.1 link#4 UHS lo0
> #
>
> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> ether 00:bd:0f:fc:01:10
> inet 172.16.32.1 netmask 0xffffff00 broadcast 172.16.32.255
> nd6 options=9<PERFORMNUD,IFDISABLED>
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> ifmaxaddr 0 port 6 priority 128 path cost 2000000
> lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> ...
> inet
netmask 0xffffff00 broadcast 10.10.2.255
> nd6 options=9<PERFORMNUD,IFDISABLED>
> media: Ethernet autoselect
> status: active
> laggproto failover lagghash l2,l3,l4
> laggport: alc0 flags=1<MASTER>
> laggport: wlan0 flags=4<ACTIVE>
> tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=80000<LINKSTATE>
> ether 00:bd:8f:62:67:10
> nd6 options=9<PERFORMNUD,IFDISABLED>
> media: Ethernet autoselect
> status: no carrier
> wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> ...
> pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
> tap9: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=80000<LINKSTATE>
> ether 00:bd:cb:46:02:09
> nd6 options=1<PERFORMNUD>
> media: Ethernet autoselect
> status: no carrier
> tap1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=80000<LINKSTATE>
> ether 00:bd:58:61:02:01
> nd6 options=1<PERFORMNUD>
> media: Ethernet autoselect
> status: no carrier
Well one of the things of concern is the fact that your tap interfaces
have:
status: no carrier
My connected bhyve vm's have, amongst others:
status: active
groups: tap
Opened by PID 20763
And my bridge device tells me:
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500
ether 02:76:2d:3d:9c:00
inet xxx.xxx.xxx.xxx netmask 0xff000000 broadcast 37.255.255.255
nd6 options=9<PERFORMNUD,IFDISABLED>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap651 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 11 priority 128 path cost 2000000
member: tap6 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 10 priority 128 path cost 55
member: tap14041 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 9 priority 128 path cost 2000000
member: tap13101 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 8 priority 128 path cost 2000000
member: tap12041 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000000
member: tap13 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
So I think you first need to connect your VM's, before anything else
will start to work. Like adding the tap-ifs to the bridge.
And on the host itself you don't really need to add routing for the VM's
because everything is actually already connected. Which is what the
netstat output tells you. The routing table tells you that traffic for
172.16.32.0/24 link#4 U bridge0
is send into the the bridge0 devices, which is directly connected.
And ip-nrs in that range should appear in the the arp table.
And the host then knows how to get to them directly.
Routing for 172.16.32.0/24, if any needed, will be required on other
hosts on you network on lagg0. Unless all hosts there have 10.10.2.252
as their default route.
Regards,
--WjW
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54B2FD59.9000407>