From owner-freebsd-net@FreeBSD.ORG Sun Jan 11 22:47:02 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E4FEC876; Sun, 11 Jan 2015 22:47:02 +0000 (UTC) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8FAA3BD5; Sun, 11 Jan 2015 22:47:01 +0000 (UTC) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 3272716A403; Sun, 11 Jan 2015 23:46:53 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hNsD98Uk7GBQ; Sun, 11 Jan 2015 23:46:51 +0100 (CET) Received: from [IPv6:2001:4cb8:3:1:a07a:8688:5c0:df34] (unknown [IPv6:2001:4cb8:3:1:a07a:8688:5c0:df34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 2E7B016A401; Sun, 11 Jan 2015 23:46:51 +0100 (CET) Message-ID: <54B2FD59.9000407@digiware.nl> Date: Sun, 11 Jan 2015 23:46:49 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: williamecowell@hush.ai, freebsd-net@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: ipv4 routing from bhyve References: <20150111213203.60018E00B1@smtp.hushmail.com> In-Reply-To: <20150111213203.60018E00B1@smtp.hushmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2015 22:47:03 -0000 On 11-1-2015 22:32, williamecowell@hush.ai wrote: > Hello, I hope I can have some assistance. > > I am trying to get networking via wlan0 but without NAT or bridging (doesn't work on wifi unless WDS). > > say my my main network is 10.10.2.0/24, gateway/internet is 10.10.2.1, my ip is 10.10.2.252. > > I started to config my bhyve network on 172.16.32.0/24 > > I added a bridge interface with an ip of 172.16.32.1 > > enable forwarding and fastforwarding. from my understanding of the handbook chapter things should work when I type: > > # route add -net 172.16.32.0/24 10.10.2.252 > route: writing to routing socket: File exists > add net 172.16.32.0: gateway 10.10.2.252 fib 0: route already in table > # > > # netstat -4nr > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > default 10.10.2.1 UGS lagg0 > 127.0.0.1 link#3 UH lo0 > 10.10.2.0/24 link#5 U lagg0 > 10.10.2.252 link#5 UHS lo0 > 172.16.32.0/24 link#4 U bridge0 > 172.16.32.1 link#4 UHS lo0 > # > > bridge0: flags=8843 metric 0 mtu 1500 > ether 00:bd:0f:fc:01:10 > inet 172.16.32.1 netmask 0xffffff00 broadcast 172.16.32.255 > nd6 options=9 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap0 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > lagg0: flags=8843 metric 0 mtu 1500 > ... > inet netmask 0xffffff00 broadcast 10.10.2.255 > nd6 options=9 > media: Ethernet autoselect > status: active > laggproto failover lagghash l2,l3,l4 > laggport: alc0 flags=1 > laggport: wlan0 flags=4 > tap0: flags=8903 metric 0 mtu 1500 > options=80000 > ether 00:bd:8f:62:67:10 > nd6 options=9 > media: Ethernet autoselect > status: no carrier > wlan0: flags=8843 metric 0 mtu 1500 > ... > pflog0: flags=141 metric 0 mtu 33160 > tap9: flags=8802 metric 0 mtu 1500 > options=80000 > ether 00:bd:cb:46:02:09 > nd6 options=1 > media: Ethernet autoselect > status: no carrier > tap1: flags=8802 metric 0 mtu 1500 > options=80000 > ether 00:bd:58:61:02:01 > nd6 options=1 > media: Ethernet autoselect > status: no carrier Well one of the things of concern is the fact that your tap interfaces have: status: no carrier My connected bhyve vm's have, amongst others: status: active groups: tap Opened by PID 20763 And my bridge device tells me: bridge0: flags=8843 metric 0 mtu 1500 ether 02:76:2d:3d:9c:00 inet xxx.xxx.xxx.xxx netmask 0xff000000 broadcast 37.255.255.255 nd6 options=9 groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap651 flags=143 ifmaxaddr 0 port 11 priority 128 path cost 2000000 member: tap6 flags=143 ifmaxaddr 0 port 10 priority 128 path cost 55 member: tap14041 flags=143 ifmaxaddr 0 port 9 priority 128 path cost 2000000 member: tap13101 flags=143 ifmaxaddr 0 port 8 priority 128 path cost 2000000 member: tap12041 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000000 member: tap13 flags=143 ifmaxaddr 0 port 4 priority 128 path cost 2000000 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 20000 So I think you first need to connect your VM's, before anything else will start to work. Like adding the tap-ifs to the bridge. And on the host itself you don't really need to add routing for the VM's because everything is actually already connected. Which is what the netstat output tells you. The routing table tells you that traffic for 172.16.32.0/24 link#4 U bridge0 is send into the the bridge0 devices, which is directly connected. And ip-nrs in that range should appear in the the arp table. And the host then knows how to get to them directly. Routing for 172.16.32.0/24, if any needed, will be required on other hosts on you network on lagg0. Unless all hosts there have 10.10.2.252 as their default route. Regards, --WjW