From owner-freebsd-net@FreeBSD.ORG Wed Apr 22 23:45:04 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79090E58 for ; Wed, 22 Apr 2015 23:45:04 +0000 (UTC) Received: from mx2.shrew.net (mx2.shrew.net [38.97.5.132]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40B0F167F for ; Wed, 22 Apr 2015 23:45:03 +0000 (UTC) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx2.shrew.net (8.14.7/8.14.7) with ESMTP id t3MNGBqt021816 for ; Wed, 22 Apr 2015 18:16:11 -0500 (CDT) (envelope-from mgrooms@shrew.net) Received: from [10.16.32.30] (72-48-144-84.static.grandenetworks.net [72.48.144.84]) by mail.shrew.net (Postfix) with ESMTPSA id 14BCB18A8DD for ; Wed, 22 Apr 2015 18:16:01 -0500 (CDT) Message-ID: <55382C0A.1040505@shrew.net> Date: Wed, 22 Apr 2015 18:17:30 -0500 From: Matthew Grooms User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: bhyve with vlans - host and vm can't pass traffic References: <5537C5F8.1090000@sds.com> In-Reply-To: <5537C5F8.1090000@sds.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx2.shrew.net [10.24.10.11]); Wed, 22 Apr 2015 18:16:11 -0500 (CDT) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Apr 2015 23:45:04 -0000 On 4/22/2015 11:02 AM, Scott O'Connell wrote: > I'm very new to bhyve and am having an issue. I'm trying to get VM's > and VLAN's working. > > I'm able to get VLAN's working in a VM, but the VM and the VMHOST, > can't communicate with each other on the same vlan. > > Using 10.1-RELEASE-p9 for both VMHOST01 and DEV. Upstream from the > VMHOST on lagg0 is a Cisco 3750G. > > VMHOST01 before starting VM: > > bge0: flags=8843 > metric 0 mtu 1500 > options=c019b > > ether f0:1f:af:dd:2e:c5 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > bge1: flags=8843 > metric 0 mtu 1500 > options=c019b > > ether f0:1f:af:dd:2e:c5 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > lagg0: flags=8843 > metric 0 mtu 1500 > options=c019b > > ether f0:1f:af:dd:2e:c5 > nd6 options=29 > media: Ethernet autoselect > status: active > laggproto lacp lagghash l2,l3,l4 > laggport: bge1 flags=1c > laggport: bge0 flags=1c > vlan100: flags=8843 > metric 0 mtu 1500 > options=103 > ether f0:1f:af:dd:2e:c5 > inet 10.0.1.17 netmask 0xffffff00 broadcast 10.0.1.255 > nd6 options=29 > media: Ethernet autoselect > status: active > vlan: 100 parent interface: lagg0 > > VMHOST after starting VM (added tap0 & bridge0): > > tap0: flags=8902 metric 0 > mtu 1500 > options=80000 > ether 00:bd:70:71:1d:00 > nd6 options=29 > media: Ethernet autoselect > status: no carrier > bridge0: flags=8843 > metric 0 mtu 1500 > ether 02:d3:e4:02:03:00 > nd6 options=1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap0 flags=143 > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > member: lagg0 flags=143 > ifmaxaddr 0 port 4 priority 128 path cost 10000 > > > Note that the "status: no carrier" is because I hadn't brought up the > VM yet. It properly changes to the following after the VM is started: > > tap0: > flags=8943 metric 0 > mtu 1500 > options=80000 > ether 00:bd:70:71:1d:00 > nd6 options=29 > media: Ethernet autoselect > status: active > Opened by PID 70827 > > VM: > vtnet0: > flags=8943 metric 0 > mtu 1500 > options=80028 > ether 00:a0:98:2b:34:37 > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > vlan100: flags=8843 > metric 0 mtu 1500 > ether 00:a0:98:2b:34:37 > inet 10.0.1.6 netmask 0xffffff00 broadcast 10.0.1.255 > nd6 options=29 > media: Ethernet 10Gbase-T > status: active > vlan: 100 parent interface: vtnet0 > > > With this configuration, both VMHOST01 and DEV can communicate > anywhere, EXCEPT to each other using their IP on VLAN100. > > The ultimate goal is to have more than one VLAN presented to the VM, > whether it exists on the VMHOST or not. > > Where did I go wrong? > > Thanks in advance, > scotto > Scott, Have you tried creating the bridge on vlan100 device instead of lagg0 and assigning 10.0.1.6/24 directly to vtnet0 in the VM? I understand that you would prefer to do the VLAN tagging inside the VM, but have you tried it the other way just to make sure that untagged packets are being passed properly? If so, it could be that either the vtnet0 or the tap0 interface is choking on the VLAN tag. Another thing to try would be to run 'tcpdump -i tap0' in vmhost0 while the VM is trying to send packets to see if any frames are captured and, consequently, if they contain a VLAN tag at the head of the frame. -Matthew