From owner-freebsd-net@freebsd.org Tue Aug 22 08:39:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E0D6DD8E0F for ; Tue, 22 Aug 2017 08:39:35 +0000 (UTC) (envelope-from borisbsd@gmail.com) Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CABDE7C35B for ; Tue, 22 Aug 2017 08:39:34 +0000 (UTC) (envelope-from borisbsd@gmail.com) Received: by mail-qk0-x236.google.com with SMTP id k126so10677634qkb.5 for ; Tue, 22 Aug 2017 01:39:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=jlgMEOgHm9mCfHzQH75oP1zR7Pxj2FFRNip6kPJAPjo=; b=iJ2TajMjCwPG8ggi3BAPLswdVuDiPZXfRiT+kRyXWs3/bRT+D1uKiMN8S5h++MdUeG +Q7XrOXFyiT8+3xj93HULkowST/KURJfh3M3fUyetu8/wOV1oga8Q/1hrzsgvjIjl46u PTyOaVE6/3L+SzQEVMaxZBe2Xy6LJjSa2rY2MXr1VdC99kTFylhziw21XdgX9AqDdfWH B6u0nkJE1RoehYg5Jgnk+Xh6W6qHwioBwl1VlMVN1kwaVykwi95fcEIkRMNLs9uleHL7 OQZ+75e6MIvl2VEVeIOeK59jEFBIU+FDO4xqJKdPFA/kOAHKjZ0pz34C/+ap5RJbkOwA c+MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=jlgMEOgHm9mCfHzQH75oP1zR7Pxj2FFRNip6kPJAPjo=; b=Ody+AkXSLmpwXDt82Eg7MBanUjGRDltASnLd8xlrUEpGPsN+HWo/DviKQjgHMf1NN8 EEJcdqcSrnq7xRbMs2E+E/Tt/ClPVXFRGivEEdCEbuDeSi3RFam8W578nMpBqrT6MAgj qgGwvYnvnLF+cHoR7+GnWfXf8reRPAqhL7Z15VphRaeP7bg5JIpO1d60bTIcvKDEVEju 5TeDMKv/v6ZGDyIaUk06EkvHiKY+FOGpaEgBQjD6n82TU1kXF5OZSPOJOjUke1x2Y80E qQZ5ofYWkIVAGCAr4U2THpsUeKLX40XcQ2ey0GgwFqbIUqGIoSxON1KIR3JMhsryDhUC s1ug== X-Gm-Message-State: AHYfb5irXxVyzwbtXG//mCayK89F2kYO75D7nBp9dM7fKjHsdvm2Jsdm 0tWd3KxduB7KtfiCswLrdBjnpYnE0Hkz X-Received: by 10.55.221.155 with SMTP id u27mr25911301qku.242.1503391173775; Tue, 22 Aug 2017 01:39:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.82.166 with HTTP; Tue, 22 Aug 2017 01:39:33 -0700 (PDT) In-Reply-To: <599B8576.8030801@grosbein.net> References: <599B8576.8030801@grosbein.net> From: Boris Date: Tue, 22 Aug 2017 04:39:33 -0400 Message-ID: Subject: Re: bridge interface IP connectivity issue when using oce interface To: Eugene Grosbein Cc: freebsd-net@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Aug 2017 08:39:35 -0000 Ok thanks Eugene. net.link.bridge.inherit_mac=3D1 helped get the connectivity from the bridge however, when I start a FreeBSD bhyve VM and attached that to a tap interface in the bridge, I don't get connectivity from the VM. SETUP: Gateway - 192.168.0.222/29 Server - 192.168.0.218/29 VM - 192.168.0.219/29 On the VM, I see the ARP entries for the GW and the VM itself but cannot ping the gateway nor the host. --------- on the VM -------------- # uname -a FreeBSD 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 # ifconfig vtnet0: flags=3D8943 metric= 0 mtu 1500 options=3D80028 ether 00:a0:98:52:c8:33 hwaddr 00:a0:98:52:c8:33 inet 192.168.0.219 netmask 0xfffffff8 broadcast 192.168.0.223 nd6 options=3D29 media: Ethernet 10Gbase-T status: active # ping -c4 192.168.0.222 PING 192.168.0.222 (192.168.0.222): 56 data bytes ^C --- 192.168.0.222 ping statistics --- 4 packets transmitted, 0 packets received, 100.0% packet loss # arp -an ? (192.168.0.219) at 00:a0:98:52:c8:33 on vtnet0 permanent [ethernet] ? (192.168.0.218) at (incomplete) on vtnet0 expired [ethernet] ? (192.168.0.222) at 00:08:e3:ff:fd:90 on vtnet0 expires in 1126 seconds [ethernet] ----------- end of VM ---------------- ----------- on the host --------------- root@bsdcan:~ # uname -a FreeBSD bsdcan 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GEN= ERIC amd64 root@bsdcan:~ # ifconfig [..] oce3: flags=3D8143 metric 0 mtu 150= 0 options=3D500b9 ether 90:1b:0e:98:d3:93 hwaddr 90:1b:0e:98:d3:93 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active [..] tap0: flags=3D8942 metric 0 mt= u 1500 options=3D80000 ether 00:bd:0f:bb:27:00 hwaddr 00:bd:0f:bb:27:00 nd6 options=3D29 media: Ethernet autoselect status: active groups: tap Opened by PID 81874 bridge0: flags=3D8843 metric 0 mtu 1500 ether 90:1b:0e:98:d3:93 inet 192.168.0.218 netmask 0xfffffff8 broadcast 192.168.0.223 nd6 options=3D9 groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=3D143 ifmaxaddr 0 port 7 priority 128 path cost 55 member: oce3 flags=3D143 ifmaxaddr 0 port 4 priority 128 path cost 2000 root@bsdcan:~ # ifconfig bridge0 addr 00:08:e3:ff:fd:90 Vlan1 oce3 1200 flags=3D0<> root@bsdcan:~ # ps aux | grep vmrun root 47167 0.0 0.0 14828 2396 1 S+ 04:08 0:00.00 grep vmrun root 73264 0.0 0.0 13180 2740 2 I+ 03:39 0:00.00 sh /usr/share/examples/bhyve/vmrun.sh -c 10 -m 8192M -t tap0 -d guest.img -i -I FreeBSD-11.1-RELEASE-amd root@bsdcan:~ # arp -an ? (192.168.0.218) at 90:1b:0e:98:d3:93 on bridge0 permanent [bridge] ? (192.168.0.222) at 00:08:e3:ff:fd:90 on bridge0 expires in 1191 seconds [bridge] root@bsdcan:~ # sysctl net.link.bridge net.link.bridge.ipfw: 0 net.link.bridge.allow_llz_overlap: 1 net.link.bridge.inherit_mac: 1 net.link.bridge.log_stp: 0 net.link.bridge.pfil_local_phys: 1 net.link.bridge.pfil_member: 0 net.link.bridge.ipfw_arp: 0 net.link.bridge.pfil_bridge: 1 net.link.bridge.pfil_onlyip: 1 --------------- end of host ---------- Shouldn't the VM mac address show up in the MAC address table of the bridge0 ? When I 'tcpdump -i tap0 -vv' I see literally only the ARP request from the .222 towards the VM and nothing back from the VM at all which does not make a lot of sense since I get the '-t tap0' when launching the VM, I would expect some traffic on the tap0 intf from the VM. Any thoughts on where filtering could happen? I assume the VM should be able to ping the IP set on the bridge0. Is it fair assumption? Thanks. On Mon, Aug 21, 2017 at 9:14 PM, Eugene Grosbein wrote= : > 22.08.2017 7:49, Boris =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > > Hi all, > > > > I have two environments. > > > > Environment A: > > Server running fresh install of 11.1-RELEASE with bge physical NIC. > > If I just configure a bridge interface, add a physical NIC which has > > working connectivity, say bge3, and add an IP address on the bridge > > interface in the same subnet as bge3, I can ping that IP from any host = on > > the LAN. > > > > Environment B: > > Server running fresh install of 11.1-RELEASE with oce physical NIC. > > If I just configure a bridge interface, add a physical NIC which has > > working connectivity, say oce3, and add an IP address on the bridge > > interface in the same subnet as oce3, I CANNOT ping that IP from anywhe= re > > on the LAN. > > First, when you add member interfaces to a bridge, you should move all > their > IP addresses to the bridge. That is, bridge member interfaces should > have no IP addresses, only bridge itself. > > Second, you should re-read bridge(4) manual page and use > sysctl net.link.bridge.inherit_mac=3D1 and use physical NIC as first > bridge member so that your uplink has no reasons to filter > traffic of the bridge due to its fabricated MAC. > > > > > > I need the bridge as I would like to have bhyve VM's connected through > that > > bridge to the outside - plain bridged networking, no NAT or anything > else. > > Unfortunately, the VM does not have any connectivity to the outside. Wh= at > > is weird is that I see the ARP entries in the VM for its gateway, I see > the > > MAC addresses in the bridge for the VM and the gateway, but no IP > > connectivity seems to work - ping fails. > > I disabled TX checksum and other things using 'ifconfig oce3 -txcsum -l= ro > > -tso' to avoid messages around capabilities issues when adding a tap > > interface which does not have the same features as the physical > interface. > > So far, I have not been able to get IP connectivity to the VM. > > > > In terms of documentation, I have used the handbook to create the VM: > > https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html > > > > I have used the handbook to create the bridge: > > https://www.freebsd.org/doc/handbook/network-bridging.html > > > > Under 30.6.1, it says, I should be able to configure the bridge with an > IP > > address which seems to fail when the 'oce' interface is used. > > > > Would anybody have any pointer at what to do next to help identify the > > issue? > > > > Thanks ! > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > >