From owner-freebsd-virtualization@freebsd.org Mon Apr 2 13:08:02 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E430F57B4A for ; Mon, 2 Apr 2018 13:08:02 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F7317D8E1 for ; Mon, 2 Apr 2018 13:08:00 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bk.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1f2zBd-000914-UD; Mon, 02 Apr 2018 16:07:49 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: bhyve and arp problem From: Daniel Braniss In-Reply-To: <5AC22328.8060306@omnilan.de> Date: Mon, 2 Apr 2018 16:07:49 +0300 Cc: freebsd-virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <5AC22328.8060306@omnilan.de> To: Harry Schmalzbauer X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Apr 2018 13:08:02 -0000 > On 2 Apr 2018, at 15:33, Harry Schmalzbauer = wrote: >=20 > Bez=C3=BCglich Daniel Braniss's Nachricht vom 30.03.2018 13:16 = (localtime): >> hi, >> this is my first attempt at bhyve, and so far all seems ok, except >> in my guest, the mac address of the hosting keeps flipping, ie, every = 20 minutes >> i see a message : >> =E2=80=A6 arp: nnn (the hosting ip) moved from xxxx to yyyy >> on both the host and guest I=E2=80=99m running a very resent -stable. >> the yyyy is the mac of the host nic, while the xxxx is the tap0 >>=20 >> i know this looks harmless, but it=E2=80=99s annoying >=20 > You can calm it with > 'sysctl net.link.ether.inet.log_arp_movements=3D0' >=20 > There's also "net.link.ether.inet.log_arp_wrong_iface" and > "net.inet.ip.check_interface" which influence related behaviour. >=20 > You also posted (documentationized IP-addresses): >> I think the problem starts with the host seeing the client/guest on 2 = interfaces, the nic (mlnxen0) and the tap(tap0) >> on the host: >>=20 >> arp -a >> ... >> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on tap0 expires = in 1001 seconds [ethernet] >> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on mlxen0 = expires in 644 seconds [ethernet] >=20 the above 2 lines are on the host running bhyve (server?) and the MACs = belong to the client, and they are identical, there is no complaints. (BTW, did you change the ip=E2=80=99s?) > Initially, you reference two MAC-addresses with xxxx and yyyy. this is on the client, where the MAC are different (it=E2=80=99s of the = hosting computer). > The recent post indicates non-different MAC-addresses. >=20 > If xxxx and yyyy - resp. xx.xx.xx.xx.xx - are equal (but seen on > different interfaces), this wouldn't get logged I think. > But it was the only harmless case for straight forward setups. > Even with STP/LACP/CARP/etc. in place, "arp: IP-address moved" always > indicates a misconfiguration and I don't know any example where the = two > different MAC-Addresses for one IP-address were harmless. > While using a single (locally administrated?) MAC address more than = once > sitewide _can_ make sense, having two interfaces on one host which = both > are on the same ethernet segment like the two interfaces with the same > MAC address, looks like an unintended setup. >=20 > So I strongly suggest to analyze your setup before altering the > mentioned sysctl!!! >=20 I do want to know if there are ip/mac issues, it usually happens when = more than one host has the same ip, which is not the case here :-( thanks, danny > -harry