From owner-freebsd-virtualization@freebsd.org Mon Apr 2 12:33:56 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3432F5390E for ; Mon, 2 Apr 2018 12:33:55 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A9E37C83D for ; Mon, 2 Apr 2018 12:33:55 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w32CXiJi045698; Mon, 2 Apr 2018 14:33:45 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id BCF5F3A3; Mon, 2 Apr 2018 14:33:44 +0200 (CEST) Message-ID: <5AC22328.8060306@omnilan.de> Date: Mon, 02 Apr 2018 14:33:44 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Daniel Braniss CC: freebsd-virtualization@freebsd.org Subject: Re: bhyve and arp problem References: In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Mon, 02 Apr 2018 14:33:45 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Apr 2018 12:33:56 -0000 Bezüglich Daniel Braniss's Nachricht vom 30.03.2018 13:16 (localtime): > hi, > this is my first attempt at bhyve, and so far all seems ok, except > in my guest, the mac address of the hosting keeps flipping, ie, every 20 minutes > i see a message : > … arp: nnn (the hosting ip) moved from xxxx to yyyy > on both the host and guest I’m running a very resent -stable. > the yyyy is the mac of the host nic, while the xxxx is the tap0 > > i know this looks harmless, but it’s annoying You can calm it with 'sysctl net.link.ether.inet.log_arp_movements=0' There's also "net.link.ether.inet.log_arp_wrong_iface" and "net.inet.ip.check_interface" which influence related behaviour. You also posted (documentationized IP-addresses): > I think the problem starts with the host seeing the client/guest on 2 interfaces, the nic (mlnxen0) and the tap(tap0) > on the host: > > arp -a > ... > bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on tap0 expires in 1001 seconds [ethernet] > bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on mlxen0 expires in 644 seconds [ethernet] Initially, you reference two MAC-addresses with xxxx and yyyy. The recent post indicates non-different MAC-addresses. If xxxx and yyyy - resp. xx.xx.xx.xx.xx - are equal (but seen on different interfaces), this wouldn't get logged I think. But it was the only harmless case for straight forward setups. Even with STP/LACP/CARP/etc. in place, "arp: IP-address moved" always indicates a misconfiguration and I don't know any example where the two different MAC-Addresses for one IP-address were harmless. While using a single (locally administrated?) MAC address more than once sitewide _can_ make sense, having two interfaces on one host which both are on the same ethernet segment like the two interfaces with the same MAC address, looks like an unintended setup. So I strongly suggest to analyze your setup before altering the mentioned sysctl!!! -harry