From owner-freebsd-virtualization@freebsd.org Mon Apr 2 13:52:09 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59B94F6B60F for ; Mon, 2 Apr 2018 13:52:09 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC8CC7FBB8 for ; Mon, 2 Apr 2018 13:52:08 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w32Dpt3n082638; Mon, 2 Apr 2018 06:51:55 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w32DptK9082637; Mon, 2 Apr 2018 06:51:55 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201804021351.w32DptK9082637@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve and arp problem In-Reply-To: To: Daniel Braniss Date: Mon, 2 Apr 2018 06:51:55 -0700 (PDT) CC: Harry Schmalzbauer , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Apr 2018 13:52:09 -0000 > > On 2 Apr 2018, at 15:33, Harry Schmalzbauer wrote: > > > > Bez?glich Daniel Braniss's Nachricht vom 30.03.2018 13:16 (localtime): > >> hi, > >> this is my first attempt at bhyve, and so far all seems ok, except > >> in my guest, the mac address of the hosting keeps flipping, ie, every 20 minutes > >> i see a message : > >> ? arp: nnn (the hosting ip) moved from xxxx to yyyy > >> on both the host and guest I?m running a very resent -stable. > >> the yyyy is the mac of the host nic, while the xxxx is the tap0 > >> > >> i know this looks harmless, but it?s annoying > > > > You can calm it with > > 'sysctl net.link.ether.inet.log_arp_movements=0' > > > > There's also "net.link.ether.inet.log_arp_wrong_iface" and > > "net.inet.ip.check_interface" which influence related behaviour. > > > > You also posted (documentationized IP-addresses): > >> I think the problem starts with the host seeing the client/guest on 2 interfaces, the nic (mlnxen0) and the tap(tap0) > >> on the host: > >> > >> arp -a > >> ... > >> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on tap0 expires in 1001 seconds [ethernet] > >> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on mlxen0 expires in 644 seconds [ethernet] > > > > the above 2 lines are on the host running bhyve (server?) and the MACs belong to the client, and they are identical, > there is no complaints. > (BTW, did you change the ip?s?) > > > Initially, you reference two MAC-addresses with xxxx and yyyy. > this is on the client, where the MAC are different (it?s of the hosting computer). > > > The recent post indicates non-different MAC-addresses. > > > > If xxxx and yyyy - resp. xx.xx.xx.xx.xx - are equal (but seen on > > different interfaces), this wouldn't get logged I think. > > But it was the only harmless case for straight forward setups. > > Even with STP/LACP/CARP/etc. in place, "arp: IP-address moved" always > > indicates a misconfiguration and I don't know any example where the two > > different MAC-Addresses for one IP-address were harmless. > > While using a single (locally administrated?) MAC address more than once > > sitewide _can_ make sense, having two interfaces on one host which both > > are on the same ethernet segment like the two interfaces with the same > > MAC address, looks like an unintended setup. > > > > So I strongly suggest to analyze your setup before altering the > > mentioned sysctl!!! > > > I do want to know if there are ip/mac issues, it usually happens when more than one host has the same ip, > which is not the case here :-( Are you trying to use the HOSTS ip address in the GUEST? And how do you have an mlxen interface in a GUEST? Is this being done with PCI passthrough? -- Rod Grimes rgrimes@freebsd.org