Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 2 Apr 2018 09:07:56 -0700 (PDT)
From:      "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
To:        Daniel Braniss <danny@cs.huji.ac.il>
Cc:        Harry Schmalzbauer <freebsd@omnilan.de>, freebsd-virtualization@freebsd.org
Subject:   Re: bhyve and arp problem
Message-ID:  <201804021607.w32G7uFa083223@pdx.rh.CN85.dnsmgr.net>
In-Reply-To: <BCDB257E-3CA3-48FB-8340-05DE52638904@cs.huji.ac.il>

next in thread | previous in thread | raw e-mail | index | archive | help
> > On 2 Apr 2018, at 16:51, Rodney W. Grimes <freebsd-rwg@pdx.rh.CN85.dnsmgr.net> wrote:
> > 
> >>> On 2 Apr 2018, at 15:33, Harry Schmalzbauer <freebsd@omnilan.de> wrote:
> >>> 
> >>> Bez?glich Daniel Braniss's Nachricht vom 30.03.2018 13:16 (localtime):
> >>>> hi,
> >>>> this is my first attempt at bhyve, and so far all seems ok, except
> >>>> in my guest, the mac address of the hosting keeps flipping, ie, every 20 minutes
> >>>> i see a message :
> >>>> 	? arp: nnn (the hosting ip)  moved from xxxx to yyyy
> >>>> on both the host and guest I?m running a very resent -stable.
> >>>> the yyyy is the mac of the host nic, while the xxxx is the tap0
> >>>> 
> >>>> i know this looks harmless, but it?s annoying
> >>> 
> >>> You can calm it with
> >>> 'sysctl net.link.ether.inet.log_arp_movements=0'
> >>> 
> >>> There's also "net.link.ether.inet.log_arp_wrong_iface" and
> >>> "net.inet.ip.check_interface" which influence related behaviour.
> >>> 
> >>> You also posted (documentationized IP-addresses):
> >>>> I think the problem starts with the host seeing the client/guest on 2 interfaces, the nic (mlnxen0) and the tap(tap0)
> >>>> on the host:
> >>>> 
> >>>> arp -a
> >>>> ...
> >>>> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on tap0 expires in 1001 seconds [ethernet]
> >>>> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on mlxen0 expires in 644 seconds [ethernet]
> >>> 
> >> 
> >> the above 2 lines are on the host running bhyve (server?) and the MACs belong to the client, and they are identical,
> >> there is no complaints.
> >> (BTW, did you change the ip?s?)
> >> 
> >>> Initially, you reference two MAC-addresses with xxxx and yyyy.
> >> this is on the client, where the MAC are different (it?s of the hosting computer).
> >> 
> >>> The recent post indicates non-different MAC-addresses.
> >>> 
> >>> If xxxx and yyyy - resp. xx.xx.xx.xx.xx - are equal (but seen on
> >>> different interfaces), this wouldn't get logged I think.
> >>> But it was the only harmless case for straight forward setups.
> >>> Even with STP/LACP/CARP/etc. in place, "arp: IP-address moved" always
> >>> indicates a misconfiguration and I don't know any example where the two
> >>> different MAC-Addresses for one IP-address were harmless.
> >>> While using a single (locally administrated?) MAC address more than once
> >>> sitewide _can_ make sense, having two interfaces on one host which both
> >>> are on the same ethernet segment like the two interfaces with the same
> >>> MAC address, looks like an unintended setup.
> >>> 
> >>> So I strongly suggest to analyze your setup before altering the
> >>> mentioned sysctl!!!
> >>> 
> >> I do want to know if there are ip/mac issues, it usually happens when more than one host has the same ip,
> >> which is not the case here :-(
> > 
> > Are you trying to use the HOSTS ip address in the GUEST?
> 
> the client is using the server?s /usr/local, which is mounted via nfs.
> so I guess the answer is yes.

I didnt mean to access it, I meant did you assign the same IP to
the GUEST that is assigned on the HOST.  I suspect the answer
here is no from other context.

> > And how do you have an mlxen interface in a GUEST?
> no
> the guest has only vtnet0 and lo0
> the ip of the client is obtained via dhcp
> on the server, there is a bridge, bridge0 and it bridges between the taps and the mxlen0

Is 192.0.2.246 on the mlxen0 interface, or on the bridge0?
I believe you need to move the ip from the interface to the bridge to have this
work right.

> 
> > Is this being done with PCI passthrough?
> again, no.
> 
> cheers,
> 	danny

-- 
Rod Grimes                                                 rgrimes@freebsd.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201804021607.w32G7uFa083223>