Date: Sat, 6 Jan 2018 01:24:00 +0530 From: Reshad Patuck <reshadpatuck1@gmail.com> To: freebsd-net@freebsd.org Subject: [vnet][epair] epair interface stops working after some time Message-ID: <CADaJeD2LZy=RU0vtqD7%2BdkZkUs0GKW%2B7duGDQkZ19GR-_cS=MQ@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hey,
I am having a strange issue with one of my servers.
I have a couple of VNET jails FreeBSD 12 r321619 set up using if_bridge and
epairs.
Each VNET jail (and the host too) has a pf firewall limiting inbound
traffic.
Everything works as intended for some time (1-5 days), services inside the
jail work and the jail can connect out to the rest of the network.
After some time of working fine I suddenly find that the jails stop
receiving traffic and can not send traffic out.
Essentially the traffic on one end of the epair does not come out the other.
I have linked to a diagram with my network setup for the jails.
Essentially the same setup is running on another identical server at
another location and has been running for atleast two weeks without any
issues.
The symptoms are as follows:
- I can connect to the server via ssh (on igb0 at IP 192.168.1.50).
- All connections from outside the jails work fine from (192.168.1.50 to
external IPs)
- I can not connect to any services running inside the jails from either
outside or inside the server
- I can not connect out from the jails (jexec in to the jails and then
attempt to connect out)
- When I attempt to connect out from one of the jails:
- I see arp traffic (via tcpdump) on the epair inside the jail (epair0b)
- I cant see the same arp traffic (via tcpdump) on the epair outside
the jail (epair0a)
- 'arp -a' insde the jails shows incomplete arps for any external IP I
try to reach.
- When I tcpdump on igb0, bridge0 or epair0a I see
broadcast/multicast/general network traffic.
- When I tcpdump on epair0b I see no traffic at all.
I have done the following on both servers to test what happens:
- Created a new epair interface epair3a and epair3b
- upped both interfaces
- given epair3a IP address 10.20.30.40/24 (I don't have this subnet
anywhere in my network)
- attempted to ping 10.20.30.50
- checked for any packets on epair3b
On the server where epairs are working, I can see APR packets for
10.20.30.50, but on the server where epairs are not working I cant see any
packets on epair3b.
I can however see the arp packets on epair3a on both servers.
This is the third time I have found this on the same server and the other
server is still going strong.
After rebooting the server this problem seems to go away temporarily, but
seems to manifest itself again after some time.
Any commands, ideas, thoughts on how to troubleshoot what is wrong here
will be much appreciated.
Please let me know if there is anything I can do the debug this issue or if
you need any other information.
Thanks and best regards,
Reshad
Link to network diagram: https://i.imgur.com/1XdRjt0.jpg
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADaJeD2LZy=RU0vtqD7%2BdkZkUs0GKW%2B7duGDQkZ19GR-_cS=MQ>