Date: Fri, 4 Mar 2016 11:16:50 -0800 From: Julian Elischer <julian@freebsd.org> To: freebsd-net@freebsd.org, Pakhom Golynga <pakhom706@gmail.com> Subject: Re: routing issue Message-ID: <56D9DF22.2080504@freebsd.org> In-Reply-To: <56D81418.9020500@pakhom.spb.ru> References: <56D81418.9020500@pakhom.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On 3/03/2016 2:38 AM, Pakhom Golynga wrote: > Hello all! > > Please help me to investigate this issue. > I have problem on 10.2-RELEASE-p12 with multiple network interfaces > and PF (rules, NAT) > # ifconfig > <--cut--> > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO> > > ether 00:25:90:64:14:50 > inet 172.27.27.254 netmask 0xffffff00 broadcast 172.27.27.255 > inet 172.27.27.240 netmask 0xffffffff broadcast 172.27.27.240 > inet 172.27.27.252 netmask 0xffffffff broadcast 172.27.27.252 > inet 172.27.27.251 netmask 0xffffffff broadcast 172.27.27.251 > inet 172.27.27.250 netmask 0xffffffff broadcast 172.27.27.250 > inet 172.27.27.249 netmask 0xffffffff broadcast 172.27.27.249 > inet 172.27.27.248 netmask 0xffffffff broadcast 172.27.27.248 > inet 172.27.27.247 netmask 0xffffffff broadcast 172.27.27.247 > inet 172.27.27.246 netmask 0xffffffff broadcast 172.27.27.246 > inet 172.27.27.245 netmask 0xffffffff broadcast 172.27.27.245 > inet 172.27.27.244 netmask 0xffffffff broadcast 172.27.27.244 > inet 172.27.27.243 netmask 0xffffffff broadcast 172.27.27.243 > inet 172.27.27.242 netmask 0xffffffff broadcast 172.27.27.242 > inet 172.27.27.241 netmask 0xffffffff broadcast 172.27.27.241 > inet 172.27.27.239 netmask 0xffffffff broadcast 172.27.27.239 > inet 172.27.27.238 netmask 0xffffffff broadcast 172.27.27.238 > inet 172.27.27.237 netmask 0xffffffff broadcast 172.27.27.237 > inet 172.27.27.236 netmask 0xffffffff broadcast 172.27.27.236 > inet 172.27.27.235 netmask 0xffffffff broadcast 172.27.27.235 > media: Ethernet autoselect (1000baseT <full-duplex>) > status: active > em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> > metric 0 mtu 1500 > options=4209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO> > > ether 00:25:90:64:14:51 > inet 10.24.44.50 netmask 0xfffffc00 broadcast 10.24.47.255 > media: Ethernet autoselect (100baseTX <full-duplex>) > status: active > <--cut--> > > em0 - local network. Aliases on em0 for Jail's > em1 - connected to ISP > > This configuration successfully operate a long time but recently > (for no apparent reason) suddenly became manifest bug with routing. > Regardless from uptime (sometimes 12 hours, and maybe a couple of > days) after a certain time the traffic running between jails begins > to route via em1 (instead lo0 (net.link.ether.inet.useloopback=1)): this sounds much like another report we have seen. you may be able to fix it by re-adding your routes.. The report I have seen suggested that some routes seem to stop being effective after some period. It's still not understood. > # tcpdump -n -e -ttt -i em1 host 172.27.27.252 or host 172.27.27.247 > or host 172.27.27.248 > 00:00:00.054989 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25: > Flags [S], seq 2075254199, win 65535, options [mss 16344,nop,wscale > 6,sackOK,TS val 161568673 ecr 0], length 0 > 00:00:00.318036 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.60176: > Flags [.], ack 772660872, win 1275, options [nop,nop,TS val > 1737287913 ecr 161568991], length 0 > 00:00:00.036971 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.33080: > Flags [.], ack 1, win 1275, options [nop,nop,TS val 1701623130 ecr > 161568238], length 0 > 00:00:00.050001 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 66: 172.27.27.247.80 > 172.27.27.248.40200: > Flags [.], ack 1, win 1275, options [nop,nop,TS val 1477098721 ecr > 161568288], length 0 > 00:00:02.794992 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25: > Flags [S], seq 2075254199, win 65535, options [mss 16344,nop,wscale > 6,sackOK,TS val 161571873 ecr 0], length 0 > 00:00:03.312026 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 74: 172.27.27.247.29581 > 172.27.27.252.25: > Flags [S], seq 1217253021, win 65535, options [mss 16344,nop,wscale > 6,sackOK,TS val 161575185 ecr 0], length 0 > 00:00:02.887982 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 74: 172.27.27.247.24868 > 172.27.27.252.25: > Flags [S], seq 2075254199, win 65535, options [mss 16344,nop,wscale > 6,sackOK,TS val 161578073 ecr 0], length 0 > 00:00:00.111989 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 74: 172.27.27.247.29581 > 172.27.27.252.25: > Flags [S], seq 1217253021, win 65535, options [mss 16344,nop,wscale > 6,sackOK,TS val 161578185 ecr 0], length 0 > 00:00:03.471007 00:25:90:64:14:51 > 68:bd:ab:5f:ea:d8, ethertype > IPv4 (0x0800), length 108: 172.27.27.247.80 > 172.27.27.248.26056: > Flags [FP.], seq 1448:1490, ack 1, win 1275, options [nop,nop,TS val > 2142270801 ecr 161575507], length 42 > > In the same time: > # route -vn get -host > RTA_DST: inet 172.27.27.247; RTA_IFP: link ; RTM_GET: Report > Metrics: len 224, pid: 0, seq 1, errno 0, > flags:<UP,GATEWAY,HOST,STATIC> > locks: inits: > sockaddrs: <DST,IFP> > 172.27.27.247 link#0 > route to: 172.27.27.247 > destination: 172.27.27.247 > fib: 0 > interface: lo0 > flags: <UP,HOST,DONE,STATIC> > recvpipe sendpipe ssthresh rtt,msec mtu weight expire > 0 0 0 0 16384 1 0 > > locks: inits: > sockaddrs: <DST,GATEWAY,IFP,IFA> > 172.27.27.247 link#4 lo0 127.0.0.1 > RTA_DST: inet 172.27.27.250; RTA_IFP: link ; RTM_GET: Report > Metrics: len 224, pid: 0, seq 1, errno 0, > flags:<UP,GATEWAY,HOST,STATIC> > locks: inits: > sockaddrs: <DST,IFP> > 172.27.27.250 link#0 > route to: 172.27.27.250 > destination: 172.27.27.250 > fib: 0 > interface: lo0 > flags: <UP,HOST,DONE,STATIC> > recvpipe sendpipe ssthresh rtt,msec mtu weight expire > 0 0 0 0 16384 1 0 > > locks: inits: > sockaddrs: <DST,GATEWAY,IFP,IFA> > 172.27.27.250 link#4 lo0 127.0.0.1 > RTA_DST: inet 172.27.27.252; RTA_IFP: link ; RTM_GET: Report > Metrics: len 224, pid: 0, seq 1, errno 0, > flags:<UP,GATEWAY,HOST,STATIC> > locks: inits: > sockaddrs: <DST,IFP> > 172.27.27.252 link#0 > route to: 172.27.27.252 > destination: 172.27.27.252 > fib: 0 > interface: lo0 > flags: <UP,HOST,DONE,STATIC> > recvpipe sendpipe ssthresh rtt,msec mtu weight expire > 0 0 0 0 16384 1 0 > > locks: inits: > sockaddrs: <DST,GATEWAY,IFP,IFA> > 172.27.27.252 link#4 lo0 127.0.0.1 > > > # netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > default 10.24.44.1 UGS em1 > 10.24.44.0/22 link#5 U em1 > 10.24.44.50 link#5 UHS lo0 > 127.0.0.1 link#7 UH lo0 > 172.27.27.0/24 link#4 U em0 > 172.27.27.235 link#4 UHS lo0 > 172.27.27.235/32 link#4 U em0 > 172.27.27.236 link#4 UHS lo0 > 172.27.27.236/32 link#4 U em0 > 172.27.27.237 link#4 UHS lo0 > 172.27.27.237/32 link#4 U em0 > 172.27.27.238 link#4 UHS lo0 > 172.27.27.238/32 link#4 U em0 > 172.27.27.239 link#4 UHS lo0 > 172.27.27.239/32 link#4 U em0 > 172.27.27.240 link#4 UHS lo0 > 172.27.27.240/32 link#4 U em0 > 172.27.27.241 link#4 UHS lo0 > 172.27.27.241/32 link#4 U em0 > 172.27.27.242 link#4 UHS lo0 > 172.27.27.242/32 link#4 U em0 > 172.27.27.243 link#4 UHS lo0 > 172.27.27.243/32 link#4 U em0 > 172.27.27.244 link#4 UHS lo0 > 172.27.27.244/32 link#4 U em0 > 172.27.27.245 link#4 UHS lo0 > 172.27.27.245/32 link#4 U em0 > 172.27.27.246 link#4 UHS lo0 > 172.27.27.246/32 link#4 U em0 > 172.27.27.247 link#4 UHS lo0 > 172.27.27.247/32 link#4 U em0 > 172.27.27.248 link#4 UHS lo0 > 172.27.27.248/32 link#4 U em0 > 172.27.27.249 link#4 UHS lo0 > 172.27.27.249/32 link#4 U em0 > 172.27.27.250 link#4 UHS lo0 > 172.27.27.250/32 link#4 U em0 > 172.27.27.251 link#4 UHS lo0 > 172.27.27.251/32 link#4 U em0 > 172.27.27.252 link#4 UHS lo0 > 172.27.27.252/32 link#4 U em0 > 172.27.27.254 link#4 UHS lo0 > 172.27.28.0/24 link#8 U bridge0 > 172.27.28.254 link#8 UHS lo0 > 172.27.30.0/24 172.27.30.2 UGS tun0 > 172.27.30.1 link#12 UHS lo0 > 172.27.30.2 link#12 UH tun0 > > After the reboot everything works well for some time. > > Thanks! > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56D9DF22.2080504>