Date: Mon, 29 Dec 2014 14:11:37 +0100 From: Nikolay Denev <ndenev@gmail.com> To: Julian Elischer <julian@freebsd.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: setfib and RSTs Message-ID: <CA%2BP_MZGsX3_UB2wigrD-WWB0aN=bSf9Xxw147y2vWLAJ6_syng@mail.gmail.com> In-Reply-To: <54A0FDD9.4090009@freebsd.org> References: <CA%2BP_MZHi1hMGq8qbOvpYSMLk4Usr4CAXjR1PdKRt7dE_vMJ7UQ@mail.gmail.com> <54A0FDD9.4090009@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Dec 29, 2014 at 8:08 AM, Julian Elischer <julian@freebsd.org> wrote: > On 12/26/14 10:41 PM, Nikolay Denev wrote: > >> Hi, >> >> I have a process (bittorrent client) running in a non-default fib and >> using >> a VPN for default gateway: >> >> from /etc/rc.local : >> >> /usr/sbin/setfib 1 route add $vpn_provider 10.0.0.1 >> /usr/sbin/setfib 1 /usr/local/sbin/openvpn --config >> /usr/local/etc/openvpn/provider.ovpn >> /usr/sbin/setfib 1 /usr/sbin/service transmission onestart >> >> Then openvpn installs default gateway in fib 1 to point to the tun(4) >> interface. >> >> Stil, I'm seeing RST packets from the bittorrent client process to be sent >> not via the tunnel, but to the default gateway of the lan which seems >> wrong. As if when the kernel generates the RST it does not take into >> account the FIB of the socket? >> > it's possible that you are correct. > I checked that RST and other generated packets used the FIB for the > session if it existed when they are generated, > but I don't know what they do when a single unexpected packet enters..You > may need toset the fib in received packets using either the ipfw setfib > command or the ifconfig fib command. The RST should then use the same FIB > to respond. > > let me know.. > > > Any ideas? >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> > The weird thing was that the RST was a response to a SYN-ACK. What I saw was the transmission bittorent daemon to try to connect to some peer over tun2 (the openvpn connection), so it sends the SYN, the peer responds with SYN-ACK, then I see RST back to the peer, but not over the tun2, but over lagg0 (as originally mentioned). Here 10.8.0.31 is the tun2 address, and XX.XX.XX.XX is one of the peers that I was able to capture this happening. (both SYN-ACK and RST's are retransmitted, as the RSTs get blocked by the pf running on the default gw as it does not have state for them, as the VPN is a running on a host inside the network) # Captured on tun2: [14:03][ndenev@macbook:~]%tcpdump -vnr 1.pcap host XX.XX.XX.XX reading from file 1.pcap, link-type NULL (BSD loopback) 20:43:39.623920 IP (tos 0x0, ttl 64, id 12450, offset 0, flags [DF], proto TCP (6), length 60) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [S], cksum 0xace8 (correct), seq 2985341180, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val 515982338 ecr 0], length 0 20:43:39.912558 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5a3d (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207923992 ecr 515982338,nop,wscale 7], length 0 20:43:41.132751 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x590a (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207924299 ecr 515982338,nop,wscale 7], length 0 20:43:43.141474 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x5716 (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207924799 ecr 515982338,nop,wscale 7], length 0 20:43:47.129657 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x532e (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207925799 ecr 515982338,nop,wscale 7], length 0 20:43:55.133735 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x4b5e (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207927799 ecr 515982338,nop,wscale 7], length 0 20:44:11.134635 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.60633: Flags [S.], cksum 0x3bbe (correct), seq 809114142, ack 2985341181, win 28960, options [mss 1368,sackOK,TS val 207931799 ecr 515982338,nop,wscale 7], length 0 20:45:11.290740 IP (tos 0x0, ttl 64, id 61675, offset 0, flags [DF], proto TCP (6), length 60) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [S], cksum 0xdbfb (correct), seq 2733189104, win 8192, options [mss 1460,nop,wscale 6,sackOK,TS val 516074004 ecr 0], length 0 20:45:11.595504 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x81f1 (correct), seq 2072870049, ack 2733189105, win 28960, options [mss 1368,sackOK,TS val 207946912 ecr 516074004,nop,wscale 7], length 0 20:45:12.748574 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x80cf (correct), seq 2072870049, ack 2733189105, win 28960, options [mss 1368,sackOK,TS val 207947202 ecr 516074004,nop,wscale 7], length 0 20:45:14.746692 IP (tos 0x0, ttl 51, id 0, offset 0, flags [DF], proto TCP (6), length 60) XX.XX.XX.XX.51413 > 10.8.0.31.61382: Flags [S.], cksum 0x7edb (correct), seq 2072870049, ack 2733189105, win 28960, options [mss 1368,sackOK,TS val 207947702 ecr 516074004,nop,wscale 7], length 0 # Captured on lagg0: [14:04][ndenev@macbook:~]%tcpdump -vnr 2.pcap host XX.XX.XX.XX reading from file 2.pcap, link-type EN10MB (Ethernet) 20:43:39.912570 IP (tos 0x0, ttl 64, id 12524, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->a2c)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:41.132761 IP (tos 0x0, ttl 64, id 13085, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->7fb)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:43.141485 IP (tos 0x0, ttl 64, id 14702, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->1aa)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:47.129669 IP (tos 0x0, ttl 64, id 16915, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->f904)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:43:55.133748 IP (tos 0x0, ttl 64, id 23534, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->df29)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:44:11.134645 IP (tos 0x0, ttl 64, id 30066, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->c5a5)!) 10.8.0.31.60633 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x978a), seq 2985341181, win 0, length 0 20:45:11.595514 IP (tos 0x0, ttl 64, id 62112, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->4877)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:12.748583 IP (tos 0x0, ttl 64, id 62741, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->4602)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:14.746703 IP (tos 0x0, ttl 64, id 63874, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->4195)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:18.754657 IP (tos 0x0, ttl 64, id 65318, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->3bf1)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:26.759522 IP (tos 0x0, ttl 64, id 3473, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->2d87)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 20:45:42.758524 IP (tos 0x0, ttl 64, id 10814, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 0 (->10da)!) 10.8.0.31.61382 > XX.XX.XX.XX.51413: Flags [R], cksum 0xffd2 (incorrect -> 0x2cb1), seq 2733189105, win 0, length 0 [14:04][ndenev@macbook:~]% I'll try the IPFW workaround, I'm not seeing this at the moment (looks like it was related to how many active torrents there were, as now there are none). --Nikolay
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BP_MZGsX3_UB2wigrD-WWB0aN=bSf9Xxw147y2vWLAJ6_syng>