Date: Sun, 23 Oct 2005 05:16:28 +1000 From: Michael VInce <mv@roq.com> To: Volker <volker@vwsoft.com> Cc: freebsd-net@freebsd.org Subject: Re: IPSec tcp session stalling Message-ID: <435A900C.3060602@roq.com> In-Reply-To: <435A5D9B.7080309@vwsoft.com> References: <435A5D9B.7080309@vwsoft.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Try sending different sized pings or other packet size control utils to really make sure its not MTU related. Maybe there is an upstream router thats blocking ICMP fragment packets, have you ever seen them? try forcing the creation of some. Mike Volker wrote: >Still having the same problem with an IPSec tunnel between FreeBSD 5.4R >hosts. > >Problem description: >scp session tries to transfer a large file through an IPSec tunnel. The >file is being transmitted but scp says 'stalled' after 56K (49152 bytes >file size). The IPSec tunnel itself is still up even after the scp >abort. Other tcp sessions break, too when sending too much traffic >through the tunnel. > >I've taken a closer look to it and tried to get something useful out of >the tcpdump but I'm unable to see any errors or I'm misinterpreting >something. > >The connection looks like: > >extIP: A.B.C.D >extIP: E.F.G.H >host A ------------------ (internet) ------------------ host B >tunnelIP: 10.128.1.6 tunnelIP: >10.128.6.1 > >host A just has an external interface (em1) connected to a leased line >with a fixed IP address (IP-addr A.B.C.D). >host B has an S-DSL connection at xl0, PPPoE at ng0 (IP-addr. E.F.G.H). > >Both hosts are using gif for the IPSec tunnel. > >The routing tables (netstat -rnWf inet) are looking good and IMHO the >MTU is fine. > >host A: >em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > options=b<RXCSUM,TXCSUM,VLAN_MTU> > inet A.B.C.D netmask 0xfffffff8 broadcast A.B.C.z > ether 00:c0:9f:46:ec:c7 > media: Ethernet autoselect (100baseTX <full-duplex>) > status: active >gif6: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280 > tunnel inet A.B.C.D --> E.F.G.H > inet 10.128.1.6 --> 10.128.6.1 netmask 0xffffffff > inet6 fe80::2c0:9fff:fe46:ecc6%gif6 prefixlen 64 scopeid 0x4 > >Routing tables (shortened) >Destination Gateway Flags Refs Use Mtu >Netif Expire >default A.B.C.x UGS 2 516686 1500 em1 >10.128.1.6 127.0.0.1 UH 0 14 16384 lo0 >10.128.6.1 10.128.1.6 UH 0 6017 1280 gif6 >127.0.0.1 127.0.0.1 UH 0 31633 16384 lo0 >A.B.C.x/29 link#2 UC 0 0 1500 em1 >A.B.C.D 00:c0:9f:46:ec:c7 UHLW 0 112 1500 lo0 > >On host B the interfaces and routing tables are looking like: >xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > options=8<VLAN_MTU> > inet 0.0.0.0 netmask 0xff000000 broadcast 0.255.255.255 > inet6 fe80::260:8ff:fe6c:e73c%xl0 prefixlen 64 scopeid 0x1 > ether 00:60:08:6c:e7:3c > media: Ethernet 10baseT/UTP (10baseT/UTP <half-duplex>) > status: active >gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280 > tunnel inet E.F.G.H --> A.B.C.D > inet6 fe80::260:8ff:fe6c:e73c%gif1 prefixlen 64 scopeid 0x4 > inet 10.128.6.1 --> 10.128.1.6 netmask 0xffffffff >ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> mtu 1456 > inet E.F.G.H --> 217.5.98.186 netmask 0xffffffff > >Routing tables (shortened) >Destination Gateway Flags Refs Use Mtu >Netif Expire >0 link#1 UC 0 0 1500 > xl0 => >default 217.5.98.186 UGS 1 38474 1456 ng0 >10.128.1.6 10.128.6.1 UH 4 2196 1280 gif1 >127.0.0.1 127.0.0.1 UH 0 80424 16384 lo0 >217.5.98.186 E.F.G.H UH 1 0 1456 ng0 >E.F.G.H lo0 UHS 0 0 16384 lo0 > >While trying to fetch a file by scp on host A (receiver) from host B >(sender), I captured the following tcpdump on host B: > >tcpdump -netttvvi gif1: > > >>000023 AF 2 1280: IP (tos 0x8, ttl 64, id 13202, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 43864:45092(1228) ack 1330 win 33156 <nop,nop,timestamp 481770567 565002838> >>000207 AF 2 1280: IP (tos 0x8, ttl 64, id 52187, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 45092:46320(1228) ack 1330 win 33156 <nop,nop,timestamp 481770567 565002838> >>000220 AF 2 1280: IP (tos 0x8, ttl 64, id 33774, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 46320:47548(1228) ack 1330 win 33156 <nop,nop,timestamp 481770568 565002838> >>003524 AF 2 52: IP (tos 0x8, ttl 64, id 42063, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 38952 win 33156 <nop,nop,timestamp 565002844 481770524> >>000024 AF 2 1280: IP (tos 0x8, ttl 64, id 48541, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 47548:48776(1228) ack 1330 win 33156 <nop,nop,timestamp 481770571 565002844> >>011203 AF 2 52: IP (tos 0x8, ttl 64, id 60517, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 41408 win 32542 <nop,nop,timestamp 565002855 481770530> >>000058 AF 2 1280: IP (tos 0x8, ttl 64, id 15798, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 48776:50004(1228) ack 1330 win 33156 <nop,nop,timestamp 481770582 565002855> >>000246 AF 2 1280: IP (tos 0x8, ttl 64, id 31721, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 50004:51232(1228) ack 1330 win 33156 <nop,nop,timestamp 481770583 565002855> >>005147 AF 2 52: IP (tos 0x8, ttl 64, id 22347, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 42636 win 33156 <nop,nop,timestamp 565002861 481770542> >>000024 AF 2 1280: IP (tos 0x8, ttl 64, id 61057, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 51232:52460(1228) ack 1330 win 33156 <nop,nop,timestamp 481770588 565002861> >>020769 AF 2 52: IP (tos 0x8, ttl 64, id 27692, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 45092 win 32542 <nop,nop,timestamp 565002881 481770547> >>000027 AF 2 1280: IP (tos 0x8, ttl 64, id 64167, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 52460:53688(1228) ack 1330 win 33156 <nop,nop,timestamp 481770609 565002881> >>000209 AF 2 1280: IP (tos 0x8, ttl 64, id 45457, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 53688:54916(1228) ack 1330 win 33156 <nop,nop,timestamp 481770609 565002881> >>005260 AF 2 52: IP (tos 0x8, ttl 64, id 53832, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 46320 win 33156 <nop,nop,timestamp 565002887 481770567> >>000024 AF 2 1280: IP (tos 0x8, ttl 64, id 3515, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 54916:56144(1228) ack 1330 win 33156 <nop,nop,timestamp 481770614 565002887> >>011020 AF 2 52: IP (tos 0x8, ttl 64, id 11608, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 48776 win 32542 <nop,nop,timestamp 565002898 481770568> >>000026 AF 2 1280: IP (tos 0x8, ttl 64, id 5848, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 56144:57372(1228) ack 1330 win 33156 <nop,nop,timestamp 481770625 565002898> >>000211 AF 2 1280: IP (tos 0x8, ttl 64, id 39892, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 57372:58600(1228) ack 1330 win 33156 <nop,nop,timestamp 481770625 565002898> >>005641 AF 2 52: IP (tos 0x8, ttl 64, id 7943, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 50004 win 33156 <nop,nop,timestamp 565002904 481770582> >>000024 AF 2 1280: IP (tos 0x8, ttl 64, id 8678, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 58600:59828(1228) ack 1330 win 33156 <nop,nop,timestamp 481770631 565002904> >>011072 AF 2 52: IP (tos 0x8, ttl 64, id 38257, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 52460 win 32542 <nop,nop,timestamp 565002915 481770583> >>000025 AF 2 1280: IP (tos 0x8, ttl 64, id 12255, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 59828:61056(1228) ack 1330 win 33156 <nop,nop,timestamp 481770642 565002915> >>000209 AF 2 1280: IP (tos 0x8, ttl 64, id 46257, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 61056:62284(1228) ack 1330 win 33156 <nop,nop,timestamp 481770642 565002915> >>000222 AF 2 1280: IP (tos 0x8, ttl 64, id 4093, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 62284:63512(1228) ack 1330 win 33156 <nop,nop,timestamp 481770643 565002915> >>007065 AF 2 52: IP (tos 0x8, ttl 64, id 18720, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 53688 win 33156 <nop,nop,timestamp 565002922 481770609> >>000025 AF 2 1280: IP (tos 0x8, ttl 64, id 38378, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 63512:64740(1228) ack 1330 win 33156 <nop,nop,timestamp 481770650 565002922> >>011034 AF 2 52: IP (tos 0x8, ttl 64, id 18718, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 56144 win 32542 <nop,nop,timestamp 565002934 481770609> >>000024 AF 2 1280: IP (tos 0x8, ttl 64, id 8148, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 64740:65968(1228) ack 1330 win 33156 <nop,nop,timestamp 481770661 565002934> >>005991 AF 2 52: IP (tos 0x8, ttl 64, id 62285, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 57372 win 33156 <nop,nop,timestamp 565002939 481770625> >>010726 AF 2 52: IP (tos 0x8, ttl 64, id 1549, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 59828 win 32542 <nop,nop,timestamp 565002950 481770625> >>005670 AF 2 52: IP (tos 0x8, ttl 64, id 61504, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 61056 win 33156 <nop,nop,timestamp 565002956 481770642> >>011260 AF 2 52: IP (tos 0x8, ttl 64, id 32633, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 63512 win 32542 <nop,nop,timestamp 565002967 481770642> >>005510 AF 2 52: IP (tos 0x8, ttl 64, id 54614, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 64740 win 33156 <nop,nop,timestamp 565002973 481770650> >>104909 AF 2 52: IP (tos 0x8, ttl 64, id 50471, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 65968 win 33156 <nop,nop,timestamp 565003078 481770661> >> >> > >tcpdump -netttvvi ng0 host A.B.C.D: > > >>000227 AF 2 1352: IP (tos 0x8, ttl 64, id 25895, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10b) >>011042 AF 2 128: IP (tos 0x8, ttl 61, id 5786, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf0) >>000226 AF 2 1352: IP (tos 0x8, ttl 64, id 36701, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10c) >>000216 AF 2 1352: IP (tos 0x8, ttl 64, id 8789, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10d) >>004853 AF 2 128: IP (tos 0x8, ttl 61, id 17128, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf1) >>000227 AF 2 1352: IP (tos 0x8, ttl 64, id 34888, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10e) >>018747 AF 2 128: IP (tos 0x8, ttl 61, id 14828, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf2) >>000248 AF 2 1352: IP (tos 0x8, ttl 64, id 34356, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10f) >>000223 AF 2 1352: IP (tos 0x8, ttl 64, id 34151, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x110) >>005030 AF 2 128: IP (tos 0x8, ttl 61, id 45476, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf3) >>000228 AF 2 1352: IP (tos 0x8, ttl 64, id 39765, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x111) >>011247 AF 2 128: IP (tos 0x8, ttl 61, id 63692, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf4) >>000226 AF 2 1352: IP (tos 0x8, ttl 64, id 29240, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x112) >>000222 AF 2 1352: IP (tos 0x8, ttl 64, id 43306, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x113) >>005663 AF 2 128: IP (tos 0x8, ttl 61, id 32980, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf5) >>000228 AF 2 1352: IP (tos 0x8, ttl 64, id 56920, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x114) >>010190 AF 2 128: IP (tos 0x8, ttl 61, id 3206, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf6) >>000227 AF 2 1352: IP (tos 0x8, ttl 64, id 4655, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x115) >>000215 AF 2 1352: IP (tos 0x8, ttl 64, id 62740, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x116) >>000203 AF 2 1352: IP (tos 0x8, ttl 64, id 35642, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x117) >>006875 AF 2 128: IP (tos 0x8, ttl 61, id 37801, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf7) >>000234 AF 2 1352: IP (tos 0x8, ttl 64, id 41803, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x118) >>010651 AF 2 128: IP (tos 0x8, ttl 61, id 54256, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf8) >>000235 AF 2 1352: IP (tos 0x8, ttl 64, id 30732, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x119) >>007913 AF 2 128: IP (tos 0x8, ttl 61, id 7647, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf9) >>011166 AF 2 128: IP (tos 0x8, ttl 61, id 58037, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfa) >>005483 AF 2 128: IP (tos 0x8, ttl 61, id 65275, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfb) >>011250 AF 2 128: IP (tos 0x8, ttl 61, id 47289, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfc) >>005505 AF 2 128: IP (tos 0x8, ttl 61, id 203, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfd) >>104747 AF 2 128: IP (tos 0x8, ttl 61, id 45263, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfe) >>8. 338674 AF 2 128: IP (tos 0x8, ttl 61, id 36351, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xff) >>319992 AF 2 128: IP (tos 0x8, ttl 61, id 18085, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x100) >>441837 AF 2 128: IP (tos 0x8, ttl 61, id 58323, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x101) >>684077 AF 2 128: IP (tos 0x8, ttl 61, id 35487, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x102) >>1. 167602 AF 2 128: IP (tos 0x8, ttl 61, id 34442, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x103) >>2. 136032 AF 2 128: IP (tos 0x8, ttl 61, id 8345, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x104) >>2. 984665 AF 2 128: IP (tos 0x8, ttl 61, id 35456, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x105) >> >> > > >>From what I'm seeing host B just stops sending without any reason. At >least I don't see any fragmented packets. The only thing I've seen is >some packets doesn't get ack'ed by the receiver. > >These packets never get ack'ed: >46320:47548(1228) >50004:51232(1228) >53688:54916(1228) >57372:58600(1228) >61056:62284(1228) > >On host A I dumped the following: > >tcpdump -netttvvi gif6 > > >>1129985378.941282 AF 2 52: IP (tos 0x8, ttl 64, id 41637, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 45092 win 32542 <nop,nop,timestamp 574090240 490857876> >>1129985378.952628 AF 2 1280: IP (tos 0x8, ttl 64, id 14004, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 45092:46320(1228) ack 1330 win 33156 <nop,nop,timestamp 490857901 574090210> >>1129985378.952657 AF 2 52: IP (tos 0x8, ttl 64, id 23243, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 46320 win 33156 <nop,nop,timestamp 574090251 490857901> >>1129985378.958250 AF 2 1280: IP (tos 0x8, ttl 64, id 4306, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 46320:47548(1228) ack 1330 win 33156 <nop,nop,timestamp 490857901 574090210> >>1129985378.971118 AF 2 1280: IP (tos 0x8, ttl 64, id 33534, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 47548:48776(1228) ack 1330 win 33156 <nop,nop,timestamp 490857920 574090229> >>1129985378.971137 AF 2 52: IP (tos 0x8, ttl 64, id 60095, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 48776 win 32542 <nop,nop,timestamp 574090270 490857901> >>1129985378.982488 AF 2 1280: IP (tos 0x8, ttl 64, id 11459, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 48776:50004(1228) ack 1330 win 33156 <nop,nop,timestamp 490857931 574090240> >>1129985378.982516 AF 2 52: IP (tos 0x8, ttl 64, id 33184, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 50004 win 33156 <nop,nop,timestamp 574090281 490857931> >>1129985378.987989 AF 2 1280: IP (tos 0x8, ttl 64, id 54180, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 50004:51232(1228) ack 1330 win 33156 <nop,nop,timestamp 490857931 574090240> >>1129985378.994231 AF 2 1280: IP (tos 0x8, ttl 64, id 24535, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 51232:52460(1228) ack 1330 win 33156 <nop,nop,timestamp 490857942 574090251> >>1129985378.994250 AF 2 52: IP (tos 0x8, ttl 64, id 30647, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 52460 win 32542 <nop,nop,timestamp 574090293 490857931> >>1129985379.012101 AF 2 1280: IP (tos 0x8, ttl 64, id 61397, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 52460:53688(1228) ack 1330 win 33156 <nop,nop,timestamp 490857960 574090270> >>1129985379.012132 AF 2 52: IP (tos 0x8, ttl 64, id 60550, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 53688 win 33156 <nop,nop,timestamp 574090311 490857960> >>1129985379.017754 AF 2 1280: IP (tos 0x8, ttl 64, id 28408, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 53688:54916(1228) ack 1330 win 33156 <nop,nop,timestamp 490857961 574090270> >>1129985379.023720 AF 2 1280: IP (tos 0x8, ttl 64, id 27558, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 54916:56144(1228) ack 1330 win 33156 <nop,nop,timestamp 490857972 574090281> >>1129985379.023741 AF 2 52: IP (tos 0x8, ttl 64, id 21502, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 56144 win 32542 <nop,nop,timestamp 574090322 490857961> >>1129985379.035333 AF 2 1280: IP (tos 0x8, ttl 64, id 18885, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 56144:57372(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984 574090293> >>1129985379.035362 AF 2 52: IP (tos 0x8, ttl 64, id 59875, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 57372 win 33156 <nop,nop,timestamp 574090334 490857984> >>1129985379.040830 AF 2 1280: IP (tos 0x8, ttl 64, id 37252, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 57372:58600(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984 574090293> >>1129985379.046576 AF 2 1280: IP (tos 0x8, ttl 64, id 18349, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 58600:59828(1228) ack 1330 win 33156 <nop,nop,timestamp 490857984 574090293> >>1129985379.046595 AF 2 52: IP (tos 0x8, ttl 64, id 43697, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 59828 win 32542 <nop,nop,timestamp 574090345 490857984> >>1129985379.064961 AF 2 1280: IP (tos 0x8, ttl 64, id 38300, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 59828:61056(1228) ack 1330 win 33156 <nop,nop,timestamp 490858013 574090322> >>1129985379.064993 AF 2 52: IP (tos 0x8, ttl 64, id 47539, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 61056 win 33156 <nop,nop,timestamp 574090364 490858013> >>1129985379.070688 AF 2 1280: IP (tos 0x8, ttl 64, id 30345, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 61056:62284(1228) ack 1330 win 33156 <nop,nop,timestamp 490858013 574090322> >>1129985379.076184 AF 2 1280: IP (tos 0x8, ttl 64, id 37536, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 62284:63512(1228) ack 1330 win 33156 <nop,nop,timestamp 490858014 574090322> >>1129985379.076202 AF 2 52: IP (tos 0x8, ttl 64, id 34201, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 63512 win 32542 <nop,nop,timestamp 574090375 490858013> >>1129985379.081680 AF 2 1280: IP (tos 0x8, ttl 64, id 20637, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 63512:64740(1228) ack 1330 win 33156 <nop,nop,timestamp 490858025 574090334> >>1129985379.081709 AF 2 52: IP (tos 0x8, ttl 64, id 59866, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 64740 win 33156 <nop,nop,timestamp 574090380 490858025> >>1129985379.087678 AF 2 1280: IP (tos 0x8, ttl 64, id 35213, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 64740:65968(1228) ack 1330 win 33156 <nop,nop,timestamp 490858036 574090345> >>1129985379.186906 AF 2 52: IP (tos 0x8, ttl 64, id 2465, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 65968 win 33156 <nop,nop,timestamp 574090486 490858036> >> >> > >tcpdump -netttvvi em1 host E.F.G.H > > >>1129985379.064825 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 45003, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x3e) >>1129985379.065024 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 1195, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x2f) >>1129985379.070572 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 36820, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x3f) >>1129985379.076069 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 44971, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x40) >>1129985379.076233 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 56964, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x30) >>1129985379.081565 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 24742, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x41) >>1129985379.081741 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 9390, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x31) >>1129985379.087562 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 48065, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x42) >>1129985379.186945 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 36315, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x32) >> >> > >If I'm not misleaded, this also doesn't show any errors except the >missing ack's. host B just stops sending. If there's an ack missing, >doesn't have the sending host to just repeat the un-ack'ed packet? > >The IPSec tunnel does not die. Even shortly after the (scp) transfer >stalls the tunnel itself is still usable (for small amounts of data). To >make it more worse, when disabling pf at the senders side, the transfer >works. I've tripple checked pflog for denied packets on both sides but >pf didn't filter any packets out. > >When disabling the IPSec rules using `setkey -F; setkey -FP' on the >tunnel for a moment, the scp transfer does not stall. So it's not a gif >issue. > >It doesn't seem to be an MTU issue (pf has also the rule 'scrub in/out >all no-df'), but what kind of issue is that?? Has anybody ever >experienced similar things? Or am I misinterpreting the tcpdump output? > > >Any help and hint is appreciated! Without an error message I'm lost. > >Volker > >_______________________________________________ >freebsd-net@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-net >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?435A900C.3060602>