From owner-freebsd-net@FreeBSD.ORG Sat Oct 22 14:44:37 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D97A916A41F for ; Sat, 22 Oct 2005 14:44:37 +0000 (GMT) (envelope-from volker@vwsoft.com) Received: from tce71.tce85.de (tce71.tce85.de [195.145.102.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7DB3543D60 for ; Sat, 22 Oct 2005 14:44:31 +0000 (GMT) (envelope-from volker@vwsoft.com) Received-SPF: pass (tce71.tce85.de: domain of vwsoft.com designates 84.245.164.24 as permitted sender) client-ip=84.245.164.24; envelope-from=volker@vwsoft.com; helo=mail.vtec.ipme.de; Received: from mail.vtec.ipme.de (unknown [84.245.164.24]) by tce71.tce85.de (Postfix) with ESMTP id 2AD6117122 for ; Sat, 22 Oct 2005 16:44:35 +0200 (CEST) Received: from [192.168.16.3] (cesar.sz.vwsoft.com [192.168.16.3]) by mail.vtec.ipme.de (Postfix) with ESMTP id 0A5515C0E for ; Sat, 22 Oct 2005 16:41:13 +0200 (CEST) Message-ID: <435A5D9B.7080309@vwsoft.com> Date: Sat, 22 Oct 2005 16:41:15 +0100 From: Volker User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.10) Gecko/20050716 Thunderbird/1.0.6 Mnenhy/0.6.0.101 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-net@freebsd.org X-Enigmail-Version: 0.92.1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-VWSoft-MailScanner: Found to be clean X-TarmacCE-MailScanner: Found to be clean X-MailScanner-From: volker@vwsoft.com Subject: IPSec tcp session stalling X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Oct 2005 14:44:38 -0000 Still having the same problem with an IPSec tunnel between FreeBSD 5.4R hosts. Problem description: scp session tries to transfer a large file through an IPSec tunnel. The file is being transmitted but scp says 'stalled' after 56K (49152 bytes file size). The IPSec tunnel itself is still up even after the scp abort. Other tcp sessions break, too when sending too much traffic through the tunnel. I've taken a closer look to it and tried to get something useful out of the tcpdump but I'm unable to see any errors or I'm misinterpreting something. The connection looks like: extIP: A.B.C.D extIP: E.F.G.H host A ------------------ (internet) ------------------ host B tunnelIP: 10.128.1.6 tunnelIP: 10.128.6.1 host A just has an external interface (em1) connected to a leased line with a fixed IP address (IP-addr A.B.C.D). host B has an S-DSL connection at xl0, PPPoE at ng0 (IP-addr. E.F.G.H). Both hosts are using gif for the IPSec tunnel. The routing tables (netstat -rnWf inet) are looking good and IMHO the MTU is fine. host A: em1: flags=8843 mtu 1500 options=b inet A.B.C.D netmask 0xfffffff8 broadcast A.B.C.z ether 00:c0:9f:46:ec:c7 media: Ethernet autoselect (100baseTX ) status: active gif6: flags=8051 mtu 1280 tunnel inet A.B.C.D --> E.F.G.H inet 10.128.1.6 --> 10.128.6.1 netmask 0xffffffff inet6 fe80::2c0:9fff:fe46:ecc6%gif6 prefixlen 64 scopeid 0x4 Routing tables (shortened) Destination Gateway Flags Refs Use Mtu Netif Expire default A.B.C.x UGS 2 516686 1500 em1 10.128.1.6 127.0.0.1 UH 0 14 16384 lo0 10.128.6.1 10.128.1.6 UH 0 6017 1280 gif6 127.0.0.1 127.0.0.1 UH 0 31633 16384 lo0 A.B.C.x/29 link#2 UC 0 0 1500 em1 A.B.C.D 00:c0:9f:46:ec:c7 UHLW 0 112 1500 lo0 On host B the interfaces and routing tables are looking like: xl0: flags=8843 mtu 1500 options=8 inet 0.0.0.0 netmask 0xff000000 broadcast 0.255.255.255 inet6 fe80::260:8ff:fe6c:e73c%xl0 prefixlen 64 scopeid 0x1 ether 00:60:08:6c:e7:3c media: Ethernet 10baseT/UTP (10baseT/UTP ) status: active gif1: flags=8051 mtu 1280 tunnel inet E.F.G.H --> A.B.C.D inet6 fe80::260:8ff:fe6c:e73c%gif1 prefixlen 64 scopeid 0x4 inet 10.128.6.1 --> 10.128.1.6 netmask 0xffffffff ng0: flags=88d1 mtu 1456 inet E.F.G.H --> 217.5.98.186 netmask 0xffffffff Routing tables (shortened) Destination Gateway Flags Refs Use Mtu Netif Expire 0 link#1 UC 0 0 1500 xl0 => default 217.5.98.186 UGS 1 38474 1456 ng0 10.128.1.6 10.128.6.1 UH 4 2196 1280 gif1 127.0.0.1 127.0.0.1 UH 0 80424 16384 lo0 217.5.98.186 E.F.G.H UH 1 0 1456 ng0 E.F.G.H lo0 UHS 0 0 16384 lo0 While trying to fetch a file by scp on host A (receiver) from host B (sender), I captured the following tcpdump on host B: tcpdump -netttvvi gif1: > 000023 AF 2 1280: IP (tos 0x8, ttl 64, id 13202, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 43864:45092(1228) ack 1330 win 33156 > 000207 AF 2 1280: IP (tos 0x8, ttl 64, id 52187, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 45092:46320(1228) ack 1330 win 33156 > 000220 AF 2 1280: IP (tos 0x8, ttl 64, id 33774, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 46320:47548(1228) ack 1330 win 33156 > 003524 AF 2 52: IP (tos 0x8, ttl 64, id 42063, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 38952 win 33156 > 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 48541, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 47548:48776(1228) ack 1330 win 33156 > 011203 AF 2 52: IP (tos 0x8, ttl 64, id 60517, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 41408 win 32542 > 000058 AF 2 1280: IP (tos 0x8, ttl 64, id 15798, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 48776:50004(1228) ack 1330 win 33156 > 000246 AF 2 1280: IP (tos 0x8, ttl 64, id 31721, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 50004:51232(1228) ack 1330 win 33156 > 005147 AF 2 52: IP (tos 0x8, ttl 64, id 22347, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 42636 win 33156 > 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 61057, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 51232:52460(1228) ack 1330 win 33156 > 020769 AF 2 52: IP (tos 0x8, ttl 64, id 27692, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 45092 win 32542 > 000027 AF 2 1280: IP (tos 0x8, ttl 64, id 64167, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 52460:53688(1228) ack 1330 win 33156 > 000209 AF 2 1280: IP (tos 0x8, ttl 64, id 45457, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 53688:54916(1228) ack 1330 win 33156 > 005260 AF 2 52: IP (tos 0x8, ttl 64, id 53832, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 46320 win 33156 > 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 3515, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 54916:56144(1228) ack 1330 win 33156 > 011020 AF 2 52: IP (tos 0x8, ttl 64, id 11608, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 48776 win 32542 > 000026 AF 2 1280: IP (tos 0x8, ttl 64, id 5848, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 56144:57372(1228) ack 1330 win 33156 > 000211 AF 2 1280: IP (tos 0x8, ttl 64, id 39892, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 57372:58600(1228) ack 1330 win 33156 > 005641 AF 2 52: IP (tos 0x8, ttl 64, id 7943, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 50004 win 33156 > 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 8678, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 58600:59828(1228) ack 1330 win 33156 > 011072 AF 2 52: IP (tos 0x8, ttl 64, id 38257, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 52460 win 32542 > 000025 AF 2 1280: IP (tos 0x8, ttl 64, id 12255, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 59828:61056(1228) ack 1330 win 33156 > 000209 AF 2 1280: IP (tos 0x8, ttl 64, id 46257, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 61056:62284(1228) ack 1330 win 33156 > 000222 AF 2 1280: IP (tos 0x8, ttl 64, id 4093, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 62284:63512(1228) ack 1330 win 33156 > 007065 AF 2 52: IP (tos 0x8, ttl 64, id 18720, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 53688 win 33156 > 000025 AF 2 1280: IP (tos 0x8, ttl 64, id 38378, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 63512:64740(1228) ack 1330 win 33156 > 011034 AF 2 52: IP (tos 0x8, ttl 64, id 18718, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 56144 win 32542 > 000024 AF 2 1280: IP (tos 0x8, ttl 64, id 8148, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.53160: . 64740:65968(1228) ack 1330 win 33156 > 005991 AF 2 52: IP (tos 0x8, ttl 64, id 62285, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 57372 win 33156 > 010726 AF 2 52: IP (tos 0x8, ttl 64, id 1549, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 59828 win 32542 > 005670 AF 2 52: IP (tos 0x8, ttl 64, id 61504, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 61056 win 33156 > 011260 AF 2 52: IP (tos 0x8, ttl 64, id 32633, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 63512 win 32542 > 005510 AF 2 52: IP (tos 0x8, ttl 64, id 54614, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 64740 win 33156 > 104909 AF 2 52: IP (tos 0x8, ttl 64, id 50471, offset 0, flags [none], length: 52) 10.128.1.6.53160 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 65968 win 33156 tcpdump -netttvvi ng0 host A.B.C.D: > 000227 AF 2 1352: IP (tos 0x8, ttl 64, id 25895, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10b) > 011042 AF 2 128: IP (tos 0x8, ttl 61, id 5786, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf0) > 000226 AF 2 1352: IP (tos 0x8, ttl 64, id 36701, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10c) > 000216 AF 2 1352: IP (tos 0x8, ttl 64, id 8789, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10d) > 004853 AF 2 128: IP (tos 0x8, ttl 61, id 17128, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf1) > 000227 AF 2 1352: IP (tos 0x8, ttl 64, id 34888, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10e) > 018747 AF 2 128: IP (tos 0x8, ttl 61, id 14828, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf2) > 000248 AF 2 1352: IP (tos 0x8, ttl 64, id 34356, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x10f) > 000223 AF 2 1352: IP (tos 0x8, ttl 64, id 34151, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x110) > 005030 AF 2 128: IP (tos 0x8, ttl 61, id 45476, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf3) > 000228 AF 2 1352: IP (tos 0x8, ttl 64, id 39765, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x111) > 011247 AF 2 128: IP (tos 0x8, ttl 61, id 63692, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf4) > 000226 AF 2 1352: IP (tos 0x8, ttl 64, id 29240, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x112) > 000222 AF 2 1352: IP (tos 0x8, ttl 64, id 43306, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x113) > 005663 AF 2 128: IP (tos 0x8, ttl 61, id 32980, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf5) > 000228 AF 2 1352: IP (tos 0x8, ttl 64, id 56920, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x114) > 010190 AF 2 128: IP (tos 0x8, ttl 61, id 3206, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf6) > 000227 AF 2 1352: IP (tos 0x8, ttl 64, id 4655, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x115) > 000215 AF 2 1352: IP (tos 0x8, ttl 64, id 62740, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x116) > 000203 AF 2 1352: IP (tos 0x8, ttl 64, id 35642, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x117) > 006875 AF 2 128: IP (tos 0x8, ttl 61, id 37801, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf7) > 000234 AF 2 1352: IP (tos 0x8, ttl 64, id 41803, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x118) > 010651 AF 2 128: IP (tos 0x8, ttl 61, id 54256, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf8) > 000235 AF 2 1352: IP (tos 0x8, ttl 64, id 30732, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x078b2968,seq=0x119) > 007913 AF 2 128: IP (tos 0x8, ttl 61, id 7647, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xf9) > 011166 AF 2 128: IP (tos 0x8, ttl 61, id 58037, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfa) > 005483 AF 2 128: IP (tos 0x8, ttl 61, id 65275, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfb) > 011250 AF 2 128: IP (tos 0x8, ttl 61, id 47289, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfc) > 005505 AF 2 128: IP (tos 0x8, ttl 61, id 203, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfd) > 104747 AF 2 128: IP (tos 0x8, ttl 61, id 45263, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xfe) > 8. 338674 AF 2 128: IP (tos 0x8, ttl 61, id 36351, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0xff) > 319992 AF 2 128: IP (tos 0x8, ttl 61, id 18085, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x100) > 441837 AF 2 128: IP (tos 0x8, ttl 61, id 58323, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x101) > 684077 AF 2 128: IP (tos 0x8, ttl 61, id 35487, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x102) > 1. 167602 AF 2 128: IP (tos 0x8, ttl 61, id 34442, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x103) > 2. 136032 AF 2 128: IP (tos 0x8, ttl 61, id 8345, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x104) > 2. 984665 AF 2 128: IP (tos 0x8, ttl 61, id 35456, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x0858046f,seq=0x105) >From what I'm seeing host B just stops sending without any reason. At least I don't see any fragmented packets. The only thing I've seen is some packets doesn't get ack'ed by the receiver. These packets never get ack'ed: 46320:47548(1228) 50004:51232(1228) 53688:54916(1228) 57372:58600(1228) 61056:62284(1228) On host A I dumped the following: tcpdump -netttvvi gif6 > 1129985378.941282 AF 2 52: IP (tos 0x8, ttl 64, id 41637, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 45092 win 32542 > 1129985378.952628 AF 2 1280: IP (tos 0x8, ttl 64, id 14004, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 45092:46320(1228) ack 1330 win 33156 > 1129985378.952657 AF 2 52: IP (tos 0x8, ttl 64, id 23243, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 46320 win 33156 > 1129985378.958250 AF 2 1280: IP (tos 0x8, ttl 64, id 4306, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 46320:47548(1228) ack 1330 win 33156 > 1129985378.971118 AF 2 1280: IP (tos 0x8, ttl 64, id 33534, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 47548:48776(1228) ack 1330 win 33156 > 1129985378.971137 AF 2 52: IP (tos 0x8, ttl 64, id 60095, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 48776 win 32542 > 1129985378.982488 AF 2 1280: IP (tos 0x8, ttl 64, id 11459, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 48776:50004(1228) ack 1330 win 33156 > 1129985378.982516 AF 2 52: IP (tos 0x8, ttl 64, id 33184, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 50004 win 33156 > 1129985378.987989 AF 2 1280: IP (tos 0x8, ttl 64, id 54180, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 50004:51232(1228) ack 1330 win 33156 > 1129985378.994231 AF 2 1280: IP (tos 0x8, ttl 64, id 24535, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 51232:52460(1228) ack 1330 win 33156 > 1129985378.994250 AF 2 52: IP (tos 0x8, ttl 64, id 30647, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 52460 win 32542 > 1129985379.012101 AF 2 1280: IP (tos 0x8, ttl 64, id 61397, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 52460:53688(1228) ack 1330 win 33156 > 1129985379.012132 AF 2 52: IP (tos 0x8, ttl 64, id 60550, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 53688 win 33156 > 1129985379.017754 AF 2 1280: IP (tos 0x8, ttl 64, id 28408, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 53688:54916(1228) ack 1330 win 33156 > 1129985379.023720 AF 2 1280: IP (tos 0x8, ttl 64, id 27558, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 54916:56144(1228) ack 1330 win 33156 > 1129985379.023741 AF 2 52: IP (tos 0x8, ttl 64, id 21502, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 56144 win 32542 > 1129985379.035333 AF 2 1280: IP (tos 0x8, ttl 64, id 18885, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 56144:57372(1228) ack 1330 win 33156 > 1129985379.035362 AF 2 52: IP (tos 0x8, ttl 64, id 59875, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 57372 win 33156 > 1129985379.040830 AF 2 1280: IP (tos 0x8, ttl 64, id 37252, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 57372:58600(1228) ack 1330 win 33156 > 1129985379.046576 AF 2 1280: IP (tos 0x8, ttl 64, id 18349, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 58600:59828(1228) ack 1330 win 33156 > 1129985379.046595 AF 2 52: IP (tos 0x8, ttl 64, id 43697, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 59828 win 32542 > 1129985379.064961 AF 2 1280: IP (tos 0x8, ttl 64, id 38300, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 59828:61056(1228) ack 1330 win 33156 > 1129985379.064993 AF 2 52: IP (tos 0x8, ttl 64, id 47539, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 61056 win 33156 > 1129985379.070688 AF 2 1280: IP (tos 0x8, ttl 64, id 30345, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 61056:62284(1228) ack 1330 win 33156 > 1129985379.076184 AF 2 1280: IP (tos 0x8, ttl 64, id 37536, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 62284:63512(1228) ack 1330 win 33156 > 1129985379.076202 AF 2 52: IP (tos 0x8, ttl 64, id 34201, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 63512 win 32542 > 1129985379.081680 AF 2 1280: IP (tos 0x8, ttl 64, id 20637, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 63512:64740(1228) ack 1330 win 33156 > 1129985379.081709 AF 2 52: IP (tos 0x8, ttl 64, id 59866, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 64740 win 33156 > 1129985379.087678 AF 2 1280: IP (tos 0x8, ttl 64, id 35213, offset 0, flags [none], length: 1280) 10.128.6.1.22 > 10.128.1.6.59762: . 64740:65968(1228) ack 1330 win 33156 > 1129985379.186906 AF 2 52: IP (tos 0x8, ttl 64, id 2465, offset 0, flags [none], length: 52) 10.128.1.6.59762 > 10.128.6.1.22: . [tcp sum ok] 1330:1330(0) ack 65968 win 33156 tcpdump -netttvvi em1 host E.F.G.H > 1129985379.064825 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 45003, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x3e) > 1129985379.065024 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 1195, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x2f) > 1129985379.070572 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 36820, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x3f) > 1129985379.076069 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 44971, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x40) > 1129985379.076233 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 56964, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x30) > 1129985379.081565 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 24742, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x41) > 1129985379.081741 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 9390, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x31) > 1129985379.087562 00:13:c4:fa:6c:20 > 00:c0:9f:46:ec:c7, ethertype IPv4 (0x0800), length 1366: IP (tos 0x8, ttl 61, id 48065, offset 0, flags [none], length: 1352) E.F.G.H > A.B.C.D: ESP(spi=0x0e0dffaa,seq=0x42) > 1129985379.186945 00:c0:9f:46:ec:c7 > 00:13:c4:fa:6c:20, ethertype IPv4 (0x0800), length 142: IP (tos 0x8, ttl 64, id 36315, offset 0, flags [none], length: 128) A.B.C.D > E.F.G.H: ESP(spi=0x029a41b4,seq=0x32) If I'm not misleaded, this also doesn't show any errors except the missing ack's. host B just stops sending. If there's an ack missing, doesn't have the sending host to just repeat the un-ack'ed packet? The IPSec tunnel does not die. Even shortly after the (scp) transfer stalls the tunnel itself is still usable (for small amounts of data). To make it more worse, when disabling pf at the senders side, the transfer works. I've tripple checked pflog for denied packets on both sides but pf didn't filter any packets out. When disabling the IPSec rules using `setkey -F; setkey -FP' on the tunnel for a moment, the scp transfer does not stall. So it's not a gif issue. It doesn't seem to be an MTU issue (pf has also the rule 'scrub in/out all no-df'), but what kind of issue is that?? Has anybody ever experienced similar things? Or am I misinterpreting the tcpdump output? Any help and hint is appreciated! Without an error message I'm lost. Volker