From owner-freebsd-net@FreeBSD.ORG Mon Sep 20 23:37:29 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B018716A4CE for ; Mon, 20 Sep 2004 23:37:29 +0000 (GMT) Received: from mail.geek.sh (decoder.geek.sh [196.36.198.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id E212943D54 for ; Mon, 20 Sep 2004 23:37:27 +0000 (GMT) (envelope-from aragon@geek.sh) Received: by mail.geek.sh (Postfix, from userid 1000) id F115924D14; Mon, 20 Sep 2004 20:44:31 +0200 (SAST) Date: Mon, 20 Sep 2004 20:44:31 +0200 From: Aragon Gouveia To: freebsd-net@freebsd.org Message-ID: <20040920184431.GA89606@phat.za.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i X-Operating-System: FreeBSD 4.8-RELEASE-p1 i386 Subject: Wierd tunnel+MTU issue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Sep 2004 23:37:29 -0000 Hi, A while ago I setup a vtun tunnel between a FreeBSD 4.10-RELEASE machine and a 5.2.1-RELEASE-p9 machine. Initially everything appeared to work great, but I've just stumbled upon a seriously wierd problem that I can't figure out. I know this is not a support forum for the vtun package, but the problem I'm having is consistently reproducable with another VPN type package - OpenVPN. I'm beginning to think the problem is not related to vtun/OpenVPN and was hoping someone could shed some light on my problem. My setup is as follows. My notebook is running 5.2.1-REALEASE-p9 using userland ppp to establish a PPPoE session over an ADSL bridge. Above that, it runs vtun to establish a UDP tunnel to my VPN server sitting at an ISP. The VPN server is the 4.10-RELEASE machine also running vtun 1.6 configured in server mode. The link is configured for no compression, but encryption enabled. Now the link comes up 100% and passes most data fine. I've set my interface MTU down to 1200 on both sides. I'm also source routing on my notebook with ipfw fwd to make sure traffic flows out the links I want them to flow. This is working 100% too. The problem is this: I'm running Apache 2.0.50 on my notebook. If I request a page from my notebook from an outside machine via the VPN, request responses that exceed the interface MTU are simply not sent. For example, if I request a file sized at 1100 bytes, this is the tcpdump transcript running on my notebook, sniffing the VPN interface: tcpdump: listening on tun1 20:31:01.892060 196.15.a.z.3159 > 196.15.a.x.80: S 4115956342:4115956342(0) win 57344 (DF) [tos 0x10] 20:31:01.892228 196.15.a.x.80 > 196.15.a.z.3159: S 1941194202:1941194202(0) ack 4115956343 win 65535 (DF) 20:31:01.974087 196.15.a.z.3159 > 196.15.a.x.80: . ack 1 win 57600 (DF) [tos 0x10] 20:31:08.417478 196.15.a.z.3159 > 196.15.a.x.80: P 1:21(20) ack 1 win 57600 (DF) [tos 0x10] 20:31:08.517285 196.15.a.x.80 > 196.15.a.z.3159: . ack 21 win 33120 (DF) 20:31:10.340468 196.15.a.z.3159 > 196.15.a.x.80: P 21:23(2) ack 1 win 57600 (DF) [tos 0x10] 20:31:10.341371 196.15.a.x.80 > 196.15.a.z.3159: P 1:286(285) ack 23 win 33120 (DF) 20:31:10.341412 196.15.a.x.80 > 196.15.a.z.3159: P 286:1386(1100) ack 23 win 33120 (DF) 20:31:10.342143 196.15.a.x.80 > 196.15.a.z.3159: F 1386:1386(0) ack 23 win 33120 (DF) 20:31:10.568480 196.15.a.z.3159 > 196.15.a.x.80: . ack 286 win 57600 (DF) [tos 0x10] 20:31:10.618594 196.15.a.z.3159 > 196.15.a.x.80: . ack 1387 win 57600 (DF) [tos 0x10] 20:31:10.626417 196.15.a.z.3159 > 196.15.a.x.80: F 23:23(0) ack 1387 win 57600 (DF) [tos 0x10] 20:31:10.626532 196.15.a.x.80 > 196.15.a.z.3159: . ack 24 win 33119 (DF) The two important lines being: 20:31:10.341371 196.15.a.x.80 > 196.15.a.z.3159: P 1:286(285) ack 23 win 33120 (DF) 20:31:10.341412 196.15.a.x.80 > 196.15.a.z.3159: P 286:1386(1100) ack 23 win 33120 (DF) The first of these two is the HTTP response header, and the second the actual requested data (1100 bytes as shown). If I try request a file 1400 bytes large (MTU is 1200): tcpdump: listening on tun1 20:35:06.588068 196.15.a.z.3161 > 196.15.a.x.80: S 1461068997:1461068997(0) win 57344 (DF) [tos 0x10] 20:35:06.588242 196.15.a.x.80 > 196.15.a.z.3161: S 1654337904:1654337904(0) ack 1461068998 win 65535 (DF) 20:35:06.659998 196.15.a.z.3161 > 196.15.a.x.80: . ack 1 win 57600 (DF) [tos 0x10] 20:35:10.490089 196.15.a.z.3161 > 196.15.a.x.80: P 1:21(20) ack 1 win 57600 (DF) [tos 0x10] 20:35:10.589617 196.15.a.x.80 > 196.15.a.z.3161: . ack 21 win 33120 (DF) 20:35:11.506613 196.15.a.z.3161 > 196.15.a.x.80: P 21:23(2) ack 1 win 57600 (DF) [tos 0x10] 20:35:11.507306 196.15.a.x.80 > 196.15.a.z.3161: P 1:286(285) ack 23 win 33120 (DF) 20:35:11.716698 196.15.a.z.3161 > 196.15.a.x.80: . ack 286 win 57600 (DF) [tos 0x10] 20:35:16.619379 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 win 57600 (DF) [tos 0x10] 20:35:17.815936 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 win 57600 (DF) [tos 0x10] 20:35:20.017123 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 win 57600 (DF) [tos 0x10] 20:35:24.227404 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 win 57600 (DF) [tos 0x10] The two important lines being: 20:35:11.507306 196.15.a.x.80 > 196.15.a.z.3161: P 1:286(285) ack 23 win 33120 (DF) 20:35:11.716698 196.15.a.z.3161 > 196.15.a.x.80: . ack 286 win 57600 (DF) [tos 0x10] The first line of the important ones is the HTTP response header. The second one obviously just the TCP acknowledgement. The expected next few packets that should follow the header never get sent. I'm stumped. As I said, I've tried OpenVPN as well but the behaviour is precisely the same. Does anyone have any ideas? Thanks, Aragon