From owner-freebsd-net@FreeBSD.ORG Tue Aug 29 11:16:23 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9509516A4DA for ; Tue, 29 Aug 2006 11:16:23 +0000 (UTC) (envelope-from b.candler@pobox.com) Received: from rune.pobox.com (rune.pobox.com [208.210.124.79]) by mx1.FreeBSD.org (Postfix) with ESMTP id 267EB43D46 for ; Tue, 29 Aug 2006 11:16:23 +0000 (GMT) (envelope-from b.candler@pobox.com) Received: from rune (localhost [127.0.0.1]) by rune.pobox.com (Postfix) with ESMTP id 14B967D220; Tue, 29 Aug 2006 07:16:44 -0400 (EDT) Received: from mappit.local.linnet.org (212-74-113-67.static.dsl.as9105.com [212.74.113.67]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by rune.sasl.smtp.pobox.com (Postfix) with ESMTP id C02D77C5F3; Tue, 29 Aug 2006 07:16:42 -0400 (EDT) Received: from lists by mappit.local.linnet.org with local (Exim 4.61 (FreeBSD)) (envelope-from ) id 1GI1Zq-0001UL-Mz; Tue, 29 Aug 2006 12:16:18 +0100 Date: Tue, 29 Aug 2006 12:16:18 +0100 From: Brian Candler To: Morgan Message-ID: <20060829111618.GA5670@uk.tiscali.com> References: <02df01c6c84c$dcd564c0$4345a8c0@phobos> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <02df01c6c84c$dcd564c0$4345a8c0@phobos> User-Agent: Mutt/1.4.2.1i Cc: freebsd-net@freebsd.org Subject: Re: Optimizing a high-latency connection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Aug 2006 11:16:23 -0000 On Fri, Aug 25, 2006 at 03:46:22PM +0200, Morgan wrote: > I'm trying som file transfers across the globe. The RTT is almost 400ms and > the transfer rate is painfully slow. There are 24 router hops on the path > and I assume most of the problem is there The number of routers isn't an issue, as long as they forward packets without dropping them on the floor. > but I wonder if there are any > sysctl variables I can trim on my side to help this problem slightly? I seem > to understand that there need to be an ACK received for at least every other > packet No, the two sides negotiate a 'window' which is the amount of outstanding unacknowledged traffic in transit. tcpdump will show you the negotiated window size. As long as the window is greater than the (bandwidth * RTT) product, you should be able to fill the pipe. e.g. a 2Mbps pipe (256KBps) with 400ms RTT has a delay-bandwidth product of 0.4*256 = 102.4KB You need to define "painfully slow", but my guess is that your problems are due to packet loss. Each lost packet causes the window size to be reduced by half, and it will only gradually creep back up. This is part of TCP's congestion control mechanism, since TCP equates packet loss with congestion, indicating that the transmission speed must be reduced. You can check your packet loss like this: # ping -s1472 -c100 x.x.x.x where x.x.x.x is the remote endpoint. (This sends 1500-byte sized packets: 20 bytes IP header + 8 bytes ICMP header + 1472 bytes data padding). TCP performs extremely poorly at packet loss of around 2% and above. If you see a large level of packet loss, by changing x.x.x.x to point to intermediate routers shown by traceroute, you may be able to identify where the packet loss is taking place. An ethernet full-duplex/half-duplex mismatch can cause packet loss in the 2% range, which is not enough for the network to be visibly "broken", but enough for it to perform very badly. Worst offenders here are old Cisco Catalyst 29xx switches, which don't perform auto-negotation properly. Regards, Brian.