From owner-freebsd-net@FreeBSD.ORG Tue May 2 11:16:51 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2256616A40B for ; Tue, 2 May 2006 11:16:51 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 74F5143D48 for ; Tue, 2 May 2006 11:16:50 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 9064646B87; Tue, 2 May 2006 07:16:49 -0400 (EDT) Date: Tue, 2 May 2006 12:16:49 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Brian Candler In-Reply-To: <20060502100650.GB30251@uk.tiscali.com> Message-ID: <20060502121203.U92256@fledge.watson.org> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <20060427093916.GC84148@obiwan.tataz.chchile.org> <20060427145252.I75848@fledge.watson.org> <20060427143814.GD84148@obiwan.tataz.chchile.org> <20060427154718.J75848@fledge.watson.org> <20060502100650.GB30251@uk.tiscali.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marcos Bedinelli , freebsd-net@freebsd.org, Jeremie Le Hen , lukem.freebsd@cse.unsw.edu.au Subject: Re: [fbsd] Re: [fbsd] Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2006 11:16:51 -0000 On Tue, 2 May 2006, Brian Candler wrote: > On Mon, May 01, 2006 at 11:38:39AM +1000, lukem.freebsd@cse.unsw.edu.au wrote: >> Would it be possible to improve the behaviour of the TCP protocol >> implementation so that out-of-order reception was acceptable? > > Possibly - but if your FreeBSD box is acting as a router, and it re-orders > packets in transit to the rest of the Internet, then in principle you'd need > to upgrade the TCP implementations on every other device in the Internet :-( The behavior in question is referred to as a fast retransmit. The basic idea is this: if three datagrams are sent by the remote stack in order, and you receive two datagrams with a clear "missing" middle one, then you can assume that the middle one went missing since network topologies have been carefully crafted to meet TCP's assumptions of ordering. You hint to the remote stack this has happened by ACK'ing the first segment twice. (I may have the details not quite right here, but the effect is the same). Life gets a bit better with SACK. Changing this assumption is very difficult -- it's not a property purely of our implementation, as much as a property of the wire protocol, and written into the standards. Even if we don't play by that rule, other stacks will. In particular, this is an important case when you're routing with FreeBSD -- if you reorder packets in the routing path, end hosts on either side of it will get very upset. In this case, FreeBSD isn't even a direct party to TCP, and so can't change the behavior. This isn't just a property of TCP, many protocols perform badly in the presence of reordering, and so it's really best to avoid it wherever possible, and we go to significant lengths to do so. The trick is to work away at the edges and try to (over time) move towards the minimal ordering constraints. Robert N M Watson