From owner-freebsd-net@FreeBSD.ORG Thu Jan 13 21:40:43 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AB35616A4CE for ; Thu, 13 Jan 2005 21:40:43 +0000 (GMT) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E03C643D1F for ; Thu, 13 Jan 2005 21:40:42 +0000 (GMT) (envelope-from andre@freebsd.org) Received: (qmail 88590 invoked from network); 13 Jan 2005 21:24:05 -0000 Received: from unknown (HELO freebsd.org) ([62.48.0.54]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 13 Jan 2005 21:24:05 -0000 Message-ID: <41E6EADC.2AFAEA47@freebsd.org> Date: Thu, 13 Jan 2005 22:40:44 +0100 From: Andre Oppermann X-Mailer: Mozilla 4.8 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer References: <41E5C9D8.4090209@elischer.org> <41E6C564.50005@elischer.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: net@freebsd.org Subject: Re: TCP out-of-order packets. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jan 2005 21:40:43 -0000 Julian Elischer wrote: > > Brooks Davis wrote: > > >On Wed, Jan 12, 2005 at 05:07:36PM -0800, Julian Elischer wrote: > > > > > >>I have a link which is provided by someone else that is 7 x E1s aggregated. > >>At leat it looks that way to me when I get to see it. however I have > >>only been able to get > >>60kB.sec across this, despite having a tcp window size of 131072 bytes.. > >>After investigation it appears that the link is massively re-orderring > >>packets. > >>groups of upto 10 packets may appear in random order. (Maybe more, bu tI > >>have seen 10) > >> > >>in fact packets are rarely IN order. > >> > >>This plays havoc with the tcp sessions. > >> > >>I was thinking of writing a hacked up version of NATD that > >>instead of doing NAT, just did a pre-sort on packets from each session, > >>so that the receiver would > >>see a stream of IN-order packets, with occasional delays. > >> > >>firstly, does anyone have any tools to do this already (why build when > >>you can borrow) > >>and secondly, does anyone have any experience with this sort of problem? > >> > >>I have no control over or access to the link.. all I have is a promise > >>that they will deliver > >>14Mb/Sec. with approc 300mSec. RTT to me but there is no promise about > >>packet order. > >> > >>I just get a 100Mb ethernet cable. > >> > >> > > > >Have you tried Andre's TCP reassembly rewrite? He says he saw > >significant improvements in the face of major reordering. > > > > > > I don't think it's a problem with reassembly overhead, but rather a > symptom of sender > backoff when confronted with multiple duplicate acks due to the receiver > getting the packets out of order. > > I wonder if there's a way to turn off the sender backoff? Not directly. What you actually want is to delay the generating of ACK's for a certain amount of time (some milli-seconds) to aggregate the out-of-order packets into one ACK and to avoid the backoff from the other side. -- Andre