From owner-freebsd-net@FreeBSD.ORG Thu Jan 13 19:00:54 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ED9C516A5DE for ; Thu, 13 Jan 2005 19:00:54 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id B7A2143D2D for ; Thu, 13 Jan 2005 19:00:52 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id 527867A403; Thu, 13 Jan 2005 11:00:52 -0800 (PST) Message-ID: <41E6C564.50005@elischer.org> Date: Thu, 13 Jan 2005 11:00:52 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Brooks Davis References: <41E5C9D8.4090209@elischer.org> <20050113055111.GA11141@odin.ac.hmc.edu> In-Reply-To: <20050113055111.GA11141@odin.ac.hmc.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: net@freebsd.org Subject: Re: TCP out-of-order packets. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jan 2005 19:00:55 -0000 Brooks Davis wrote: >On Wed, Jan 12, 2005 at 05:07:36PM -0800, Julian Elischer wrote: > > >>I have a link which is provided by someone else that is 7 x E1s aggregated. >>At leat it looks that way to me when I get to see it. however I have >>only been able to get >>60kB.sec across this, despite having a tcp window size of 131072 bytes.. >>After investigation it appears that the link is massively re-orderring >>packets. >>groups of upto 10 packets may appear in random order. (Maybe more, bu tI >>have seen 10) >> >>in fact packets are rarely IN order. >> >>This plays havoc with the tcp sessions. >> >>I was thinking of writing a hacked up version of NATD that >>instead of doing NAT, just did a pre-sort on packets from each session, >>so that the receiver would >>see a stream of IN-order packets, with occasional delays. >> >>firstly, does anyone have any tools to do this already (why build when >>you can borrow) >>and secondly, does anyone have any experience with this sort of problem? >> >>I have no control over or access to the link.. all I have is a promise >>that they will deliver >>14Mb/Sec. with approc 300mSec. RTT to me but there is no promise about >>packet order. >> >>I just get a 100Mb ethernet cable. >> >> > >Have you tried Andre's TCP reassembly rewrite? He says he saw >significant improvements in the face of major reordering. > > I don't think it's a problem with reassembly overhead, but rather a symptom of sender backoff when confronted with multiple duplicate acks due to the receiver getting the packets out of order. I wonder if there's a way to turn off the sender backoff? >http://www.mail-archive.com/freebsd-net@freebsd.org/msg14064.html > These machines are production machines on a custommer site.. running 4.8 it would be significant work to put the rewrite in (to 4.8) and a lot of red tape to reboot one to the new kernel.. :-/ > >-- Brooks > > >