Date: Thu, 13 Jan 2005 11:31:35 -0800 From: Julian Elischer <julian@elischer.org> To: Brooks Davis <brooks@one-eyed-alien.net> Cc: net@freebsd.org Subject: Re: TCP out-of-order packets. Message-ID: <41E6CC97.90603@elischer.org> In-Reply-To: <20050113190953.GC28303@odin.ac.hmc.edu> References: <41E5C9D8.4090209@elischer.org> <20050113055111.GA11141@odin.ac.hmc.edu> <41E6C564.50005@elischer.org> <20050113190953.GC28303@odin.ac.hmc.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Brooks Davis wrote: >On Thu, Jan 13, 2005 at 11:00:52AM -0800, Julian Elischer wrote: > > >>Brooks Davis wrote: >> >> >> >>>On Wed, Jan 12, 2005 at 05:07:36PM -0800, Julian Elischer wrote: >>> >>> >>> >>> >>>>I have a link which is provided by someone else that is 7 x E1s >>>>aggregated. >>>>At leat it looks that way to me when I get to see it. however I have >>>>only been able to get >>>>60kB.sec across this, despite having a tcp window size of 131072 bytes.. >>>>After investigation it appears that the link is massively re-orderring >>>>packets. >>>>groups of upto 10 packets may appear in random order. (Maybe more, bu tI >>>>have seen 10) >>>> >>>>in fact packets are rarely IN order. >>>> >>>>This plays havoc with the tcp sessions. >>>> >>>>I was thinking of writing a hacked up version of NATD that >>>>instead of doing NAT, just did a pre-sort on packets from each session, >>>>so that the receiver would >>>>see a stream of IN-order packets, with occasional delays. >>>> >>>>firstly, does anyone have any tools to do this already (why build when >>>>you can borrow) >>>>and secondly, does anyone have any experience with this sort of problem? >>>> >>>>I have no control over or access to the link.. all I have is a promise >>>>that they will deliver >>>>14Mb/Sec. with approc 300mSec. RTT to me but there is no promise about >>>>packet order. >>>> >>>>I just get a 100Mb ethernet cable. >>>> >>>> >>>> >>>> >>>Have you tried Andre's TCP reassembly rewrite? He says he saw >>>significant improvements in the face of major reordering. >>> >>> >>> >>> >>I don't think it's a problem with reassembly overhead, but rather a >>symptom of sender >>backoff when confronted with multiple duplicate acks due to the receiver >>getting the packets out of order. >> >>I wonder if there's a way to turn off the sender backoff? >> >> >> >>>http://www.mail-archive.com/freebsd-net@freebsd.org/msg14064.html >>> >>> >>> >>These machines are production machines on a custommer site.. running 4.8 >>it would be significant work to put the rewrite in (to 4.8) and a lot >>of red tape to >>reboot one to the new kernel.. :-/ >> >> > >Hmm, what about using a bridge with pf's TCP normalizer? You'd probably >have to raise your socket buffer sizes to handle the added latency (not >sure how much that is), but that might be an option (assuming adding >another machine wasn't harder then installing a new kernel). That code >is at least already written so you could also potentially crib from it >for the hacked up natd idea. > I have remote access to a handful of machines on a switch. I do not have access ot the switch or the wiring or the link.. for added points the machines are in India. > >-- Brooks > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?41E6CC97.90603>