Date: Thu, 13 Jan 2005 15:34:03 -0800 From: Julian Elischer <julian@elischer.org> To: Andre Oppermann <andre@freebsd.org> Cc: net@freebsd.org Subject: Re: TCP out-of-order packets. Message-ID: <41E7056B.4010805@elischer.org> In-Reply-To: <41E6EADC.2AFAEA47@freebsd.org> References: <41E5C9D8.4090209@elischer.org> <20050113055111.GA11141@odin.ac.hmc.edu> <41E6C564.50005@elischer.org> <41E6EADC.2AFAEA47@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Andre Oppermann wrote: >Julian Elischer wrote: > > >>Brooks Davis wrote: >> >> >> >>>On Wed, Jan 12, 2005 at 05:07:36PM -0800, Julian Elischer wrote: >>> >>> >>> >>> >>>>I have a link which is provided by someone else that is 7 x E1s aggregated. >>>>At leat it looks that way to me when I get to see it. however I have >>>>only been able to get >>>>60kB.sec across this, despite having a tcp window size of 131072 bytes.. >>>>After investigation it appears that the link is massively re-orderring >>>>packets. >>>>groups of upto 10 packets may appear in random order. (Maybe more, bu tI >>>>have seen 10) >>>> >>>>in fact packets are rarely IN order. >>>> >>>>This plays havoc with the tcp sessions. >>>> >>>>I was thinking of writing a hacked up version of NATD that >>>>instead of doing NAT, just did a pre-sort on packets from each session, >>>>so that the receiver would >>>>see a stream of IN-order packets, with occasional delays. >>>> >>>>firstly, does anyone have any tools to do this already (why build when >>>>you can borrow) >>>>and secondly, does anyone have any experience with this sort of problem? >>>> >>>>I have no control over or access to the link.. all I have is a promise >>>>that they will deliver >>>>14Mb/Sec. with approc 300mSec. RTT to me but there is no promise about >>>>packet order. >>>> >>>>I just get a 100Mb ethernet cable. >>>> >>>> >>>> >>>> >>>> >>>>I wonder if there's a way to turn off the sender backoff? >>>> >>>> > >Not directly. What you actually want is to delay the generating of >ACK's for a certain amount of time (some milli-seconds) to aggregate >the out-of-order packets into one ACK and to avoid the backoff from >the other side. > > yes, here's a trace from the receiver's end showing the order of received packets.. RTT on this link is about 300mSec. My original answer would be to hack NATD to re-order the packets. and use DIVERT to it. DeltaT SEQ-SRTR:SEQ_END packet# ------------------------------------------ 346853 2856:2920(64) 1 370821 2920:4368(1448) 2 004410 8712:10160(1448) 6 007848 12608:14056(1448) 9 007057 18400:19848(1448) 13 004027 11160:12608(1448) 8 005553 4368:5816(1448) 3 002390 16952:18400(1448) 12 005745 7264:8712(1448) 5 001204 5816:7264(1448) 4 008326 14056:15504(1448) 10 002555 10160:11160(1000) 7 004091 19848:21296(1448) 14 004287 15504:16952(1448) 11 358289 22744:24192(1448) 16 001891 21296:22744(1448) 15 048289 4368:5816(1448) DUP of 3 025538 5816:7264(1448) DUP of 4 018761 10160:11608(1448) DUP of 7 062939 25640:27088(1448) 18 005493 15504:16952(1448) DUP of 11 007903 24192:25640(1448) 17 002207 27088:28536(1448) 19 028675 21296:22744(1448) DUP of 15 176400 28536:29984(1448) 20 020889 29984:31432(1448) 21 092304 24192:25640(1448) DUP of 17 038396 31432:32880(1448) 22 015146 32880:34328(1448) 23 010362 27088:28536(1448) DUP of 19 027400 35776:37224(1448) 25 008269 28536:29984(1448) DUP of 20 012537 34328:35776(1448) 24 294035 37224:38672(1448) 26 078059 34328:35776(1448) DUP of 24 267701 40120:41568(1448) 28 017393 38672:40120(1448) 27 019769 41568:43016(1448) 29 339347 44464:45912(1448) 31 003887 43016:44464(1448) 30 123755 45912:47360(1448) 32 228883 47360:48808(1448) 33 013437 48808:50256(1448) 34 219579 50256:51704(1448) 35 144501 51704:53152(1448) 36 001200 53152:54600(1448) 37 029175 54600:56048(1448) 38 183158 56048:57496(1448) 39 157127 58944:60392(1448) 40 By here the sender has backed right off. > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?41E7056B.4010805>