Date: Thu, 13 Jan 2005 11:09:53 -0800 From: Brooks Davis <brooks@one-eyed-alien.net> To: Julian Elischer <julian@elischer.org> Cc: net@freebsd.org Subject: Re: TCP out-of-order packets. Message-ID: <20050113190953.GC28303@odin.ac.hmc.edu> In-Reply-To: <41E6C564.50005@elischer.org> References: <41E5C9D8.4090209@elischer.org> <20050113055111.GA11141@odin.ac.hmc.edu> <41E6C564.50005@elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--TiqCXmo5T1hvSQQg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 13, 2005 at 11:00:52AM -0800, Julian Elischer wrote: >=20 >=20 > Brooks Davis wrote: >=20 > >On Wed, Jan 12, 2005 at 05:07:36PM -0800, Julian Elischer wrote: > >=20 > > > >>I have a link which is provided by someone else that is 7 x E1s=20 > >>aggregated. > >>At leat it looks that way to me when I get to see it. however I have=20 > >>only been able to get > >>60kB.sec across this, despite having a tcp window size of 131072 bytes.. > >>After investigation it appears that the link is massively re-orderring= =20 > >>packets. > >>groups of upto 10 packets may appear in random order. (Maybe more, bu t= I=20 > >>have seen 10) > >> > >>in fact packets are rarely IN order. > >> > >>This plays havoc with the tcp sessions. > >> > >>I was thinking of writing a hacked up version of NATD that > >>instead of doing NAT, just did a pre-sort on packets from each session,= =20 > >>so that the receiver would > >>see a stream of IN-order packets, with occasional delays. > >> > >>firstly, does anyone have any tools to do this already (why build when= =20 > >>you can borrow) > >>and secondly, does anyone have any experience with this sort of problem? > >> > >>I have no control over or access to the link.. all I have is a promise= =20 > >>that they will deliver > >>14Mb/Sec. with approc 300mSec. RTT to me but there is no promise about= =20 > >>packet order. > >> > >>I just get a 100Mb ethernet cable. > >> =20 > >> > > > >Have you tried Andre's TCP reassembly rewrite? He says he saw > >significant improvements in the face of major reordering. > >=20 > > >=20 > I don't think it's a problem with reassembly overhead, but rather a=20 > symptom of sender > backoff when confronted with multiple duplicate acks due to the receiver > getting the packets out of order. >=20 > I wonder if there's a way to turn off the sender backoff? >=20 > >http://www.mail-archive.com/freebsd-net@freebsd.org/msg14064.html > > >=20 > These machines are production machines on a custommer site.. running 4.8 > it would be significant work to put the rewrite in (to 4.8) and a lot=20 > of red tape to > reboot one to the new kernel.. :-/ Hmm, what about using a bridge with pf's TCP normalizer? You'd probably have to raise your socket buffer sizes to handle the added latency (not sure how much that is), but that might be an option (assuming adding another machine wasn't harder then installing a new kernel). That code is at least already written so you could also potentially crib from it for the hacked up natd idea. -- Brooks --=20 Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 --TiqCXmo5T1hvSQQg Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFB5seAXY6L6fI4GtQRAt6KAJ0fIJjLOwv1jpuGGXaps0R/MM+AfACg1BU6 F7Gx8BH6IzU/ni3K74O5nl8= =UVvx -----END PGP SIGNATURE----- --TiqCXmo5T1hvSQQg--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050113190953.GC28303>