Date: Fri, 01 Jun 2012 12:48:12 +1000 From: Lawrence Stewart <lstewart@freebsd.org> To: Kevin Oberman <kob6558@gmail.com> Cc: freebsd-net@freebsd.org, Andrew Gallatin <gallatin@cs.duke.edu>, Andrew Gallatin <gallatin@myri.com> Subject: Re: Major performance hit with ToS setting Message-ID: <4FC82D6C.4050309@freebsd.org> In-Reply-To: <CAN6yY1v%2Bvf=SW7WDGHxCkJtOdj8K3f450jNxFWK_Jc%2B-pFg0nA@mail.gmail.com> References: <CAN6yY1sLxFJ18ANO7nQqLetnJiT-K6pHC-X3yT1dWuWGa0VLUg@mail.gmail.com> <4FBF88CE.20209@cs.duke.edu> <CAN6yY1v%2Bvf=SW7WDGHxCkJtOdj8K3f450jNxFWK_Jc%2B-pFg0nA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 05/31/12 13:33, Kevin Oberman wrote:
[snip]
> I used SIFTR at the suggestion of Lawrence Stewart who headed the
> project to bring plugable congestion algorithms to FreeBSD and found
> really odd congestion behavior. First, I do see a triple ACK, but the
> congestion window suddenly drops from 73K to 8K. If I understand
> CUBIC, it should half the congestion window, not what is happening..
> It then increases slowly (in slow start) to 82K. while the slow-start
> bytes are INCREASING, the congestion window again goes to 8K while the
> SS size moves from 36K up to 52K. It just continues to bound wildly
> between 8K (always the low point) and between 64k and 82K. The swings
> start at 83K and, over the first few seconds the peaks drop to about
> 64K.
Oh, and a comment about this behaviour. Dropping back to 8k (1MSS) is
only nasty if the TF_{CONG|FAST}RECOVERY flags are *not* set i.e. if you
see cwnd grow, drop to 8k with those flags set, and then when the flags
are unset, cwnd starts at the value of ssthresh, then that is perfectly
normal recovery behaviour. What *is* nasty is if an RTO fires, which
will reset cwnd to 8k, ssthresh to 2*MSS and make the connection
effectively start from scratch again.
There is evidence of RTOs in your siftr output, which is bad news e.g
here's one example of 2 side-by-side log lines from your trace:
# Direction,time,ssthresh,cwnd,flags
i,1338319593.574706,27044,27044,1630544864
o,1338319593.831482,16384,8192,1092625377
Note the 300ms gap, and how cwnd resets to 1MSS and flags go from
1630544864
(TF_WASCRECOVERY|TF_CONGRECOVERY|TF_WASFRECOVERY|TF_FASTRECOVERY) to
1092625377 (TF_WASCRECOVERY|TF_WASFRECOVERY).
Cheers,
Lawrence
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4FC82D6C.4050309>