Date: Tue, 7 Oct 2008 18:19:37 GMT From: Renaud Lienhart <renaud@vmware.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/127928: The TCP bandwidth gets squeezed every time tcp_xmit_bandwidth_limit() kicks in Message-ID: <200810071819.m97IJblE018682@www.freebsd.org> Resent-Message-ID: <200810071820.m97IK18b023817@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 127928
>Category: kern
>Synopsis: The TCP bandwidth gets squeezed every time tcp_xmit_bandwidth_limit() kicks in
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Oct 07 18:20:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: Renaud Lienhart
>Release: FreeBSD 6.2+
>Organization:
VMware, Inc.
>Environment:
ESX 3.x
>Description:
FreeBSD 6,7 & 8 have a bug in their tcp_xmit_bandwidth_limit() function.
The problem is that the bandwidth calculation is a 1/16th weigthed average and that the saved bandwidth gets reset every time the RTT threshold is crossed over. This means that the bandwidth (and thus the B-D product, and thus the sending window) is briefly squeezed whenever the mechanism kicks in and it takes
a bit of time to reach sane values again.
The threshold by default is 10ms. With HZ=100, this means that a RTT of 0 tick disables the mechanism and a RTT of 1 tick activates it. Because of the poor tick granularity, the RTT is of 1 tick every 10ms and the mechanism kicks in, brutally squeezing the sending window because tp->snd_bandwidth is 0.
The problem also appears with HZ=1000, when the RTT fluctuates in the bad spot of ~10ms.
>How-To-Repeat:
Run a kernel with HZ=100 (or HZ=1000 and a latency around ~10ms, which is harder) and any TCP load with tcp_inflight_debug = 1. Notice the bandwidth reported by the log is inconsistent with the link capacity and that the tp->snd_bwnd is incorrectly low.
>Fix:
To fix this, kickstart tp->snd_bandwidth without the weighted average smoothing when it is 0.
Patch attached with submission follows:
Index: netinet/tcp_subr.c
===================================================================
--- netinet/tcp_subr.c (revision 183668)
+++ netinet/tcp_subr.c (working copy)
@@ -1783,7 +1783,9 @@
tp->t_bw_rtseq = ack_seq;
if (tp->t_bw_rtttime == 0 || (int)bw < 0)
return;
- bw = ((int64_t)tp->snd_bandwidth * 15 + bw) >> 4;
+ if (tp->snd_bandwidth != 0) {
+ bw = ((int64_t)tp->snd_bandwidth * 15 + bw) >> 4;
+ }
tp->snd_bandwidth = bw;
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200810071819.m97IJblE018682>