Date: Wed, 9 Nov 2022 11:18:21 +0800 From: Zhenlei Huang <zlei.huang@gmail.com> To: Hans Petter Selasky <hps@selasky.org> Cc: Michael Tuexen <michael.tuexen@lurchi.franken.de>, freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs Message-ID: <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com> In-Reply-To: <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org> References: <75D35F36-7759-4168-ADBA-C2414F5B53BC@gmail.com> <712641B3-5196-40CC-9B64-04637F16F649@lurchi.franken.de> <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> <0FED34A9-D093-442A-83B7-08C06D11F8B5@lurchi.franken.de> <330A9146-F7CC-4CAB-9003-2F90B872AC3E@gmail.com> <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_E9272696-4607-444D-8AB0-78AE1A156DF7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <hps@selasky.org> = wrote: >=20 > Hi, >=20 > Some thoughts about this topic. Sorry for late response. >=20 > Delaying ACKs means loss of performance when using Gigabit TCP = connections in data centers. There it is important to ACK the data as = quick as possible, to avoid running out of TCP window space. Thinking = about TCP connections at 30 GBit/s and above! In data centers, the bandwidth is much more and the latency is extremely = low (compared to WAN), sub-milliseconds . The TCP window space is bandwidth multiply RTT. For a 30 GBit/s network = it is about 750KiB . I think that is trivial for a datacenter server. 4.2.3.2 in RFC 1122 states: > in a stream of full-sized segments there SHOULD be an ACK for at least = every second segment=20 Even if the ACK every tenth segment, the impact of delayed ACKs on TCP = window is not significant ( at most ten segments not ACKed in TCP send window ). Anyway, for datacenter usage the bandwidth is symmetric and the reverse = path ( TX path of receiver ) is sufficient. Servers can even ACK every segment (no delaying ACK). >=20 > I think the implementation should be exactly like it is. >=20 > There is a software LRO in FreeBSD to coalesce the ACKs before they = hit the network stack, so there are no real problems there. I'm OK with the current implementation. I think upper layers (or application) have (business) information to = indicate whether delaying ACKs should be employed. After googling I found there's a draft [1]. [1] Sender Control of Delayed Acknowledgments in TCP: = https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml >=20 > --HPS >=20 >=20 Best regards, Zhenlei= --Apple-Mail=_E9272696-4607-444D-8AB0-78AE1A156DF7 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On = Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <<a = href=3D"mailto:hps@selasky.org" class=3D"">hps@selasky.org</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"">Hi,<br class=3D""><br class=3D"">Some thoughts about this = topic.<br class=3D""></div></div></blockquote><div><br = class=3D""></div><div>Sorry for late response.</div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = class=3D""><br class=3D"">Delaying ACKs means loss of performance when = using Gigabit TCP connections in data centers. There it is important to = ACK the data as quick as possible, to avoid running out of TCP window = space. Thinking about TCP connections at 30 GBit/s and above!<br = class=3D""></div></div></blockquote><div><br class=3D""></div><div>In = data centers, the bandwidth is much more and the latency is extremely = low (compared to WAN), <span style=3D"caret-color: rgb(0, 0, 0); = color: rgb(0, 0, 0);" class=3D"">sub-milliseconds</span><span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" = class=3D""> </span>.</div><div>The TCP window space is <span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" = class=3D"">bandwidth multiply RTT. For a 30 GBit/s network it is about = 750KiB . I think that is trivial for a</span></div><div><span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" = class=3D"">datacenter server.</span></div><div><span style=3D"caret-color:= rgb(0, 0, 0); color: rgb(0, 0, 0);" class=3D""><br = class=3D""></span></div><div><font color=3D"#000000" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">4.2.3.2 = in </span></font><span style=3D"caret-color: rgb(0, 0, 0); color: = rgb(0, 0, 0);" class=3D"">RFC 1122 states:</span></div><div><span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" = class=3D"">> </span><font color=3D"#000000" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">in a stream of = full-sized </span></font><span style=3D"caret-color: rgb(0, 0, 0); = color: rgb(0, 0, 0);" class=3D"">segments there SHOULD be an ACK for at = least every second </span><span style=3D"caret-color: rgb(0, 0, 0); = color: rgb(0, 0, 0);" class=3D"">segment </span></div><div><span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=3D"">Even = if the ACK every tenth </span><span style=3D"caret-color: rgb(0, 0, = 0); color: rgb(0, 0, 0);" class=3D"">segment, the impact of delayed ACKs = on TCP window is not significant ( at most</span></div><div><span = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" = class=3D""> ten segments </span><span style=3D"caret-color: = rgb(0, 0, 0); color: rgb(0, 0, 0);" class=3D"">not ACKed in TCP send = window ).</span></div><div><br class=3D""></div><div>Anyway, for = datacenter usage the <font color=3D"#000000" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">bandwidth = is symmetric and the reverse path ( TX path of receiver ) is = sufficient.</span></font></div><div><font color=3D"#000000" = class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Servers = can even ACK every </span></font><span style=3D"caret-color: rgb(0, = 0, 0); color: rgb(0, 0, 0);" class=3D"">segment (no delaying = ACK).</span></div><div><br class=3D""></div><blockquote type=3D"cite" = class=3D""><div class=3D""><div class=3D""><br class=3D"">I think the = implementation should be exactly like it is.<br class=3D""><br = class=3D"">There is a software LRO in FreeBSD to coalesce the ACKs = before they hit the network stack, so there are no real problems = there.<br class=3D""></div></div></blockquote><div><br = class=3D""></div><div>I'm OK with the current = implementation.</div><div><br class=3D""></div><div>I think upper layers = (or application) have (business) information to indicate whether = delaying ACKs should be employed.</div><div>After googling I found = there's a draft [1].</div><div><br class=3D""></div><div>[1] Sender = Control of Delayed Acknowledgments in TCP: <a = href=3D"https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs= -01.xml" = class=3D"">https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-r= eqs-01.xml</a></div><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div class=3D""><br class=3D"">--HPS<br = class=3D""><br class=3D""><br = class=3D""></div></div></blockquote></div><br class=3D""><div = class=3D""><div style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, = 0);">Best regards,</div><div style=3D"caret-color: rgb(0, 0, 0); color: = rgb(0, 0, 0);">Zhenlei</div></div></body></html>= --Apple-Mail=_E9272696-4607-444D-8AB0-78AE1A156DF7--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5A501643-1E81-4A8C-8DDC-094371DC03D7>