Date: Wed, 9 Nov 2022 11:18:21 +0800 From: Zhenlei Huang <zlei.huang@gmail.com> To: Hans Petter Selasky <hps@selasky.org> Cc: Michael Tuexen <michael.tuexen@lurchi.franken.de>, freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs Message-ID: <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com> In-Reply-To: <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org> References: <75D35F36-7759-4168-ADBA-C2414F5B53BC@gmail.com> <712641B3-5196-40CC-9B64-04637F16F649@lurchi.franken.de> <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> <0FED34A9-D093-442A-83B7-08C06D11F8B5@lurchi.franken.de> <330A9146-F7CC-4CAB-9003-2F90B872AC3E@gmail.com> <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <hps@selasky.org> wrote: > > Hi, > > Some thoughts about this topic. Sorry for late response. > > Delaying ACKs means loss of performance when using Gigabit TCP connections in data centers. There it is important to ACK the data as quick as possible, to avoid running out of TCP window space. Thinking about TCP connections at 30 GBit/s and above! In data centers, the bandwidth is much more and the latency is extremely low (compared to WAN), sub-milliseconds . The TCP window space is bandwidth multiply RTT. For a 30 GBit/s network it is about 750KiB . I think that is trivial for a datacenter server. 4.2.3.2 in RFC 1122 states: > in a stream of full-sized segments there SHOULD be an ACK for at least every second segment Even if the ACK every tenth segment, the impact of delayed ACKs on TCP window is not significant ( at most ten segments not ACKed in TCP send window ). Anyway, for datacenter usage the bandwidth is symmetric and the reverse path ( TX path of receiver ) is sufficient. Servers can even ACK every segment (no delaying ACK). > > I think the implementation should be exactly like it is. > > There is a software LRO in FreeBSD to coalesce the ACKs before they hit the network stack, so there are no real problems there. I'm OK with the current implementation. I think upper layers (or application) have (business) information to indicate whether delaying ACKs should be employed. After googling I found there's a draft [1]. [1] Sender Control of Delayed Acknowledgments in TCP: https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml > > --HPS > > Best regards, Zhenlei [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <<a href="mailto:hps@selasky.org" class="">hps@selasky.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi,<br class=""><br class="">Some thoughts about this topic.<br class=""></div></div></blockquote><div><br class=""></div><div>Sorry for late response.</div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">Delaying ACKs means loss of performance when using Gigabit TCP connections in data centers. There it is important to ACK the data as quick as possible, to avoid running out of TCP window space. Thinking about TCP connections at 30 GBit/s and above!<br class=""></div></div></blockquote><div><br class=""></div><div>In data centers, the bandwidth is much more and the latency is extremely low (compared to WAN), <span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">sub-milliseconds</span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""> </span>.</div><div>The TCP window space is <span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">bandwidth multiply RTT. For a 30 GBit/s network it is about 750KiB . I think that is trivial for a</span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">datacenter server.</span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class=""></span></div><div><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">4.2.3.2 in </span></font><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">RFC 1122 states:</span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">> </span><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">in a stream of full-sized </span></font><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">segments there SHOULD be an ACK for at least every second </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">segment </span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">Even if the ACK every tenth </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">segment, the impact of delayed ACKs on TCP window is not significant ( at most</span></div><div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""> ten segments </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">not ACKed in TCP send window ).</span></div><div><br class=""></div><div>Anyway, for datacenter usage the <font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">bandwidth is symmetric and the reverse path ( TX path of receiver ) is sufficient.</span></font></div><div><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">Servers can even ACK every </span></font><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">segment (no delaying ACK).</span></div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class=""><br class="">I think the implementation should be exactly like it is.<br class=""><br class="">There is a software LRO in FreeBSD to coalesce the ACKs before they hit the network stack, so there are no real problems there.<br class=""></div></div></blockquote><div><br class=""></div><div>I'm OK with the current implementation.</div><div><br class=""></div><div>I think upper layers (or application) have (business) information to indicate whether delaying ACKs should be employed.</div><div>After googling I found there's a draft [1].</div><div><br class=""></div><div>[1] Sender Control of Delayed Acknowledgments in TCP: <a href="https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml" class="">https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml</a></div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">--HPS<br class=""><br class=""><br class=""></div></div></blockquote></div><br class=""><div class=""><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Best regards,</div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Zhenlei</div></div></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5A501643-1E81-4A8C-8DDC-094371DC03D7>