Date: Thu, 10 Nov 2022 16:07:08 +0800 From: Zhenlei Huang <zlei.huang@gmail.com> To: Hans Petter Selasky <hps@selasky.org> Cc: Michael Tuexen <michael.tuexen@lurchi.franken.de>, freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs Message-ID: <7EDD65B7-5FCD-42E1-A9E8-AA5139B0A81E@gmail.com> In-Reply-To: <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com> References: <75D35F36-7759-4168-ADBA-C2414F5B53BC@gmail.com> <712641B3-5196-40CC-9B64-04637F16F649@lurchi.franken.de> <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> <0FED34A9-D093-442A-83B7-08C06D11F8B5@lurchi.franken.de> <330A9146-F7CC-4CAB-9003-2F90B872AC3E@gmail.com> <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org> <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On Nov 9, 2022, at 11:18 AM, Zhenlei Huang <zlei.huang@gmail.com> wrote: > > >> On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <hps@selasky.org <mailto:hps@selasky.org>> wrote: >> >> Hi, >> >> Some thoughts about this topic. > > Sorry for late response. > >> >> Delaying ACKs means loss of performance when using Gigabit TCP connections in data centers. There it is important to ACK the data as quick as possible, to avoid running out of TCP window space. Thinking about TCP connections at 30 GBit/s and above! > > In data centers, the bandwidth is much more and the latency is extremely low (compared to WAN), sub-milliseconds . > The TCP window space is bandwidth multiply RTT. For a 30 GBit/s network it is about 750KiB . I think that is trivial for a > datacenter server. > > 4.2.3.2 in RFC 1122 states: > > in a stream of full-sized segments there SHOULD be an ACK for at least every second segment > Even if the ACK every tenth segment, the impact of delayed ACKs on TCP window is not significant ( at most > ten segments not ACKed in TCP send window ). > > Anyway, for datacenter usage the bandwidth is symmetric and the reverse path ( TX path of receiver ) is sufficient. > Servers can even ACK every segment (no delaying ACK). > >> >> I think the implementation should be exactly like it is. >> >> There is a software LRO in FreeBSD to coalesce the ACKs before they hit the network stack, so there are no real problems there. > > I'm OK with the current implementation. > > I think upper layers (or application) have (business) information to indicate whether delaying ACKs should be employed. > After googling I found there's a draft [1]. > > [1] Sender Control of Delayed Acknowledgments in TCP: https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml <https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml>; Found the html / pdf / txt version of the draft RFC. https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/ > >> >> --HPS >> >> > > Best regards, > Zhenlei [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div><blockquote type="cite" class=""><div class="">On Nov 9, 2022, at 11:18 AM, Zhenlei Huang <<a href="mailto:zlei.huang@gmail.com" class="">zlei.huang@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=us-ascii" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <<a href="mailto:hps@selasky.org" class="">hps@selasky.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi,<br class=""><br class="">Some thoughts about this topic.<br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">Sorry for late response.</div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">Delaying ACKs means loss of performance when using Gigabit TCP connections in data centers. There it is important to ACK the data as quick as possible, to avoid running out of TCP window space. Thinking about TCP connections at 30 GBit/s and above!<br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">In data centers, the bandwidth is much more and the latency is extremely low (compared to WAN), <span style="caret-color: rgb(0, 0, 0);" class="">sub-milliseconds</span><span style="caret-color: rgb(0, 0, 0);" class=""> </span>.</div><div class="">The TCP window space is <span style="caret-color: rgb(0, 0, 0);" class="">bandwidth multiply RTT. For a 30 GBit/s network it is about 750KiB . I think that is trivial for a</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class="">datacenter server.</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class=""><br class=""></span></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class="">4.2.3.2 in </span></font><span style="caret-color: rgb(0, 0, 0);" class="">RFC 1122 states:</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class="">> </span><font class=""><span style="caret-color: rgb(0, 0, 0);" class="">in a stream of full-sized </span></font><span style="caret-color: rgb(0, 0, 0);" class="">segments there SHOULD be an ACK for at least every second </span><span style="caret-color: rgb(0, 0, 0);" class="">segment </span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class="">Even if the ACK every tenth </span><span style="caret-color: rgb(0, 0, 0);" class="">segment, the impact of delayed ACKs on TCP window is not significant ( at most</span></div><div class=""><span style="caret-color: rgb(0, 0, 0);" class=""> ten segments </span><span style="caret-color: rgb(0, 0, 0);" class="">not ACKed in TCP send window ).</span></div><div class=""><br class=""></div><div class="">Anyway, for datacenter usage the <font class=""><span style="caret-color: rgb(0, 0, 0);" class="">bandwidth is symmetric and the reverse path ( TX path of receiver ) is sufficient.</span></font></div><div class=""><font class=""><span style="caret-color: rgb(0, 0, 0);" class="">Servers can even ACK every </span></font><span style="caret-color: rgb(0, 0, 0);" class="">segment (no delaying ACK).</span></div><div class=""><br class=""></div><blockquote type="cite" class=""><div class=""><div class=""><br class="">I think the implementation should be exactly like it is.<br class=""><br class="">There is a software LRO in FreeBSD to coalesce the ACKs before they hit the network stack, so there are no real problems there.<br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">I'm OK with the current implementation.</div><div class=""><br class=""></div><div class="">I think upper layers (or application) have (business) information to indicate whether delaying ACKs should be employed.</div><div class="">After googling I found there's a draft [1].</div><div class=""><br class=""></div><div class="">[1] Sender Control of Delayed Acknowledgments in TCP: <a href="https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml" class="">https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml</a></div></div></div></div></blockquote><div><br class=""></div><div>Found the html / pdf / txt version of the draft RFC.</div><div><a href="https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/" class="">https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/</a></div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">--HPS<br class=""><br class=""><br class=""></div></div></blockquote></div><br class=""><div class=""><div style="caret-color: rgb(0, 0, 0);" class="">Best regards,</div><div style="caret-color: rgb(0, 0, 0);" class="">Zhenlei</div></div></div></div></blockquote></div><br class=""></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7EDD65B7-5FCD-42E1-A9E8-AA5139B0A81E>