Date: Fri, 11 Nov 2022 13:02:04 +0800 From: Zhenlei Huang <zlei.huang@gmail.com> To: tuexen@freebsd.org Cc: Hans Petter Selasky <hps@selasky.org>, freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs Message-ID: <3911101F-3010-45F4-97C1-9E2D25149676@gmail.com> In-Reply-To: <6FE15D58-D19F-4B7F-8548-40B9573A7F7F@freebsd.org> References: <75D35F36-7759-4168-ADBA-C2414F5B53BC@gmail.com> <712641B3-5196-40CC-9B64-04637F16F649@lurchi.franken.de> <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> <0FED34A9-D093-442A-83B7-08C06D11F8B5@lurchi.franken.de> <330A9146-F7CC-4CAB-9003-2F90B872AC3E@gmail.com> <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org> <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com> <7EDD65B7-5FCD-42E1-A9E8-AA5139B0A81E@gmail.com> <6FE15D58-D19F-4B7F-8548-40B9573A7F7F@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_14863C5B-763C-4E1F-9ED6-5D1E38BD0372 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Nov 10, 2022, at 5:28 PM, tuexen@freebsd.org wrote: >=20 >> On 10. Nov 2022, at 08:07, Zhenlei Huang <zlei.huang@gmail.com> = wrote: >>=20 >>> On Nov 9, 2022, at 11:18 AM, Zhenlei Huang <zlei.huang@gmail.com> = wrote: >>>=20 >>>=20 >>>> On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <hps@selasky.org> = wrote: >>>>=20 >>>> Hi, >>>>=20 >>>> Some thoughts about this topic. >>>=20 >>> Sorry for late response. >>>=20 >>>>=20 >>>> Delaying ACKs means loss of performance when using Gigabit TCP = connections in data centers. There it is important to ACK the data as = quick as possible, to avoid running out of TCP window space. Thinking = about TCP connections at 30 GBit/s and above! >>>=20 >>> In data centers, the bandwidth is much more and the latency is = extremely low (compared to WAN), sub-milliseconds . >>> The TCP window space is bandwidth multiply RTT. For a 30 GBit/s = network it is about 750KiB . I think that is trivial for a >>> datacenter server. >>>=20 >>> 4.2.3.2 in RFC 1122 states: >>>> in a stream of full-sized segments there SHOULD be an ACK for at = least every second segment=20 >>> Even if the ACK every tenth segment, the impact of delayed ACKs on = TCP window is not significant ( at most >>> ten segments not ACKed in TCP send window ). >>>=20 >>> Anyway, for datacenter usage the bandwidth is symmetric and the = reverse path ( TX path of receiver ) is sufficient. >>> Servers can even ACK every segment (no delaying ACK). >>>=20 >>>>=20 >>>> I think the implementation should be exactly like it is. >>>>=20 >>>> There is a software LRO in FreeBSD to coalesce the ACKs before they = hit the network stack, so there are no real problems there. >>>=20 >>> I'm OK with the current implementation. >>>=20 >>> I think upper layers (or application) have (business) information to = indicate whether delaying ACKs should be employed. >>> After googling I found there's a draft [1]. >>>=20 >>> [1] Sender Control of Delayed Acknowledgments in TCP: = https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml >>=20 >>=20 >> Found the html / pdf / txt version of the draft RFC. >> https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/ > Can you specify the problem you are facing or trying to solve? For me, no problems currently. > I think upper layers (or application) have (business) information to = indicate whether delaying ACKs should be employed. That is from my experience in software developing. In typical layered = architectures, Callers have more information than callee. To be flexible generally callee should not presume too much but instead = should have options to support (what callers intend). It is a little off-topic. For "Too aggressive TCP ACKs" if delaying ACK = every third or more full segments, it has known issue "Stretch ACK violation". See section 2.13 from RFC 2525. >=20 > Best regards > Michael >>=20 >>>=20 >>>>=20 >>>> --HPS >>>>=20 >>>>=20 >>>=20 >>> Best regards, >>> Zhenlei Best regards, Zhenlei --Apple-Mail=_14863C5B-763C-4E1F-9ED6-5D1E38BD0372 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D""><div><br class=3D""></div></div><div><blockquote type=3D"cite" = class=3D""><div class=3D"">On Nov 10, 2022, at 5:28 PM, <a = href=3D"mailto:tuexen@freebsd.org" class=3D"">tuexen@freebsd.org</a> = wrote:</div><br class=3D"Apple-interchange-newline"><div = class=3D""><blockquote type=3D"cite" style=3D"font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; orphans: auto; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D"">On = 10. Nov 2022, at 08:07, Zhenlei Huang <<a = href=3D"mailto:zlei.huang@gmail.com" = class=3D"">zlei.huang@gmail.com</a>> wrote:<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D"">On Nov 9, 2022, at 11:18 = AM, Zhenlei Huang <<a href=3D"mailto:zlei.huang@gmail.com" = class=3D"">zlei.huang@gmail.com</a>> wrote:<br class=3D""><br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D"">On Oct = 22, 2022, at 6:14 PM, Hans Petter Selasky <<a = href=3D"mailto:hps@selasky.org" class=3D"">hps@selasky.org</a>> = wrote:<br class=3D""><br class=3D"">Hi,<br class=3D""><br class=3D"">Some = thoughts about this topic.<br class=3D""></blockquote><br class=3D"">Sorry= for late response.<br class=3D""><br class=3D""><blockquote type=3D"cite"= class=3D""><br class=3D"">Delaying ACKs means loss of performance when = using Gigabit TCP connections in data centers. There it is important to = ACK the data as quick as possible, to avoid running out of TCP window = space. Thinking about TCP connections at 30 GBit/s and above!<br = class=3D""></blockquote><br class=3D"">In data centers, the bandwidth is = much more and the latency is extremely low (compared to WAN), = sub-milliseconds .<br class=3D"">The TCP window space is bandwidth = multiply RTT. For a 30 GBit/s network it is about 750KiB . I think that = is trivial for a<br class=3D"">datacenter server.<br class=3D""><br = class=3D"">4.2.3.2 in RFC 1122 states:<br class=3D""><blockquote = type=3D"cite" class=3D"">in a stream of full-sized segments there SHOULD = be an ACK for at least every second segment<span = class=3D"Apple-converted-space"> </span><br = class=3D""></blockquote>Even if the ACK every tenth segment, the impact = of delayed ACKs on TCP window is not significant ( at most<br = class=3D"">ten segments not ACKed in TCP send window ).<br class=3D""><br = class=3D"">Anyway, for datacenter usage the bandwidth is symmetric and = the reverse path ( TX path of receiver ) is sufficient.<br = class=3D"">Servers can even ACK every segment (no delaying ACK).<br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D""><br = class=3D"">I think the implementation should be exactly like it is.<br = class=3D""><br class=3D"">There is a software LRO in FreeBSD to coalesce = the ACKs before they hit the network stack, so there are no real = problems there.<br class=3D""></blockquote><br class=3D"">I'm OK with = the current implementation.<br class=3D""><br class=3D"">I think upper = layers (or application) have (business) information to indicate whether = delaying ACKs should be employed.<br class=3D"">After googling I found = there's a draft [1].<br class=3D""><br class=3D"">[1] Sender Control of = Delayed Acknowledgments in TCP: <a = href=3D"https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs= -01.xml" = class=3D"">https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-r= eqs-01.xml</a><br class=3D""></blockquote><br class=3D""><br = class=3D"">Found the html / pdf / txt version of the draft RFC.<br = class=3D""><a = href=3D"https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/" = class=3D"">https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/</a>= <br class=3D""></blockquote><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">Can you specify the problem you are facing or trying to = solve?</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 12px; font-style: normal; font-variant-caps: = normal; font-weight: normal; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D""></div></blockquote><div><br class=3D""></div><div>For = me, no problems currently.</div><div><br class=3D""></div><div>> I = think upper layers (or application) have (business) information to = indicate whether delaying ACKs should be employed.</div><div><br = class=3D""></div><div>That is from my experience in software developing. = In typical layered architectures, Callers have more information than = callee.</div><div>To be flexible generally callee should = not presume too much but instead should have options to support = (what callers intend).</div><div><br class=3D""></div><div>It is a = little off-topic. For "Too aggressive TCP ACKs" if delaying ACK every = third or more full segments, it has known issue</div><div>"Stretch ACK = violation". See section 2.13 from RFC 2525.</div><div><br = class=3D""></div><div><br class=3D""></div><blockquote type=3D"cite" = class=3D""><div class=3D""><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">Best regards</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">Michael</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><blockquote type=3D"cite" = style=3D"font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><br class=3D""><blockquote = type=3D"cite" class=3D""><br class=3D""><blockquote type=3D"cite" = class=3D""><br class=3D"">--HPS<br class=3D""><br class=3D""><br = class=3D""></blockquote><br class=3D"">Best regards,<br = class=3D"">Zhenlei</blockquote></blockquote></div></blockquote><br = class=3D""></div><div><div style=3D"caret-color: rgb(0, 0, 0); color: = rgb(0, 0, 0);">Best regards,</div><div style=3D"caret-color: rgb(0, 0, = 0); color: rgb(0, 0, 0);">Zhenlei</div></div><br class=3D""></body></html>= --Apple-Mail=_14863C5B-763C-4E1F-9ED6-5D1E38BD0372--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3911101F-3010-45F4-97C1-9E2D25149676>