Date: Thu, 10 Nov 2022 16:07:08 +0800 From: Zhenlei Huang <zlei.huang@gmail.com> To: Hans Petter Selasky <hps@selasky.org> Cc: Michael Tuexen <michael.tuexen@lurchi.franken.de>, freebsd-net@freebsd.org Subject: Re: Too aggressive TCP ACKs Message-ID: <7EDD65B7-5FCD-42E1-A9E8-AA5139B0A81E@gmail.com> In-Reply-To: <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com> References: <75D35F36-7759-4168-ADBA-C2414F5B53BC@gmail.com> <712641B3-5196-40CC-9B64-04637F16F649@lurchi.franken.de> <62A0DD30-B3ED-48BE-9C01-146487599092@gmail.com> <0FED34A9-D093-442A-83B7-08C06D11F8B5@lurchi.franken.de> <330A9146-F7CC-4CAB-9003-2F90B872AC3E@gmail.com> <1ed66217-5463-fd4d-7e7a-58d9981bc44c@selasky.org> <5A501643-1E81-4A8C-8DDC-094371DC03D7@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_F4D23FB3-B3A1-4605-A3C4-410D8B3EDD26 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Nov 9, 2022, at 11:18 AM, Zhenlei Huang <zlei.huang@gmail.com> = wrote: >=20 >=20 >> On Oct 22, 2022, at 6:14 PM, Hans Petter Selasky <hps@selasky.org = <mailto:hps@selasky.org>> wrote: >>=20 >> Hi, >>=20 >> Some thoughts about this topic. >=20 > Sorry for late response. >=20 >>=20 >> Delaying ACKs means loss of performance when using Gigabit TCP = connections in data centers. There it is important to ACK the data as = quick as possible, to avoid running out of TCP window space. Thinking = about TCP connections at 30 GBit/s and above! >=20 > In data centers, the bandwidth is much more and the latency is = extremely low (compared to WAN), sub-milliseconds . > The TCP window space is bandwidth multiply RTT. For a 30 GBit/s = network it is about 750KiB . I think that is trivial for a > datacenter server. >=20 > 4.2.3.2 in RFC 1122 states: > > in a stream of full-sized segments there SHOULD be an ACK for at = least every second segment=20 > Even if the ACK every tenth segment, the impact of delayed ACKs on TCP = window is not significant ( at most > ten segments not ACKed in TCP send window ). >=20 > Anyway, for datacenter usage the bandwidth is symmetric and the = reverse path ( TX path of receiver ) is sufficient. > Servers can even ACK every segment (no delaying ACK). >=20 >>=20 >> I think the implementation should be exactly like it is. >>=20 >> There is a software LRO in FreeBSD to coalesce the ACKs before they = hit the network stack, so there are no real problems there. >=20 > I'm OK with the current implementation. >=20 > I think upper layers (or application) have (business) information to = indicate whether delaying ACKs should be employed. > After googling I found there's a draft [1]. >=20 > [1] Sender Control of Delayed Acknowledgments in TCP: = https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml = <https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs-01.xml= > Found the html / pdf / txt version of the draft RFC. https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/ >=20 >>=20 >> --HPS >>=20 >>=20 >=20 > Best regards, > Zhenlei --Apple-Mail=_F4D23FB3-B3A1-4605-A3C4-410D8B3EDD26 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On = Nov 9, 2022, at 11:18 AM, Zhenlei Huang <<a = href=3D"mailto:zlei.huang@gmail.com" = class=3D"">zlei.huang@gmail.com</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><meta = http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii" = class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: = space; line-break: after-white-space;" class=3D""><br class=3D""><div = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D"">On Oct = 22, 2022, at 6:14 PM, Hans Petter Selasky <<a = href=3D"mailto:hps@selasky.org" class=3D"">hps@selasky.org</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"">Hi,<br class=3D""><br class=3D"">Some thoughts about this = topic.<br class=3D""></div></div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">Sorry for late response.</div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = class=3D""><br class=3D"">Delaying ACKs means loss of performance when = using Gigabit TCP connections in data centers. There it is important to = ACK the data as quick as possible, to avoid running out of TCP window = space. Thinking about TCP connections at 30 GBit/s and above!<br = class=3D""></div></div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">In data centers, the bandwidth is much = more and the latency is extremely low (compared to WAN), <span = style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">sub-milliseconds</span><span style=3D"caret-color: rgb(0, 0, = 0);" class=3D""> </span>.</div><div class=3D"">The TCP window space = is <span style=3D"caret-color: rgb(0, 0, 0);" class=3D"">bandwidth = multiply RTT. For a 30 GBit/s network it is about 750KiB . I think that = is trivial for a</span></div><div class=3D""><span style=3D"caret-color: = rgb(0, 0, 0);" class=3D"">datacenter server.</span></div><div = class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" class=3D""><br = class=3D""></span></div><div class=3D""><font class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">4.2.3.2 = in </span></font><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">RFC 1122 states:</span></div><div class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">> </span><font = class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" class=3D"">in a = stream of full-sized </span></font><span style=3D"caret-color: = rgb(0, 0, 0);" class=3D"">segments there SHOULD be an ACK for at least = every second </span><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">segment </span></div><div class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Even if the ACK every = tenth </span><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">segment, the impact of delayed ACKs on TCP window is not = significant ( at most</span></div><div class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D""> ten = segments </span><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">not ACKed in TCP send window ).</span></div><div class=3D""><br= class=3D""></div><div class=3D"">Anyway, for datacenter usage = the <font class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">bandwidth is symmetric and the reverse path ( TX path of = receiver ) is sufficient.</span></font></div><div class=3D""><font = class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Servers = can even ACK every </span></font><span style=3D"caret-color: rgb(0, = 0, 0);" class=3D"">segment (no delaying ACK).</span></div><div = class=3D""><br class=3D""></div><blockquote type=3D"cite" class=3D""><div = class=3D""><div class=3D""><br class=3D"">I think the implementation = should be exactly like it is.<br class=3D""><br class=3D"">There is a = software LRO in FreeBSD to coalesce the ACKs before they hit the network = stack, so there are no real problems there.<br = class=3D""></div></div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">I'm OK with the current = implementation.</div><div class=3D""><br class=3D""></div><div = class=3D"">I think upper layers (or application) have (business) = information to indicate whether delaying ACKs should be = employed.</div><div class=3D"">After googling I found there's a draft = [1].</div><div class=3D""><br class=3D""></div><div = class=3D"">[1] Sender Control of Delayed Acknowledgments in TCP: <a = href=3D"https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-reqs= -01.xml" = class=3D"">https://www.ietf.org/archive/id/draft-gomez-tcpm-delack-suppr-r= eqs-01.xml</a></div></div></div></div></blockquote><div><br = class=3D""></div><div>Found the html / pdf / txt version of the draft = RFC.</div><div><a = href=3D"https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/" = class=3D"">https://datatracker.ietf.org/doc/draft-gomez-tcpm-ack-pull/</a>= </div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: = space; line-break: after-white-space;" class=3D""><div class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = class=3D""><br class=3D"">--HPS<br class=3D""><br class=3D""><br = class=3D""></div></div></blockquote></div><br class=3D""><div = class=3D""><div style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Best = regards,</div><div style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">Zhenlei</div></div></div></div></blockquote></div><br = class=3D""></body></html>= --Apple-Mail=_F4D23FB3-B3A1-4605-A3C4-410D8B3EDD26--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7EDD65B7-5FCD-42E1-A9E8-AA5139B0A81E>