Date: Wed, 21 Apr 1999 21:05:15 -0600 From: Alex Rousskov <rousskov@ircache.net> To: freebsd-net@FreeBSD.ORG Subject: _Some_ acks delayed for 200 msec? Message-ID: <Pine.SGI.4.10.9904211955350.6184-100000@Meta-Bug>
next in thread | raw e-mail | index | archive | help
Hi there, I am stuck with what looks like a TCP or kernel timing thing. Setup: A single open (persistent) TCP connection between two PCs running FreeBSD 2.2.8-RELEASE. Switched 100Mbit FDX network. HTTP-like test: a small client request, a larger response from a server, client request, server response, etc. All on the same connection. All message sizes are almost constant for this test. Non-blocking sockets write/read data as soon as possible. No external activity. No packet loss. With various kernel and test configurations, I get exactly about 5 requests per second (about ~6.2Mbps) on a persistent connection. With non-persistent connections, I get at least 65Mbps throughput. Finally, here is a portion of a tcpdump@client that illustrates the problem. Only the client side is shown. Watch for time "jumps" in the first column. 20:34:32.433366 . ack 464131 20:34:32.433614 . ack 467011 20:34:32.630026 . ack 468451 <--- jump 20:34:32.630379 . ack 469613 20:34:32.630527 P 8100:8280(180) ack <--- client request N 20:34:32.631236 . ack 471661 20:34:32.631508 . ack 474541 20:34:32.631923 . ack 478861 20:34:32.632278 . ack 480023 20:34:32.632442 P 8280:8460(180) ack <--- client request N+1 20:34:32.633333 . ack 482071 20:34:32.633922 . ack 487831 20:34:32.830024 . ack 489271 <--- jump 20:34:32.830375 . ack 490433 20:34:32.830531 P 8460:8640(180) ack <--- client request N+2 20:34:32.831235 . ack 492481 20:34:32.831398 . ack 493921 20:34:32.831636 . ack 496801 20:34:32.831881 . ack 499681 20:34:32.832233 . ack 500843 20:34:32.832381 P 8640:8820(180) ack <--- client request N+3 20:34:32.833083 . ack 502891 20:34:32.833247 . ack 504331 20:34:32.833484 . ack 507211 20:34:32.833730 . ack 510091 20:34:32.834082 . ack 511253 20:34:32.834232 P 8820:9000(180) ack <--- ... 20:34:32.834938 . ack 513301 20:34:32.835330 . ack 517621 20:34:32.835580 . ack 520501 20:34:32.835934 . ack 521663 20:34:32.836085 P 9000:9180(180) ack 20:34:32.836787 . ack 523711 20:34:32.836950 . ack 525151 20:34:32.837189 . ack 528031 20:34:32.837433 . ack 530911 20:34:32.837787 . ack 532073 20:34:32.837936 P 9180:9360(180) ack 20:34:32.838659 . ack 534121 20:34:32.838930 . ack 537001 20:34:32.839176 . ack 539881 20:34:33.030027 . ack 541321 <---- jump 20:34:33.030382 . ack 542483 .... Note that jumps occur every 200 msec. The number of packets (and replies/requests in between) is irrelevant. Every 200 msec an "unlucky" ack will be delayed until the end of that 200 msec "slice". The dataflow from a server is stable as illustrated by the dump below. In other words, there is always something to ack (8080=server)... 20:34:32.633665 .8080 > .1084: . 484951:486391(1440) ack 8460 win 17280 20:34:32.633787 .8080 > .1084: . 486391:487831(1440) ack 8460 win 17280 20:34:32.633909 .8080 > .1084: . 487831:489271(1440) ack 8460 win 17280 20:34:32.633922 .1084 > .8080: . ack 487831 win 17280 <delay is here> 20:34:32.830024 .1084 > .8080: . ack 489271 win 17280 20:34:32.830338 .8080 > .1084: P 489271:490433(1162) ack 8460 win 17280 Now, if that is the infamous delayed ack problem, then - why only *some* acks are delayed? - why Nagle-like optimization is used on a switched (no routing) network? - why disabling Nagle (TCP_NODELAY) does not help? - why disabling delayed acks in the kernel (options TCP_ACK_HACK) does not help? If it is not a delayed ack or Nagle, then what is it _and_ how can I avoid it? Thanks, Alex. P.S. Web Polygraph (http://polygraph.ircache.net/) is the application doing the tests, but ( hopefully :) that is irrelevant. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.SGI.4.10.9904211955350.6184-100000>