From owner-freebsd-net Wed Jan 24 5:14:23 2001 Delivered-To: freebsd-net@freebsd.org Received: from mail2.netcologne.de (mail2.netcologne.de [194.8.194.103]) by hub.freebsd.org (Postfix) with ESMTP id 0DC6337B400 for ; Wed, 24 Jan 2001 05:14:04 -0800 (PST) Received: from husten.security.at12.de (dial-213-168-64-251.netcologne.de [213.168.64.251]) by mail2.netcologne.de (8.9.3/8.9.3) with ESMTP id OAA21070 for ; Wed, 24 Jan 2001 14:14:01 +0100 (MET) Received: from localhost (localhost.security.at12.de [127.0.0.1]) by husten.security.at12.de (8.11.1/8.11.1) with ESMTP id f0ODDsf75150 for ; Wed, 24 Jan 2001 14:13:54 +0100 (CET) (envelope-from pherman@frenchfries.net) Date: Wed, 24 Jan 2001 14:13:54 +0100 (CET) From: Paul Herman To: Subject: I have delayed ACK problems Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org A question for the TCP stack gurus... We were testing a FreeBSD 4.1 client with a Solaris 7 Oracle server. The FreeBSD client would take 10 times longer (60 sec) than a Linux client (~6 sec.) to complete an *identical* request. Configuration, client software and hardware problems have been ruled out. After comparing the tcpdump outputs, I found that the FreeBSD client was sending quite a few ACKs exactly 100ms later. Ah, delayed acks. sysctl -w net.inet.tcp.delayed_ack=0 "solves" the problem (or reducing the delack_timeout gives a corresponding performance win), and the FreeBSD client performs just as well as the Linux box, but I don't like this solution. Here's an example snippet from tcpdump where it trips up: 10:49:54.278268 192.168.111.16.1035 > 192.168.111.10.40438: P 4340:4420(80) ack 15362 win 17520 (DF) 10:49:54.279213 192.168.111.10.40438 > 192.168.111.16.1035: . 15362:16822(1460) ack 4420 win 8760 (DF) 10:49:54.279271 192.168.111.10.40438 > 192.168.111.16.1035: P 16822:17410(588) ack 4420 win 8760 (DF) 10:49:54.279387 192.168.111.10.40438 > 192.168.111.16.1035: . 17410:18870(1460) ack 4420 win 8760 (DF) 10:49:54.279443 192.168.111.10.40438 > 192.168.111.16.1035: P 18870:19433(563) ack 4420 win 8760 (DF) 10:49:54.279497 192.168.111.16.1035 > 192.168.111.10.40438: . ack 19433 win 17520 (DF) 10:49:54.279569 192.168.111.10.40438 > 192.168.111.16.1035: . 19433:20893(1460) ack 4420 win 8760 (DF) 10:49:54.279615 192.168.111.10.40438 > 192.168.111.16.1035: P 20893:21481(588) ack 4420 win 8760 (DF) 10:49:54.371025 192.168.111.16.1035 > 192.168.111.10.40438: . ack 21481 win 17520 (DF) 10:49:54.371223 192.168.111.10.40438 > 192.168.111.16.1035: P 21481:21752(271) ack 4420 win 8760 (DF) 192.168.111.10 Solaris Oracle Server 192.168.111.16 FreeBSD PHP client The four packets before get ACKed properly, but the next two don't. This happens a lot in this one connection, which slows it down by a factor of 10. I realize that in tcp_input() that the header prediction tries to decide if it is a "sender" or "receiver", and indeed this connection has a lot of small packets going back and forth before 192.168.111.10 starts a large data transfer, so I can understand that this might cause problems. We could live with just turning off delacks (it serves data more often than it receives), but I'd /really/ like to know what the problem is. My gut feeling is, it's not delack's problem, but a symtom of something else... This is a BIG performance hit in the default installation for FreeBSD, and I'm sure it isn't a rare occurance (Database connections, X11, etc. all do a little back & forth chat before a big file or pixmap is transfered...) What gets me is, at the end of the day is that FreeBSD can't, and Linux can (both implementing delayed ACKs) and management starts having their doubts. :( Environment: 100Mbit switched LAN, Intel fxp0 for FreeBSD, stock Sun E450 on the other side. Ideas? I'd appreciate any tips. Thanks, -Paul. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message