From owner-freebsd-hackers Fri Nov 30 14:59: 2 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from cu518.adelaide.adsl.on.net (cu518.adelaide.adsl.on.net [150.101.236.10]) by hub.freebsd.org (Postfix) with ESMTP id CDE9837B405 for ; Fri, 30 Nov 2001 14:58:57 -0800 (PST) Received: from ns.aus.com (laptop.ns.aus.com [10.0.2.6]) by cu518.adelaide.adsl.on.net (8.11.0/8.11.0) with ESMTP id fB116M703803; Sat, 1 Dec 2001 11:36:22 +1030 Message-ID: <3C081867.9090900@ns.aus.com> Date: Sat, 01 Dec 2001 10:08:15 +1030 From: Richard Sharpe Reply-To: rsharpe@ns.aus.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4) Gecko/20010917 X-Accept-Language: en-us MIME-Version: 1.0 To: Alfred Perlstein Cc: rsharpe@ns.aus.com, Matthew Dillon , Alexander Haderer , freebsd-hackers@FreeBSD.ORG Subject: Re: FreeBSD performing worse than Linux? References: <20011128153817.T61580@monorchid.lemis.com> <15364.38174.938500.946169@caddis.yogotech.com> <20011128104629.A43642@walton.maths.tcd.ie> <5.1.0.14.1.20011130181236.00a80160@postamt1.charite.de> <200111302047.fAUKlT811090@apollo.backplane.com> <3C07FCFF.4070008@ns.aus.com> <20011130150843.L46769@elvis.mu.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, I think that there are two different problems here. My situation involves a LAN (actually, a crossover cable). I have captured a trace of a 1 client run between the Linux driver and the FreeBSD test system as well as between the Linux driver and the same test system running Linux. I am noticing some interesting things. Linux uses the timestamp option in all the TCP segments I have looked at, so it is sending 12 more bytes per segment that FreeBSD. However, more interesting is that for small messages (less that 1460), FreeBSD does not seem to delay sending ACKs, so we get the following pattern: FREEBSD Driver -> Test system: 94 byte IP DG with simulated command Test System -> Driver: Ack after 83uS Test System -> Driver: Psh Ack after 29uS with 79 total bytes in IP DG LINUX Driver -> Test system: 106 byte IP DG with simulated command Test System -> Driver: Psh Ack after 89uS with 91 total bytes in IP DG So, as you can see, Linux seems to shave some time off each transaction by avoiding sending extra ACKs. Also, what I am seeing is that neither FreeBSD nor Linux is doing ACK coalescing (if that is possible). While I understand that coalescing ACKs will mess up RTT calculations and SRTT a bit, it would serve to reduce the time taken until responses come back. What I am seeing for large transmits is the following: FreeBSD (Test) Linux (Driver) <---- Request, 1500 bytes including request and some data <---- More segments from the request Some ACKs -----> About one every two segments <---- Last data segment, usually less that 1500 Lots of ACKs ----> one per segment Usually with large window (ie 16020 when the max window seems to be 16384). Response ----> Less than 1500 Now, I have seen something like 10+ ACKS after the driver has finished sending. They appear to be one per sent segment. Then the FreeBSD system sends its response. The optimal would be for the FreeBSD system to delay the ack until it has data to send, which it probably already has. What I see with the Linux trace is that Linux coalesces ACKs. However, the most I have seen it coalesce is two segments. HTH. -- Richard Sharpe, rsharpe@ns.aus.com, LPIC-1 www.samba.org, www.ethereal.com, SAMS Teach Yourself Samba in 24 Hours, Special Edition, Using Samba To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message