From owner-freebsd-stable@FreeBSD.ORG Thu Feb 25 09:03:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BAC4106566B for ; Thu, 25 Feb 2010 09:03:11 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (host.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0AB2F8FC17 for ; Thu, 25 Feb 2010 09:03:10 +0000 (UTC) Received: from titan.flintsbach.schmalzbauer.de (titan.flintsbach.schmalzbauer.de [172.21.1.150]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id o1P9321X085984 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Feb 2010 10:03:02 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <4B863CC5.80906@omnilan.de> Date: Thu, 25 Feb 2010 10:03:01 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Thunderbird 2.0.0.23 (X11/20090906) MIME-Version: 1.0 To: Nikos Ntarmos References: <20100218192320.B34251CC09@ptavv.es.net> <4B7DB408.9010706@omnilan.de> <20100219095539.GD14413@asgard.cs.uoi.gr> <20100224153703.GA11729@asgard.cs.uoi.gr> In-Reply-To: <20100224153703.GA11729@asgard.cs.uoi.gr> X-Enigmail-Version: 0.95.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigC85738FB85F4303CBCFC03A6" X-Mailman-Approved-At: Thu, 25 Feb 2010 12:21:11 +0000 Cc: Subject: Re: RELENG_8 ignoring TCP window size? [Was: Re: Help for TCP understanding wanted, ACK-MSS-Window [Was: Re: best practice to watch TCP parms of established sockets]] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 09:03:11 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigC85738FB85F4303CBCFC03A6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Nikos Ntarmos schrieb am 24.02.2010 16:37 (localtime): > On Fri, Feb 19, 2010 at 11:55:39AM +0200, Nikos Ntarmos wrote: >> On Thu, Feb 18, 2010 at 10:41:28PM +0100, Harald Schmalzbauer wrote: >>> Kevin Oberman schrieb am 18.02.2010 20:23 (localtime): >>> ... >>>> window allows for many packets to be in flight and with a 3 Gbps flo= w, >>>> that is a LOT of data. While an ACK is sent every two packets of >>>> received data, the transmitting side does not wait for the ACKs. It >>> ... >>>> That is a VERY simple and incomplete explanation of what is happenin= g >>>> with the window, but most of that is irrelevant in local transfers w= ith >>> Thanks a lot, then I understood it at least half correct ;) My >>> missunderstanding was that I thought the receiver would reduce >>> ACKs... Now I know more :) >>> But unfortunately that makes it more mysterious where the throughput >>> problem lies... >>> >>> Thanks to everyone so far, >> Hi there. >> >> This is a long shot but have you tried disabling checksum and >> segmentation offloading? I've found that they cause trouble with some >> NICs. FYI on FreeBSD the first is done through 'ifconfig -rxcsum' whil= e >> the latter through 'sysctl net.inet.tcp.tso=3D0'. If you try this out,= >> remember to disable these features on both communicating boxes for the= >> period of the test, just to be sure that it's not the other box causin= g >> these issues. You mentioned samba so if one of them is a win32 box, yo= u >> can access these settings through the hardware options of your NIC. >=20 > Hi again. >=20 > Just a friendly nudge. :) Did you find the root of these issues yet? I don't have solid conclusions. But I ruled out some things. First, the em driver has no problems in my setup. Neither disabling=20 rx/txcsum nor disabling TSO made any differenz in trhoughput, though no=20 direct recognizable load behaviour on my 2x3GHz machine. Mybe checsum=20 offload is beneficial on slower machines. One major problem was that one of the two FreeBSD Boxes was on a VMware=20 which slowed things down a bit, although the FreeBSD System showed=20 plenty free resources. Disabling rfc1323 defnetily increases the throughput on gigabit=20 ethernet. I can rsync between two (native) FreeBSD machines with 72-92=20 MB/s averaging at 80+MB/s. What I couldn't investigeate yet is why I always get 10% more throughput = when one side is windwos, no matter which direction, no matter what=20 application (cifs, rsync, ftp). Another big point on my todo list is to find out why tcp.inflight brakes = my webserver downloads really often to less than a quarter of the=20 available bandwith (client bw, server bw is 100mb). I saw many 10mb/s E3 = pipes with 15ms delay, but limited transfer rates to 200kb/s. Disabling=20 tcp.inflight opens that brake. Since I could only capture "bad IP length" packets with checsum=20 offloading enabled on the em, I guess it does also IP checksum=20 offloading, not only TCP. I'll have to set up a port mirroring test bed=20 to get the line packets. I hope I'll find the rfc1323 slowdown and the=20 difference between FreeBSD clients and Windows clients. But at the=20 moment I don't have spare equipment. Greets, -Harry --------------enigC85738FB85F4303CBCFC03A6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAkuGPMYACgkQLDqVQ9VXb8jc1QCfbEcNCFswIXfTVrPANLYZJwaZ z+QAnieRBqX1mkNh7e1k193w0R7YMbPh =xfkZ -----END PGP SIGNATURE----- --------------enigC85738FB85F4303CBCFC03A6--