From owner-freebsd-performance@FreeBSD.ORG Mon Oct 19 01:36:37 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86F64106566B for ; Mon, 19 Oct 2009 01:36:37 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f202.google.com (mail-pz0-f202.google.com [209.85.222.202]) by mx1.freebsd.org (Postfix) with ESMTP id 5855E8FC14 for ; Mon, 19 Oct 2009 01:36:37 +0000 (UTC) Received: by pzk40 with SMTP id 40so2933330pzk.7 for ; Sun, 18 Oct 2009 18:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=lm0lzEZp5H+M+bXqnnsbCEYa/t32SL8xGL0gf7hI9dM=; b=YEeDsA7VWu0BzwZVXwX1VpZfmUGyFJ6DX/wvu5OHu7oizxzsOaycVvmyN/eQhqKZ7m CPjG1FaGgDcsdFJdEHicFdGJ9o3LvnXpoVJEDq7Sa7kL5tM6eyJ+FvWFygkB+hdng8rs qGeaFl/wdA7Sygp2g80UtVbaN0C6oAGD7gvwE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=W2MiE9fpXXaKK3vVTk9pSL6r9ShJpUESiOJz912wQHjRUaH3WwGD0PHQM0lK1HX0G5 OmWFdhoR6+1oWC7qDy1fR4/b9vCxNUIzhN4Ik3SGHZh+gsIKzx0/LtJ4Gc/ETDKXuw1Y CnLht2Q/bg9j3L8kQsRFR3OsQA2npk8X9DPzY= MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.115.113.9 with SMTP id q9mr4858618wam.224.1255916196870; Sun, 18 Oct 2009 18:36:36 -0700 (PDT) In-Reply-To: References: <78DB4AE8EF5F4A1EBD3992D7404B2725@china.huawei.com> Date: Mon, 19 Oct 2009 09:36:36 +0800 X-Google-Sender-Auth: 38f9c9dd18bd5be1 Message-ID: From: Adrian Chadd To: Brent Jones Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Hongtao Yin , freebsd-performance@freebsd.org Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2009 01:36:37 -0000 uhm: kristy# netperf -H 192.168.10.2 -p 22113 -l 10 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 (192.168.10.2) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 8192 65536 65536 10.00 862.48 1 megabyte socket buffers threw an error. I'll see why later. Now, as for why 64k socket buffers gave a slower result than 8k socket buffers... ah. If I change the sending end to use 64k socket buffers: TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 (192.168.10.2) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 65536 65536 65536 10.00 916.23 Adrian 2009/10/19 Brent Jones : > On Sun, Oct 18, 2009 at 7:55 AM, Adrian Chadd wrote: >> FYI, I installed netperf on my local p4-D test boxes that I use for >> other testing. >> >> 128 byte send/receive buffers on the client side: >> >> kristy# netperf -H 192.168.10.2 -p 22113 -l 10 >> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 >> (192.168.10.2) port 0 AF_INET >> Recv =A0 Send =A0 =A0Send >> Socket Socket =A0Message =A0Elapsed >> Size =A0 Size =A0 =A0Size =A0 =A0 Time =A0 =A0 Throughput >> bytes =A0bytes =A0 bytes =A0 =A0secs. =A0 =A010^6bits/sec >> >> =A08192 =A0 =A0128 =A0 =A0128 =A0 =A010.00 =A0 =A0 426.17 >> >> 1kbyte send/receive buffers: >> >> kristy# netperf -H 192.168.10.2 -p 22113 -l 10 >> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 >> (192.168.10.2) port 0 AF_INET >> Recv =A0 Send =A0 =A0Send >> Socket Socket =A0Message =A0Elapsed >> Size =A0 Size =A0 =A0Size =A0 =A0 Time =A0 =A0 Throughput >> bytes =A0bytes =A0 bytes =A0 =A0secs. =A0 =A010^6bits/sec >> >> =A08192 =A0 1024 =A0 1024 =A0 =A010.00 =A0 =A0 903.39 >> >> 8kbyte send/receive buffers: >> >> kristy# netperf -H 192.168.10.2 -p 22113 -l 10 >> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 >> (192.168.10.2) port 0 AF_INET >> Recv =A0 Send =A0 =A0Send >> Socket Socket =A0Message =A0Elapsed >> Size =A0 Size =A0 =A0Size =A0 =A0 Time =A0 =A0 Throughput >> bytes =A0bytes =A0 bytes =A0 =A0secs. =A0 =A010^6bits/sec >> >> =A08192 =A0 8192 =A0 8192 =A0 =A010.00 =A0 =A0 913.71 >> >> Both boxes are 7.2-REL amd64 boxes on 3.4GHz Pentium-D CPUs using some >> onboard flavour of the intel e1000 NIC: >> >> =A0 =A0device =A0 =A0 =3D '82573E Intel Corporation 82573E Gigabit Ether= net >> Controller (Copper)' >> >> They are connected via a Cisco 3750G L3 switch. In fact, the traffic >> is routed, rather than switched. >> >> My /etc/sysctl.conf: >> >> net.inet.icmp.icmplim=3D0 >> net.inet.icmp.icmplim_output=3D0 >> net.inet.tcp.msl=3D3000 >> net.inet.tcp.sendspace=3D8192 >> net.inet.tcp.recvspace=3D8192 >> kern.maxfilesperproc=3D65536 >> kern.maxfiles=3D262144 >> kern.ipc.maxsockets=3D32768 >> kern.ipc.somaxconn=3D1024 >> kern.ipc.nmbclusters=3D131072 >> net.inet.ip.fw.enable=3D0 >> kern.ipc.somaxconn=3D10240 >> >> 2c, >> >> >> Adrian >> >> 2009/10/15 Hongtao Yin : >>> Hi, >>> >>> > > Can you try with 64K and up tp 1MB buffers? > > I see ~1Gbit speeds with my FreeBSD boxes using Broadcom NIC's and > cheap Netgear switches. > I'm not sure how the original tester got such poor numbers, when my > setup is relatively low end, and sustaining Gbit speeds is no major > feat. > > > > -- > Brent Jones > brent@servuhome.net >