From owner-freebsd-performance@FreeBSD.ORG Sun Oct 18 14:55:02 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4002E1065705 for ; Sun, 18 Oct 2009 14:55:02 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f202.google.com (mail-pz0-f202.google.com [209.85.222.202]) by mx1.freebsd.org (Postfix) with ESMTP id 13B4B8FC1B for ; Sun, 18 Oct 2009 14:55:01 +0000 (UTC) Received: by pzk40 with SMTP id 40so2733526pzk.7 for ; Sun, 18 Oct 2009 07:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=ejE04OtxJRQDj4Sf9DqaASury5yTuaezXbh/Gcqz5ok=; b=YoLxIHE7hFI7/9AZZ8SEluT+UJ+aMqCfgDg2TWDTKjP+gC+poE7NI9iDOBF6/8hooq w0Dcppq8k5ePXuv8Kd0iD5lwsKe/bDAw6+iPZyIhp/ktO6P5dxpu5fVrljh2iYwhwALs Tmkzg4vTRrbeG80JEBggPOpzv+eDj3KRrHgbs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=SLrhrqAm46iWSqi7eICbtyH0rccP7z+11iWJ0m8u0o4AcFWx1DolUSN0obcy6KLpY6 uH5qy43b+m+zFHFpwhwUa5KcQyAEIQPKqV3b8daHbYd1BoA5ZPSMn1XTXWu6Dx9hWgra +eYsVm3mJY5RzD4N2FMGjYrXWdAZmpr1e1xIQ= MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.115.99.11 with SMTP id b11mr3911654wam.17.1255877701511; Sun, 18 Oct 2009 07:55:01 -0700 (PDT) In-Reply-To: <78DB4AE8EF5F4A1EBD3992D7404B2725@china.huawei.com> References: <78DB4AE8EF5F4A1EBD3992D7404B2725@china.huawei.com> Date: Sun, 18 Oct 2009 22:55:01 +0800 X-Google-Sender-Auth: 1f5f7e24aeefc048 Message-ID: From: Adrian Chadd To: Hongtao Yin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2009 14:55:02 -0000 FYI, I installed netperf on my local p4-D test boxes that I use for other testing. 128 byte send/receive buffers on the client side: kristy# netperf -H 192.168.10.2 -p 22113 -l 10 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 (192.168.10.2) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 8192 128 128 10.00 426.17 1kbyte send/receive buffers: kristy# netperf -H 192.168.10.2 -p 22113 -l 10 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 (192.168.10.2) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 8192 1024 1024 10.00 903.39 8kbyte send/receive buffers: kristy# netperf -H 192.168.10.2 -p 22113 -l 10 TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2 (192.168.10.2) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 8192 8192 8192 10.00 913.71 Both boxes are 7.2-REL amd64 boxes on 3.4GHz Pentium-D CPUs using some onboard flavour of the intel e1000 NIC: device =3D '82573E Intel Corporation 82573E Gigabit Ethernet Controller (Copper)' They are connected via a Cisco 3750G L3 switch. In fact, the traffic is routed, rather than switched. My /etc/sysctl.conf: net.inet.icmp.icmplim=3D0 net.inet.icmp.icmplim_output=3D0 net.inet.tcp.msl=3D3000 net.inet.tcp.sendspace=3D8192 net.inet.tcp.recvspace=3D8192 kern.maxfilesperproc=3D65536 kern.maxfiles=3D262144 kern.ipc.maxsockets=3D32768 kern.ipc.somaxconn=3D1024 kern.ipc.nmbclusters=3D131072 net.inet.ip.fw.enable=3D0 kern.ipc.somaxconn=3D10240 2c, Adrian 2009/10/15 Hongtao Yin : > Hi, > > > > I compared TCP performance between FreeBSD and Linux by running test tool= s > Netperf and Iperf with Intel NIC. > > The kernels are full version and default values are used in the testing > except TCP Congestion Control algorithm set to Reno. > > >From the test results we can see Linux TCP performance in throughput is > better than FreeBSD. The worst case (send msg size 128) shows that FreeBS= D > throughput is only 43% of Linux's. > > > > I like to get some feedback if anyone did similar comparison test, or kno= ws > any issues with kernels or drivers. Thanks lot. > > > > FreeBSD and Linux Sysctl captures are attached for reference. > > > > Regards, > > Hongtao > > > > > > > > Test Environments: > > PC: Dell Precision T3400 (same 4 PCs) > > CPU: Intel Core 2 Duo CPU E4600@2.4Ghz > > FreeBSD: V7.1 (full version) (TCP CC: newReno) > > Linux: V2.6.31.1 (full version) (TCP CC: Reno) > > Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigab= it) > chip 82546EB (only one port used for each PC) > > Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108) > > Iperf: V2.0.4 > > Netperf: V2.4.4 > > > > Setup: > > > > > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0------= ---- > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| swit= ch | > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0------= ---- > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 ---------------------| | | |-----------------= --- > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= | | =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 | > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 =A0--------| |-----= --- =A0 =A0 =A0 =A0 =A0 =A0 | > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 | > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 =A0| =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 | =A0 =A0 =A0 =A0 =A0 =A0 | > > =A0 =A0 =A0 =A0-------------- =A0-------------- =A0-------------- =A0----= ---------- > > =A0 =A0 =A0 =A0| =A0 =A0 PC1 =A0 =A0| =A0| =A0 =A0PC2 =A0 =A0 | =A0| =A0 = =A0PC3 =A0 =A0 | =A0| =A0 =A0PC4 =A0 =A0 | > > =A0 =A0 =A0 =A0| =A0 FreeBSD =A0| =A0| =A0 FreeBSD =A0| =A0| =A0 Linux = =A0 =A0| =A0| =A0 Linux =A0 =A0| > > =A0 =A0 =A0 =A0|192.168.1.10| =A0|192.168.1.20| =A0|192.168.1.30| =A0|192= .168.1.40| > > =A0 =A0 =A0 =A0-------------- =A0-------------- =A0-------------- =A0----= ---------- > > > > > > > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > =A0 =A0 Netperf Test Results > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > > > TCP Throughput Test > > ------------------- > > > > PC2/4: #netserver -p 22113 > > PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10 > > > > =A0 =A0 =A0 =A0 =A0Recv =A0 =A0 =A0 =A0 =A0 =A0 Send =A0 =A0 =A0 =A0 =A0 = =A0Send =A0 =A0 =A0 =A0 =A0 Elapsed > Throughput > > =A0 =A0 =A0 =A0 =A0Socket =A0 =A0 =A0 =A0 =A0 Socket =A0 =A0 =A0 =A0 =A0M= essage =A0 =A0 =A0 =A0Time > 10^6 bits/sec > > =A0 =A0 =A0 =A0 =A0Size =A0 =A0 =A0 =A0 =A0 =A0 Size =A0 =A0 =A0 =A0 =A0 = =A0Size =A0 =A0 =A0 =A0 =A0 Sec. > > =A0 =A0 =A0 =A0 =A0bytes =A0 =A0 =A0 =A0 =A0 =A0bytes =A0 =A0 =A0 =A0 =A0= bytes > > FreeBSD: =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0 32768 = =A0 =A0 =A0 =A0 =A010.34 > 598.11 > > Linux: =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0 1638= 4 =A0 =A0 =A0 =A0 =A010.04 > 779.02 > > > > > > PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m > 64/128/256/512/1024/2048/4096 > > > > =A0 =A0 =A0 =A0 =A0Recv =A0 =A0 =A0 =A0 =A0 =A0 Send =A0 =A0 =A0 =A0 =A0 = =A0Send =A0 =A0 =A0 =A0 =A0 Elapsed > Throughput > > =A0 =A0 =A0 =A0 =A0Socket =A0 =A0 =A0 =A0 =A0 Socket =A0 =A0 =A0 =A0 =A0M= essage =A0 =A0 =A0 =A0Time > 10^6 bits/sec > > =A0 =A0 =A0 =A0 =A0Size =A0 =A0 =A0 =A0 =A0 =A0 Size =A0 =A0 =A0 =A0 =A0 = =A0Size =A0 =A0 =A0 =A0 =A0 Sec. > > =A0 =A0 =A0 =A0 =A0bytes =A0 =A0 =A0 =A0 =A0 =A0bytes =A0 =A0 =A0 =A0 =A0= bytes > > FreeBSD: =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0 64 =A0= =A0 =A0 =A0 =A0 =A0 =A010.19 > 417.10 > > =A0 =A0 =A0 =A0 =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0= 128 =A0 =A0 =A0 =A0 =A0 =A0 10.35 > 336.63 > > =A0 =A0 =A0 =A0 =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0= 256 =A0 =A0 =A0 =A0 =A0 =A0 10.36 > 576.99 > > =A0 =A0 =A0 =A0 =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0= 512 =A0 =A0 =A0 =A0 =A0 =A0 10.35 > 569.79 > > =A0 =A0 =A0 =A0 =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0= 1024 =A0 =A0 =A0 =A0 =A0 =A010.35 > 553.70 > > =A0 =A0 =A0 =A0 =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0= 2048 =A0 =A0 =A0 =A0 =A0 =A010.35 > 584.20 > > =A0 =A0 =A0 =A0 =A065536 =A0 =A0 =A0 =A0 =A0 =A032768 =A0 =A0 =A0 =A0 =A0= 4096 =A0 =A0 =A0 =A0 =A0 =A010.35 > 602.45 > > > > Linux: =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0 64 = =A0 =A0 =A0 =A0 =A0 =A0 =A010.03 > 778.21 > > =A0 =A0 =A0 =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0= 128 =A0 =A0 =A0 =A0 =A0 =A0 10.03 > 779.72 > > =A0 =A0 =A0 =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0= 256 =A0 =A0 =A0 =A0 =A0 =A0 10.04 > 780.16 > > =A0 =A0 =A0 =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0= 512 =A0 =A0 =A0 =A0 =A0 =A0 10.03 > 776.85 > > =A0 =A0 =A0 =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0= 1024 =A0 =A0 =A0 =A0 =A0 =A010.04 > 777.52 > > =A0 =A0 =A0 =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0= 2048 =A0 =A0 =A0 =A0 =A0 =A010.04 > 777.83 > > =A0 =A0 =A0 =A0 =A087380 =A0 =A0 =A0 =A0 =A0 =A016384 =A0 =A0 =A0 =A0 =A0= 4096 =A0 =A0 =A0 =A0 =A0 =A010.03 > 780.17 > > > > > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > > =A0 =A0 =A0Iperf Test Results > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > > > > Bandwidth Test > > -------------- > > > > PC2/4: #iperf -s > > PC1/3: #iperf -c 192.168.1.20 > > > > =A0 =A0 =A0 =A0 =A0 Interval =A0 =A0 =A0 =A0Transfer =A0 =A0 =A0 =A0Bandw= idth > > =A0 =A0 =A0 =A0 =A0 sec =A0 =A0 =A0 =A0 =A0 =A0 MBytes =A0 =A0 =A0 =A0 = =A0Mbits/sec > > FreeBSD: =A0 0.0-10.3 =A0 =A0 =A0 =A0740 =A0 =A0 =A0 =A0 =A0 =A0 600 > > Linux: =A0 =A0 0.0-10.0 =A0 =A0 =A0 =A0972 =A0 =A0 =A0 =A0 =A0 =A0 815 > > > > > > PC1/3: #iperf -c 192.168.1.20 -d > > > > =A0 =A0 =A0 =A0 =A0 Interval =A0 =A0 =A0 =A0Transfer =A0 =A0 =A0 =A0Bandw= idth > > =A0 =A0 =A0 =A0 =A0 sec =A0 =A0 =A0 =A0 =A0 =A0 MBytes =A0 =A0 =A0 =A0 = =A0Mbits/sec > > FreeBSD: =A0 0.0-10.0 =A0 =A0 =A0 =A0402 =A0 =A0 =A0 =A0 =A0 =A0 337 > > =A0 =A0 =A0 =A0 =A0 0.0-10.0 =A0 =A0 =A0 =A0404 =A0 =A0 =A0 =A0 =A0 =A0 3= 38 > > > > Linux: =A0 =A0 0.0-10.0 =A0 =A0 =A0 =A0926 =A0 =A0 =A0 =A0 =A0 =A0 776 > > =A0 =A0 =A0 =A0 =A0 0.0-10.0 =A0 =A0 =A0 =A044.1 =A0 =A0 =A0 =A0 =A0 =A03= 6.9 > > > > > > Parallel Test > > ------------- > > > > PC2/4: #iperf -s > > PC1/3: #iperf -c 192.168.1.20 -P 2 > > > > =A0 =A0 =A0 =A0 =A0 Interval =A0 =A0 =A0 =A0Transfer =A0 =A0 =A0 =A0Bandw= idth > > =A0 =A0 =A0 =A0 =A0 sec =A0 =A0 =A0 =A0 =A0 =A0 MBytes =A0 =A0 =A0 =A0 = =A0Mbits/sec > > FreeBSD: =A0 0.0-10.3 =A0 =A0 =A0 =A0370 =A0 =A0 =A0 =A0 =A0 =A0 300 > > =A0 =A0 =A0 =A0 =A0 0.0-10.3 =A0 =A0 =A0 =A0370 =A0 =A0 =A0 =A0 =A0 =A0 3= 00 > > =A0 =A0 =A0SUM: 0.0-10.3 =A0 =A0 =A0 =A0739 =A0 =A0 =A0 =A0 =A0 =A0 600 > > > > Linux: =A0 =A0 0.0-10.0 =A0 =A0 =A0 =A0479 =A0 =A0 =A0 =A0 =A0 =A0 402 > > =A0 =A0 =A0 =A0 =A0 0.0-10.0 =A0 =A0 =A0 =A0473 =A0 =A0 =A0 =A0 =A0 =A0 3= 96 > > =A0 =A0 =A0SUM: 0.0-10.0 =A0 =A0 =A0 =A0952 =A0 =A0 =A0 =A0 =A0 =A0 797 > > > > > > > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd= .org" >