Date: Thu, 10 Aug 2006 15:57:03 +1000 From: Michael Vince <mv@thebeastie.org> To: Dima Roshin <roshind@mail.ru> Cc: freebsd-net@freebsd.org Subject: Re: Gigabit ethernet questions? Message-ID: <44DACAAF.5020306@thebeastie.org> In-Reply-To: <E1GAmuq-000CR8-00.roshind-mail-ru@f60.mail.ru> References: <E1GAmuq-000CR8-00.roshind-mail-ru@f60.mail.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
JICUDN, I have been using nc,dd and systat to check TCP performance on my servers, the good thing about it is it requires little setup and gives results fast. For example on host A start the nc server. nc -4kl 3000 > /dev/null Then start another one on hostb sending data via nc from /dev/zero cat /dev/zero | dd bs=1m | nc hosta 3000 ^C0+226041 records in 0+226040 records out 14813757440 bytes transferred in 246.184967 secs (60173282 bytes/sec) You can take out the dd and just use systat -if depending on what you like. Mike Dima Roshin wrote: >Thanks Jon, I did it on both sides, thats much better now: > >gate1# sysctl kern.polling.idle_poll=1 >kern.polling.idle_poll: 0 -> 1 >gate1# sysctl net.inet.ip.fastforwarding >net.inet.ip.fastforwarding: 0 >gate1# sysctl net.inet.ip.fastforwarding=1 >net.inet.ip.fastforwarding: 0 -> 1 >gate1# iperf -c 192.168.0.2 -N -w 196000 >------------------------------------------------------------ >Client connecting to 192.168.0.2, TCP port 5001 >TCP window size: 192 KByte (WARNING: requested 191 KByte) >------------------------------------------------------------ >[ 3] local 192.168.0.1 port 63941 connected with 192.168.0.2 port 5001 >[ 3] 0.0-10.0 sec 762 MBytes 639 Mbits/sec > >But there is still some bottleneck, and I can't understand where. > > > >-----Original Message----- >From: Jon Otterholm <jon.otterholm@ide.resurscentrum.se> >To: Dima Roshin <roshind@mail.ru> >Date: Wed, 09 Aug 2006 13:54:18 +0200 >Subject: Re: Gigabit ethernet questions? > > > >>Dima Roshin wrote: >> >> >>> Greeting colleagues. I've got two DL-360(pciX bus) servers, with BCM5704 NetXtreme Dual Gigabit Adapters(bge). The Uname is 6.1-RELEASE-p3. The bge interfaces of the both servers are connected with each other with a cat6 patchcord. >>> Here are my settings: >>>kernel config: >>>options DEVICE_POLLING >>>options HZ=1000 # >>> >>>sysctl.conf: >>>kern.polling.enable=1 >>>net.inet.ip.intr_queue_maxlen=5000 >>>kern.ipc.maxsockbuf=8388608 >>>net.inet.tcp.sendspace=3217968 >>>net.inet.tcp.recvspace=3217968 >>>net.inet.tcp.rfc1323=1 >>> >>>bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9000 >>> options=5b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,POLLING> >>> inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 >>> ether 00:17:a4:3a:e1:81 >>> media: Ethernet autoselect (1000baseTX <full-duplex>) >>> status: active >>>(note mtu 9000) >>> >>>and here are tests results: >>> >>>netperf: >>> >>>TCP STREAM TEST to 192.168.0.1 >>>Recv Send Send >>>Socket Socket Message Elapsed >>>Size Size Size Time Throughput >>>bytes bytes bytes secs. 10^6bits/sec >>> >>>6217968 6217968 6217968 10.22 320.04 >>> >>>UDP UNIDIRECTIONAL SEND TEST to 192.168.0.1 >>>Socket Message Elapsed Messages >>>Size Size Time Okay Errors Throughput >>>bytes bytes secs # # 10^6bits/sec >>> >>> 9216 9216 10.00 118851 1724281 876.20 >>>41600 10.00 0 0.00] >>> >>> >>> >>>iperf: >>>gate2# iperf -s -N >>>------------------------------------------------------------ >>>Server listening on TCP port 5001 >>>TCP window size: 3.07 MByte (default) >>>------------------------------------------------------------ >>>[ 4] local 192.168.0.2 port 5001 connected with 192.168.0.1 port 52597 >>>[ 4] 0.0-10.1 sec 384 MBytes 319 Mbits/sec >>> >>>Also I can say, that I've managed to achieve about 500mbit.s by tuning tcp window with -w key in iperf. >>> >>>How can we explain such a low tcp performance? What else is to tune? Is there somebody who achieved gigabit speed with tcp on freebsd? >>> >>>Thanks. >>> >>> >>> >>>_______________________________________________ >>>freebsd-net@freebsd.org mailing list >>>http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> >>You also need kern.polling.idle_poll=1 and maybe >>net.inet.ip.fastforwarding=1 though I never noticed any difference with >>that one enabled. I got about 840Mbit/s routed through a dell 1850 >>(EMT64 running AMD64) with em-interfaces (I only used one physical IF >>though with 2 VLAN-if). >> >> >> >> >_______________________________________________ >freebsd-net@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-net >To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44DACAAF.5020306>