Date: Wed, 09 Aug 2006 16:12:04 +0400 From: Dima Roshin <roshind@mail.ru> To: freebsd-net@freebsd.org Subject: Re[2]: Gigabit ethernet questions? Message-ID: <E1GAmuq-000CR8-00.roshind-mail-ru@f60.mail.ru> In-Reply-To: =?koi8-r?Q?<44D9CCEA.2050103=40ide.resurscentrum.se>?=
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks Jon, I did it on both sides, thats much better now: gate1# sysctl kern.polling.idle_poll=1 kern.polling.idle_poll: 0 -> 1 gate1# sysctl net.inet.ip.fastforwarding net.inet.ip.fastforwarding: 0 gate1# sysctl net.inet.ip.fastforwarding=1 net.inet.ip.fastforwarding: 0 -> 1 gate1# iperf -c 192.168.0.2 -N -w 196000 ------------------------------------------------------------ Client connecting to 192.168.0.2, TCP port 5001 TCP window size: 192 KByte (WARNING: requested 191 KByte) ------------------------------------------------------------ [ 3] local 192.168.0.1 port 63941 connected with 192.168.0.2 port 5001 [ 3] 0.0-10.0 sec 762 MBytes 639 Mbits/sec But there is still some bottleneck, and I can't understand where. -----Original Message----- From: Jon Otterholm <jon.otterholm@ide.resurscentrum.se> To: Dima Roshin <roshind@mail.ru> Date: Wed, 09 Aug 2006 13:54:18 +0200 Subject: Re: Gigabit ethernet questions? > > Dima Roshin wrote: > > Greeting colleagues. I've got two DL-360(pciX bus) servers, with BCM5704 NetXtreme Dual Gigabit Adapters(bge). The Uname is 6.1-RELEASE-p3. The bge interfaces of the both servers are connected with each other with a cat6 patchcord. > > Here are my settings: > > kernel config: > > options DEVICE_POLLING > > options HZ=1000 # > > > > sysctl.conf: > > kern.polling.enable=1 > > net.inet.ip.intr_queue_maxlen=5000 > > kern.ipc.maxsockbuf=8388608 > > net.inet.tcp.sendspace=3217968 > > net.inet.tcp.recvspace=3217968 > > net.inet.tcp.rfc1323=1 > > > > bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9000 > > options=5b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,POLLING> > > inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 > > ether 00:17:a4:3a:e1:81 > > media: Ethernet autoselect (1000baseTX <full-duplex>) > > status: active > > (note mtu 9000) > > > > and here are tests results: > > > > netperf: > > > > TCP STREAM TEST to 192.168.0.1 > > Recv Send Send > > Socket Socket Message Elapsed > > Size Size Size Time Throughput > > bytes bytes bytes secs. 10^6bits/sec > > > > 6217968 6217968 6217968 10.22 320.04 > > > > UDP UNIDIRECTIONAL SEND TEST to 192.168.0.1 > > Socket Message Elapsed Messages > > Size Size Time Okay Errors Throughput > > bytes bytes secs # # 10^6bits/sec > > > > 9216 9216 10.00 118851 1724281 876.20 > > 41600 10.00 0 0.00] > > > > > > > > iperf: > > gate2# iperf -s -N > > ------------------------------------------------------------ > > Server listening on TCP port 5001 > > TCP window size: 3.07 MByte (default) > > ------------------------------------------------------------ > > [ 4] local 192.168.0.2 port 5001 connected with 192.168.0.1 port 52597 > > [ 4] 0.0-10.1 sec 384 MBytes 319 Mbits/sec > > > > Also I can say, that I've managed to achieve about 500mbit.s by tuning tcp window with -w key in iperf. > > > > How can we explain such a low tcp performance? What else is to tune? Is there somebody who achieved gigabit speed with tcp on freebsd? > > > > Thanks. > > > > > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > You also need kern.polling.idle_poll=1 and maybe > net.inet.ip.fastforwarding=1 though I never noticed any difference with > that one enabled. I got about 840Mbit/s routed through a dell 1850 > (EMT64 running AMD64) with em-interfaces (I only used one physical IF > though with 2 VLAN-if). > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1GAmuq-000CR8-00.roshind-mail-ru>