Date: Wed, 8 Apr 2015 23:13:50 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Marek Salwerowicz <marek_sal@wp.pl> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: RTT and TCP Window size doubts, bandwidth issues Message-ID: <20150408225751.D22893@sola.nimnet.asn.au> In-Reply-To: <552455EB.7080609@wp.pl> References: <552455EB.7080609@wp.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 8 Apr 2015 00:10:51 +0200, Marek Salwerowicz wrote: > Hi list, > > I am trying to find correct setup of sysctl's for following machines (VMs > under Vmware Workstation 8) to test large TCP window size: > > > There are 2 boxes, each of them has following setup: > - % uname -a > FreeBSD freeA 10.1-RELEASE-p6 FreeBSD 10.1-RELEASE-p6 #0: Tue Feb 24 19:00:21 > UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC > amd64 > > - 4GB of RAM > - 1 NIC > > without any modifications, iperf reports bandwidth speed ~1Gbit/s between > hosts: > > server-side: > # iperf -s > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 64.0 KByte (default) > ------------------------------------------------------------ > > client-side: > > # iperf -c 192.168.108.140 > ------------------------------------------------------------ > Client connecting to 192.168.108.140, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.108.141 port 35282 connected with 192.168.108.140 port > 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 1.46 GBytes 1.25 Gbits/sec > > > > I want to simulate (using dummynet) link between hosts with bandwidth > 400mbit/s and latency ~20ms. > In order to do that, I create the ipfw pipe on one box: > IPFW="ipfw -q" > > > $IPFW pipe 1 config bw 400Mbit/s delay 10ms > > $IPFW add 1500 pipe 1 ip from any to any > > > after running ipfw, bandwidth with default kernel sysctl becomes lower: > > client-side: > ------------------------------------------------------------ > Client connecting to 192.168.108.140, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.108.141 port 35340 connected with 192.168.108.140 port > 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.1 sec 12.5 MBytes 10.4 Mbits/sec > > > > I'd like to achieve bandwidth ~400Mbit/s. > > I've modified following sysctl's (both on client- and server-side): > > kern.ipc.maxsockbuf=33554432 # (default 2097152) > > net.inet.tcp.sendbuf_max=33554432 # (default 2097152) > net.inet.tcp.recvbuf_max=33554432 # (default 2097152) > > net.inet.tcp.cc.algorithm=htcp # (default newreno) #enabled in > /boot/loader.conf also > net.inet.tcp.cc.htcp.adaptive_backoff=1 # (default 0 ; disabled) > > net.inet.tcp.cc.htcp.rtt_scaling=1 # (default 0 ; disabled) > > net.inet.tcp.mssdflt=1460 # (default 536) > > net.inet.tcp.minmss=1300 # (default 216) > > net.inet.tcp.rfc1323=1 # (default 1) > net.inet.tcp.rfc3390=1 # (default 1) > > > net.inet.tcp.sendspace=8388608 # (default 32768) > net.inet.tcp.recvspace=8388608 # (default 65536) > > net.inet.tcp.sendbuf_inc=32768 # (default 8192 ) > net.inet.tcp.recvbuf_inc=65536 # (default 16384) > > > > But the results are not really good as I expected: > > server-side: > # iperf -s > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 8.00 MByte (default) > ------------------------------------------------------------ > > client-side: > # iperf -c 192.168.108.140 > ------------------------------------------------------------ > Client connecting to 192.168.108.140, TCP port 5001 > TCP window size: 8.00 MByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.108.141 port 21894 connected with 192.168.108.140 port > 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.1 sec 24.2 MBytes 20.2 Mbits/sec > > > I was trying to follow the articles: > - http://www.psc.edu/index.php/networking/641-tcp-tune > - https://fasterdata.es.net/host-tuning/freebsd/ > > But can't really figure out what / how should be tuned in order to achieve > good results. > > If anyone could find some time and give me some hints, I'd be pleased! Many of your tunings are over my head, but I can see that you've nicely emulated a half-duplex link; you want one pipe for each direction for a fast bidirectional link. See ipfw(8) /TRAFFIC SHAPING Also be sure of your setting of sysctl net.inet.ip.fw.one_pass, here and also re your later post re ipfw / dummynet, which also wants 2 pipes. cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150408225751.D22893>