Date: Tue, 25 Jun 2019 13:55:40 +0200 From: Christian M <christian.marcos@gmail.com> To: freebsd-xen@freebsd.org, freebsd-net@freebsd.org Subject: Re: Very slow and inconsistent internal network speed (between, VM's on the same host) for FreeBSD 11.0+ as guest on, XCP-ng/XenServer Message-ID: <CAKwR996s9Y_Bea95GOhPkdEJabmJM0r6Qi43gW7_8SXE2ryddA@mail.gmail.com> In-Reply-To: <20190625082103.qiiz2cikrauaqlas@MacBook-Air-de-Roger.local> References: <mailman.7.1560945600.15387.freebsd-xen@freebsd.org> <12994df1-f847-ec92-aae8-43a32e59385f@darco.dk> <CAKwR994kQTRfB0R0_WiTYx4pgOHtR-ge7h0vP8NiM_%2B-MBPGGw@mail.gmail.com> <20190625082103.qiiz2cikrauaqlas@MacBook-Air-de-Roger.local>
next in thread | previous in thread | raw e-mail | index | archive | help
I've made two tests while running tcpdump on the xcp-ng host. I'm not at all qualified to interpret the .pcap files from tcpdump, but I've put them on Google Drive and linked them below the two tests. Perhaps someone more qualified could have a look for anything useful in there. Please note the extremely uneven throughput for test 2 below. It's almost like the throughput increased when running tcpdump simultaneously. Host: XCP-ng 7.6.0 Network: Private Network on host, not connected to any PIF. VM1: 12.0-RELEASE (1 VIF, 172.31.16.125) VM2: 12.0-RELEASE (1 VIF, 172.31.15.126) On the host I listen with tcpdump on the VIF for VM1 in both tests. VM1 as client: On XCP-ng: tcpdump -i vif42.0 -s 0 -w xcp-ng-vm1-client.pcap xcp-ng-vm1-client.pcap (80M): https://drive.google.com/open?id=3D1eR3fetvKRz3vFSXCxDKuJYFrQ3wLqjrU On VM1: iperf3 -c 172.31.16.126 On VM2: iperf3 -s VM1 iperf3 output: Connecting to host 172.31.16.126, port 5201 [ 5] local 172.31.16.125 port 18182 connected to 172.31.16.126 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 8.00 MBytes 67.1 Mbits/sec 156 15.6 KBytes [ 5] 1.00-2.00 sec 14.1 MBytes 118 Mbits/sec 318 1.43 KBytes [ 5] 2.00-3.00 sec 7.51 MBytes 63.1 Mbits/sec 218 1.43 KBytes [ 5] 3.00-4.00 sec 8.29 MBytes 69.3 Mbits/sec 193 1.43 KBytes [ 5] 4.00-5.01 sec 10.7 MBytes 89.4 Mbits/sec 252 1.43 KBytes [ 5] 5.01-6.00 sec 13.6 MBytes 115 Mbits/sec 313 31.3 KBytes [ 5] 6.00-7.00 sec 8.41 MBytes 70.2 Mbits/sec 309 1.43 KBytes [ 5] 7.00-8.01 sec 12.6 MBytes 106 Mbits/sec 223 1.43 KBytes [ 5] 8.01-9.00 sec 12.5 MBytes 106 Mbits/sec 227 1.43 KBytes [ 5] 9.00-10.00 sec 13.5 MBytes 113 Mbits/sec 263 1.43 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 109 MBytes 91.6 Mbits/sec 2472 sender [ 5] 0.00-10.01 sec 109 MBytes 91.3 Mbits/sec receiver VM1 as server: On XCP-ng: tcpdump -i vif42.0 -s 0 -w xcp-ng-vm1-server.pcap xcp-ng-vm1-server.pcap: (337M): https://drive.google.com/open?id=3D1N3-GccXaBc6hlzFrgshi2vhcCt8vekxF On VM1: iperf3 -s On VM2: iperf3 -c 172.31.16.125 VM1 iperf3 output: ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 172.31.16.126, port 31523 [ 5] local 172.31.16.125 port 5201 connected to 172.31.16.126 port 34605 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 10.9 MBytes 91.6 Mbits/sec [ 5] 1.00-2.00 sec 14.1 MBytes 118 Mbits/sec [ 5] 2.00-3.01 sec 4.92 MBytes 41.0 Mbits/sec [ 5] 3.01-4.00 sec 10.4 MBytes 87.9 Mbits/sec [ 5] 4.00-5.00 sec 8.52 MBytes 71.2 Mbits/sec [ 5] 5.00-6.00 sec 22.0 MBytes 185 Mbits/sec [ 5] 6.00-7.00 sec 9.25 MBytes 77.8 Mbits/sec [ 5] 7.00-8.00 sec 70.8 MBytes 594 Mbits/sec [ 5] 8.00-9.00 sec 89.5 MBytes 752 Mbits/sec [ 5] 9.00-10.00 sec 105 MBytes 884 Mbits/sec [ 5] 10.00-10.00 sec 180 KBytes 1.19 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 346 MBytes 290 Mbits/sec receiver Den tis 25 juni 2019 kl 10:21 skrev Roger Pau Monn=C3=A9 <roger.pau@citrix.= com>: > On Mon, Jun 24, 2019 at 10:24:09PM +0200, Christian M wrote: > > Thanks for your input Bjarne. > > > > Previously I tested with iperf2, but I have made som tests with iperf3 > now > > also and noticed there are a lot of "Retr" (TCP retries) in some cases. > > Went back here to your post and saw that your results also showed a lot > of > > retries in some cases. My new tests showed similar results to yours > > (although not nearly as hi throughput as you have). When Linux is clien= t > > the retries are 0, with FreeBSD as client the number is > 0. > > Can you figure out what caused those retries? Wrong MTU, bad > checksums? > > > I'm not sure > > if our numbers are considered high though, and something that actually > is a > > problem? Although, on a internal network with no external factors like > > interference, I feel this should always be 0 no matter what? > > If properly configured yes, there should be no retries when doing > intra-VM connections. > > Thanks, Roger. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKwR996s9Y_Bea95GOhPkdEJabmJM0r6Qi43gW7_8SXE2ryddA>