From owner-freebsd-net@freebsd.org Thu Jun 27 10:19:18 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A79BD15B3A3B; Thu, 27 Jun 2019 10:19:18 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com [216.71.155.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "esa6.hc3370-68.iphmx.com", Issuer "HydrantID SSL ICA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 92A63773B8; Thu, 27 Jun 2019 10:19:16 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: klK2fiCCg2V4c0CSNpWZDbjxnXCowAICkvvkv05lN/A84M2pcCEBq7xulR/L9JMmgrNSrUM+Zn CPDvsbIjE2CNbN6xiCwgfxqpwun/1Qkz7WAk7eHDwFG+90jyT1k6/mIhVF5VjULhNmO8kcI+gv b/Fwdc5l2nOMCGrurdu0wr8INPzs4upSpvwPqImOwaG0AFLSvIc9NVZNN8IcTQXSJkCL2LuyXj V5MIc1CvYRbM/EgtvnJYTy3DYyuwFZYzOJRMzx4j5JHBgP2PrQK5KOlxVRNfJr9pxBJsnSEzz0 sZs= X-SBRS: 2.7 X-MesageID: 2313737 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.63,423,1557201600"; d="scan'208";a="2313737" Date: Thu, 27 Jun 2019 12:19:01 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Christian M CC: Eugene Grosbein , , Subject: Re: Very slow and inconsistent internal network speed (between, VM's on the same host) for FreeBSD 11.0+ as guest on, XCP-ng/XenServer Message-ID: <20190627101901.3szaaxhp4xzjiay5@MacBook-Air-de-Roger.local> References: <8ec84f75-2100-ff8a-6337-3c955f31da62@grosbein.net> <9c377fe7-5546-9e0b-fac9-076dd094cbac@grosbein.net> <47bb1bfe-ac39-78d3-c9b9-17ff1896e0ec@grosbein.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20180716 X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-Rspamd-Queue-Id: 92A63773B8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of roger.pau@citrix.com designates 216.71.155.175 as permitted sender) smtp.mailfrom=roger.pau@citrix.com X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:216.71.155.175]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[citrix.com]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com,mx1.hc3370-68.iphmx.com,mx2.hc3370-68.iphmx.com]; NEURAL_HAM_SHORT(-0.82)[-0.821,0]; RCVD_IN_DNSWL_NONE(0.00)[175.155.71.216.list.dnswl.org : 127.0.3.0]; IP_SCORE(-0.87)[ip: (-1.33), ipnet: 216.71.154.0/23(-1.86), asn: 16417(-1.11), country: US(-0.06)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16417, ipnet:216.71.154.0/23, country:US]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 10:19:19 -0000 On Thu, Jun 27, 2019 at 12:14:33PM +0200, Christian M wrote: > I've installed 12.0-STABLE on two new VM's now. 172.31.16.127 and .128. VIF > cheksum offloading is turned off, and -txcsum for xn0 for both VM's. > > I feel the throughput is more consistent now, not all over the place as > before, even between runs. But the Retr column (tcp retries) in iperf3 has > jumped up considerably from hundreds/s to thousands/s. > > Just a reminder, I have tested this with 11.0-RELEASE also, where the issue > appeared first for me. 10.4-RELEASE is as fast as I could expect it to be, > and 0 retries. > > 12.0-STABLE: > > Connecting to host 172.31.16.128, port 5201 > [ 5] local 172.31.16.127 port 16833 connected to 172.31.16.128 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 96.3 MBytes 808 Mbits/sec 2401 2.85 KBytes > > [ 5] 1.00-2.00 sec 118 MBytes 991 Mbits/sec 3120 17.0 KBytes > > [ 5] 2.00-3.00 sec 121 MBytes 1.02 Gbits/sec 3203 69.8 KBytes > > [ 5] 3.00-4.00 sec 102 MBytes 853 Mbits/sec 3126 15.6 KBytes > > [ 5] 4.00-5.00 sec 110 MBytes 921 Mbits/sec 2890 15.6 KBytes > > [ 5] 5.00-6.00 sec 108 MBytes 908 Mbits/sec 3308 17.0 KBytes > > [ 5] 6.00-7.00 sec 104 MBytes 869 Mbits/sec 3046 48.2 KBytes > > [ 5] 7.00-8.00 sec 98.9 MBytes 830 Mbits/sec 2845 2.85 KBytes > > [ 5] 8.00-9.00 sec 104 MBytes 874 Mbits/sec 2711 86.8 KBytes > > [ 5] 9.00-10.00 sec 108 MBytes 904 Mbits/sec 2696 14.2 KBytes > > [ 5] 10.00-11.00 sec 103 MBytes 864 Mbits/sec 2660 31.3 KBytes > > [ 5] 11.00-12.00 sec 98.8 MBytes 828 Mbits/sec 2476 19.9 KBytes > > [ 5] 12.00-13.00 sec 99.9 MBytes 838 Mbits/sec 2857 11.3 KBytes > > [ 5] 13.00-14.00 sec 107 MBytes 894 Mbits/sec 2685 24.1 KBytes > > [ 5] 14.00-15.00 sec 114 MBytes 953 Mbits/sec 2321 25.5 KBytes > > [ 5] 15.00-16.00 sec 93.1 MBytes 781 Mbits/sec 2427 48.3 KBytes > > [ 5] 16.00-17.00 sec 107 MBytes 895 Mbits/sec 2219 29.8 KBytes > > [ 5] 17.00-18.00 sec 92.5 MBytes 776 Mbits/sec 2441 12.8 KBytes > > [ 5] 18.00-19.00 sec 116 MBytes 976 Mbits/sec 2840 38.2 KBytes > > [ 5] 19.00-20.00 sec 102 MBytes 853 Mbits/sec 2573 43.9 KBytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-20.00 sec 2.05 GBytes 882 Mbits/sec 54845 Can you paste the output of ifconfig for both the interfaces used in the test? Are you sure all hardware offloading capabilities are turned off on both interfaces? Can you check what's causing those retries? Either using tcpdump, whireshark or some other tool to analyze the network traffic and detect the errors that cause such retries? Thanks, Roger.