From owner-freebsd-xen@freebsd.org Mon Jun 24 20:24:24 2019 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B656E15D8938 for ; Mon, 24 Jun 2019 20:24:23 +0000 (UTC) (envelope-from christian.marcos@gmail.com) Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B3B5873B6 for ; Mon, 24 Jun 2019 20:24:22 +0000 (UTC) (envelope-from christian.marcos@gmail.com) Received: by mail-io1-xd30.google.com with SMTP id u19so3996092ior.9 for ; Mon, 24 Jun 2019 13:24:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=Yn48fhCgEhx/16BISpbKwDTGsPo6khpuN3J3Vm6DWyg=; b=W0qdSrpCIJiS9WI4NQDp2meg7pHKrIfwz+BSvAmi2rQyo2jUIzhICs5a+j0eqkIRUN ymdgUvxuDIDm0jFlwUCSWYtEvevYWvEzCsYLFXeAvMZuGQCsxGrxTdLa3kfqR7QR/AXh 2kXJToH3XsgkrZ/Lf2Iv2ooWnrfoQlqVvPiQfRxIodjKAqbgzXHrmlH4yZOTn6PcnB0r LanlJzFzVJUgy1U1NpTiTExfn/lH/eC6I8Dyk0p8KIhNl3HlnqtMlAJSL64SXRb+KEud RCm8j6zM8dn+0uRDlk2iHSS6W2+r8IWtgcbpEImmtIrGRi7U1wkZLcbBtARl05ggeRLV uCuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Yn48fhCgEhx/16BISpbKwDTGsPo6khpuN3J3Vm6DWyg=; b=DUdTUPMoWxPSt6ZfDf6O+UmKnYrtxIosE5xd0jkXLQ9/cw2XzwtJ5+WcORwtplUoIb bE3Go4w8YXD3yyjuB06w3ZgYwMv1N70h/993pbY/mwXYXH6B7c0pkt5MHFNpfXJ1yXp2 6+0QbDhhZphH83IY/Y/DgE7tfyjPStiHnFKcoMqQdh3ruHzL8vELATv4MOYCOhC7eS21 TAtAGXAZmJJWQZ3VN6QTxGqqmcgJhU66RvGjSviAlI1hmmDrM3q+xU2MXq2fth/nrS3k Y/3r/3ri+weW6pZOhs14Vm+7fHrrkTM7jSExKVUCVYltnozIdnSWnLRh/QOyOCb3nxbF mU6A== X-Gm-Message-State: APjAAAUpFmXle3iOLvNcHupFAZ7YRCTf8ealvNE0OC8vMowY0JJdqV5h PWOPe8tIPuKKSvkEIAVOadgjCc8KkIfaY04KIX+gSf5Q X-Google-Smtp-Source: APXvYqx46TxzLo2I+NivkGv5gC5oIztTPhe9j/L24lVBibeIqsRFbEN8Iv9YWEhlElRxHPVZ/wxRFBqsDBR3rZ3t5zg= X-Received: by 2002:a6b:4107:: with SMTP id n7mr10962003ioa.12.1561407861218; Mon, 24 Jun 2019 13:24:21 -0700 (PDT) MIME-Version: 1.0 References: <12994df1-f847-ec92-aae8-43a32e59385f@darco.dk> In-Reply-To: <12994df1-f847-ec92-aae8-43a32e59385f@darco.dk> From: Christian M Date: Mon, 24 Jun 2019 22:24:09 +0200 Message-ID: Subject: Re: Very slow and inconsistent internal network speed (between, VM's on the same host) for FreeBSD 11.0+ as guest on, XCP-ng/XenServer To: freebsd-xen@freebsd.org X-Rspamd-Queue-Id: 3B3B5873B6 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=W0qdSrpC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of christianmarcos@gmail.com designates 2607:f8b0:4864:20::d30 as permitted sender) smtp.mailfrom=christianmarcos@gmail.com X-Spamd-Result: default: False [-6.82 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.97)[-0.965,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-xen@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.95)[ip: (-9.20), ipnet: 2607:f8b0::/32(-3.15), asn: 15169(-2.33), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[0.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2019 20:24:24 -0000 Thanks for your input Bjarne. Previously I tested with iperf2, but I have made som tests with iperf3 now also and noticed there are a lot of "Retr" (TCP retries) in some cases. Went back here to your post and saw that your results also showed a lot of retries in some cases. My new tests showed similar results to yours (although not nearly as hi throughput as you have). When Linux is client the retries are 0, with FreeBSD as client the number is > 0. I'm not sure if our numbers are considered high though, and something that actually is a problem? Although, on a internal network with no external factors like interference, I feel this should always be 0 no matter what? What hypervisor were you running in your tests? Version? Regards, Christian Den m=C3=A5n 24 juni 2019 kl 17:56 skrev Bjarne : > Well, to add some datapoints, I just did a quick test om my private > homeserver, testing with iperf3 > -s / iperf3 -c > > I have not much knowledge of iperf3 network test and it's validity, but > there is definitely some > differences to be seen. > > Summary: > > FBSD <-> FBSD 3,5Gbit/sec > FBSD -> Centos: 3GBit/Sec > Centos -> FBSD: 12 Gbit/sec > > Centos <-> Centos: 12 Gbit/sec > OpenSuse <-> OpenSuse 13 GBit/sec > > So it ssems there is some fishy when Freebsd is used as client > > xen running on opensuse, 11 vm's, freebsd , openSuse and Centos. No vm > has been optimized in any > way, everything is out of the box. > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > Test 1, Freebsd 12 -> freebsd 12 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > > Server Freebsd 12: > ccepted connection from 192.168.1.100, port 39518 > [ 5] local 192.168.1.99 port 5201 connected to 192.168.1.100 port 48518 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 374 MBytes 3.14 Gbits/sec > [ 5] 1.00-2.00 sec 444 MBytes 3.72 Gbits/sec > [ 5] 2.00-3.00 sec 411 MBytes 3.45 Gbits/sec > [ 5] 3.00-4.00 sec 438 MBytes 3.67 Gbits/sec > [ 5] 4.00-5.00 sec 413 MBytes 3.46 Gbits/sec > [ 5] 5.00-6.00 sec 417 MBytes 3.50 Gbits/sec > [ 5] 6.00-7.00 sec 412 MBytes 3.45 Gbits/sec > [ 5] 7.00-8.00 sec 432 MBytes 3.63 Gbits/sec > [ 5] 8.00-9.00 sec 418 MBytes 3.51 Gbits/sec > [ 5] 9.00-10.00 sec 375 MBytes 3.15 Gbits/sec > [ 5] 10.00-10.00 sec 332 KBytes 2.17 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.00 sec 4.04 GBytes 3.47 Gbits/sec > receiver > > Client, freebsd 12: > Connecting to host 10.192.168.99, port 5201 > [ 5] local 192.168.1.100 port 48518 connected to 192.168.1.99 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 375 MBytes 3.14 Gbits/sec 239 178 KBytes > [ 5] 1.00-2.00 sec 445 MBytes 3.73 Gbits/sec 834 74.1 KBytes > [ 5] 2.00-3.00 sec 411 MBytes 3.45 Gbits/sec 794 38.5 KBytes > [ 5] 3.00-4.00 sec 438 MBytes 3.68 Gbits/sec 808 198 KBytes > [ 5] 4.00-5.00 sec 413 MBytes 3.46 Gbits/sec 777 67.0 KBytes > [ 5] 5.00-6.00 sec 417 MBytes 3.50 Gbits/sec 795 120 KBytes > [ 5] 6.00-7.00 sec 412 MBytes 3.45 Gbits/sec 763 72.7 KBytes > [ 5] 7.00-8.00 sec 432 MBytes 3.62 Gbits/sec 810 329 KBytes > [ 5] 8.00-9.00 sec 418 MBytes 3.51 Gbits/sec 777 219 KBytes > [ 5] 9.00-10.00 sec 375 MBytes 3.15 Gbits/sec 688 304 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 4.04 GBytes 3.47 Gbits/sec 7285 > sender > [ 5] 0.00-10.00 sec 4.04 GBytes 3.47 Gbits/sec > receiver > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > test 2, centos 7.6 -> Freebsd 12 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > > Server Freebsd 12: > Accepted connection from 192.168.1.162, port 49180 > [ 5] local 192.168.1.99 port 5201 connected to 192.168.1.162 port 49182 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 1.17 GBytes 10.0 Gbits/sec > [ 5] 1.00-2.00 sec 1.47 GBytes 12.6 Gbits/sec > [ 5] 2.00-3.00 sec 1.30 GBytes 11.2 Gbits/sec > [ 5] 3.00-4.00 sec 1.32 GBytes 11.3 Gbits/sec > [ 5] 4.00-5.00 sec 1.44 GBytes 12.4 Gbits/sec > [ 5] 5.00-6.00 sec 1.29 GBytes 11.1 Gbits/sec > [ 5] 6.00-7.00 sec 1.32 GBytes 11.3 Gbits/sec > [ 5] 7.00-8.00 sec 1.36 GBytes 11.7 Gbits/sec > [ 5] 8.00-9.00 sec 1.32 GBytes 11.3 Gbits/sec > [ 5] 9.00-10.00 sec 1.41 GBytes 12.1 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.00 sec 13.4 GBytes 11.5 Gbits/sec > receiver > > Client, centos 7.6: > Connecting to host 192.168.1.99, port 5201 > [ 4] local 192.168.1.162 port 49182 connected to 192.168.1.99 port 5201 > [ ID] Interval Transfer Bandwidth Retr Cwnd > [ 4] 0.00-1.00 sec 1.17 GBytes 10.1 Gbits/sec 0 822 KBytes > [ 4] 1.00-2.00 sec 1.47 GBytes 12.6 Gbits/sec 0 1.53 MBytes > [ 4] 2.00-3.00 sec 1.30 GBytes 11.2 Gbits/sec 0 1.57 MBytes > [ 4] 3.00-4.00 sec 1.32 GBytes 11.3 Gbits/sec 0 1.64 MBytes > [ 4] 4.00-5.00 sec 1.44 GBytes 12.4 Gbits/sec 0 1.70 MBytes > [ 4] 5.00-6.00 sec 1.29 GBytes 11.1 Gbits/sec 0 1.70 MBytes > [ 4] 6.00-7.00 sec 1.32 GBytes 11.3 Gbits/sec 0 1.70 MBytes > [ 4] 7.00-8.00 sec 1.36 GBytes 11.7 Gbits/sec 0 1.70 MBytes > [ 4] 8.00-9.00 sec 1.32 GBytes 11.3 Gbits/sec 0 1.70 MBytes > [ 4] 9.00-10.00 sec 1.41 GBytes 12.1 Gbits/sec 0 1.70 MBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bandwidth Retr > [ 4] 0.00-10.00 sec 13.4 GBytes 11.5 Gbits/sec 0 sender > [ 4] 0.00-10.00 sec 13.4 GBytes 11.5 Gbits/sec > receiver > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > test 3, Freebsd 12 -> Centos 7.6 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > > Server, Centos 7.6: > Accepted connection from 192.168.1.100, port 20309 > [ 5] local 192.168.1.155 port 5201 connected to 192.168.1.100 port 17917 > [ ID] Interval Transfer Bandwidth > [ 5] 0.00-1.00 sec 335 MBytes 2.81 Gbits/sec > [ 5] 1.00-2.00 sec 379 MBytes 3.18 Gbits/sec > [ 5] 2.00-3.00 sec 368 MBytes 3.08 Gbits/sec > [ 5] 3.00-4.00 sec 371 MBytes 3.11 Gbits/sec > [ 5] 4.00-5.00 sec 374 MBytes 3.14 Gbits/sec > [ 5] 5.00-6.00 sec 362 MBytes 3.04 Gbits/sec > [ 5] 6.00-7.00 sec 381 MBytes 3.19 Gbits/sec > [ 5] 7.00-8.00 sec 384 MBytes 3.22 Gbits/sec > [ 5] 8.00-9.00 sec 374 MBytes 3.14 Gbits/sec > [ 5] 9.00-10.00 sec 366 MBytes 3.07 Gbits/sec > [ 5] 10.00-10.13 sec 56.4 MBytes 3.52 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bandwidth > [ 5] 0.00-10.13 sec 0.00 Bytes 0.00 bits/sec sender > [ 5] 0.00-10.13 sec 3.66 GBytes 3.10 Gbits/sec > receiver > > Client, Freebsd 12: > 5] local 192.168.1.100 port 17917 connected to 192.168.1.155 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 387 MBytes 3.25 Gbits/sec 83 309 KBytes > [ 5] 1.00-2.00 sec 380 MBytes 3.19 Gbits/sec 88 11.4 KBytes > [ 5] 2.00-3.00 sec 365 MBytes 3.06 Gbits/sec 83 54.2 KBytes > [ 5] 3.00-4.00 sec 372 MBytes 3.12 Gbits/sec 86 322 KBytes > [ 5] 4.00-5.00 sec 370 MBytes 3.11 Gbits/sec 83 2.85 KBytes > [ 5] 5.00-6.00 sec 366 MBytes 3.07 Gbits/sec 84 298 KBytes > [ 5] 6.00-7.00 sec 382 MBytes 3.20 Gbits/sec 87 356 KBytes > [ 5] 7.00-8.00 sec 380 MBytes 3.19 Gbits/sec 87 369 KBytes > [ 5] 8.00-9.00 sec 374 MBytes 3.14 Gbits/sec 87 29.9 KBytes > [ 5] 9.00-10.00 sec 375 MBytes 3.15 Gbits/sec 86 419 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 3.66 GBytes 3.15 Gbits/sec 854 sender > [ 5] 0.00-10.00 sec 3.66 GBytes 3.15 Gbits/sec > receiver > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > test 4, Centos 7.6 -> Centos 7.6 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D > > Server: > Accepted connection from 192.168.1.162, port 47606 > [ 5] local 192.168.1.155 port 5201 connected to 192.168.1.162 port 47608 > [ ID] Interval Transfer Bandwidth > [ 5] 0.00-1.00 sec 1.60 GBytes 13.7 Gbits/sec > [ 5] 1.00-2.00 sec 1.59 GBytes 13.7 Gbits/sec > [ 5] 2.00-3.00 sec 1.60 GBytes 13.8 Gbits/sec > [ 5] 3.00-4.00 sec 1.71 GBytes 14.7 Gbits/sec > [ 5] 4.00-5.00 sec 1.49 GBytes 12.8 Gbits/sec > [ 5] 5.00-6.00 sec 1.38 GBytes 11.9 Gbits/sec > [ 5] 6.00-7.00 sec 1.52 GBytes 13.1 Gbits/sec > [ 5] 7.00-8.00 sec 1.51 GBytes 13.0 Gbits/sec > [ 5] 8.00-9.00 sec 1.51 GBytes 13.0 Gbits/sec > [ 5] 9.00-10.00 sec 1.51 GBytes 12.9 Gbits/sec > [ 5] 10.00-10.03 sec 48.1 MBytes 11.9 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bandwidth > [ 5] 0.00-10.03 sec 0.00 Bytes 0.00 bits/sec sender > [ 5] 0.00-10.03 sec 15.5 GBytes 13.2 Gbits/sec > receiver > > Client, Centos 7.6 > Connecting to host 192.168.1.155, port 5201 > [ 4] local 192.168.1.162 port 47608 connected to 192.168.1.155 port 5201 > [ ID] Interval Transfer Bandwidth Retr Cwnd > [ 4] 0.00-1.00 sec 1.66 GBytes 14.3 Gbits/sec 0 1.41 MBytes > [ 4] 1.00-2.00 sec 1.58 GBytes 13.5 Gbits/sec 0 1.46 MBytes > [ 4] 2.00-3.00 sec 1.61 GBytes 13.9 Gbits/sec 0 1.91 MBytes > [ 4] 3.00-4.00 sec 1.71 GBytes 14.7 Gbits/sec 0 2.13 MBytes > [ 4] 4.00-5.00 sec 1.47 GBytes 12.6 Gbits/sec 0 2.49 MBytes > [ 4] 5.00-6.00 sec 1.39 GBytes 11.9 Gbits/sec 0 2.56 MBytes > [ 4] 6.00-7.00 sec 1.53 GBytes 13.1 Gbits/sec 0 2.71 MBytes > [ 4] 7.00-8.00 sec 1.50 GBytes 12.9 Gbits/sec 0 2.73 MBytes > [ 4] 8.00-9.00 sec 1.50 GBytes 12.9 Gbits/sec 0 2.81 MBytes > [ 4] 9.00-10.00 sec 1.51 GBytes 12.9 Gbits/sec 0 2.84 MBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bandwidth Retr > [ 4] 0.00-10.00 sec 15.5 GBytes 13.3 Gbits/sec 0 sender > [ 4] 0.00-10.00 sec 15.5 GBytes 13.3 Gbits/sec > receiver > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > test4, Opensuse 15.1 -> OpenSuse 15.1 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > Server, Opensuse 15.1: > [ 5] local 192.168.1.158 port 5201 connected to 192.168.1.163 port 39334 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 1.51 GBytes 13.0 Gbits/sec > [ 5] 1.00-2.00 sec 1.61 GBytes 13.8 Gbits/sec > [ 5] 2.00-3.00 sec 1.58 GBytes 13.6 Gbits/sec > [ 5] 3.00-4.00 sec 1.58 GBytes 13.6 Gbits/sec > [ 5] 4.00-5.00 sec 1.56 GBytes 13.4 Gbits/sec > [ 5] 5.00-6.00 sec 1.60 GBytes 13.7 Gbits/sec > [ 5] 6.00-7.00 sec 1.58 GBytes 13.5 Gbits/sec > [ 5] 7.00-8.00 sec 1.60 GBytes 13.8 Gbits/sec > [ 5] 8.00-9.00 sec 1.60 GBytes 13.7 Gbits/sec > [ 5] 9.00-10.00 sec 1.61 GBytes 13.8 Gbits/sec > [ 5] 10.00-10.04 sec 68.0 MBytes 14.2 Gbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.04 sec 15.9 GBytes 13.6 Gbits/sec > receiver > > Client, OpenSuse 15.1: > [ 5] local 192.168.1.163 port 39334 connected to 192.168.1.158 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 1.58 GBytes 13.6 Gbits/sec 0 723 KBytes > [ 5] 1.00-2.00 sec 1.61 GBytes 13.8 Gbits/sec 0 805 KBytes > [ 5] 2.00-3.00 sec 1.57 GBytes 13.5 Gbits/sec 0 926 KBytes > [ 5] 3.00-4.00 sec 1.58 GBytes 13.6 Gbits/sec 0 926 KBytes > [ 5] 4.00-5.00 sec 1.56 GBytes 13.4 Gbits/sec 0 970 KBytes > [ 5] 5.00-6.00 sec 1.60 GBytes 13.7 Gbits/sec 0 1.05 MBytes > [ 5] 6.00-7.00 sec 1.57 GBytes 13.5 Gbits/sec 0 1.05 MBytes > [ 5] 7.00-8.00 sec 1.61 GBytes 13.8 Gbits/sec 0 1.05 MBytes > [ 5] 8.00-9.00 sec 1.59 GBytes 13.7 Gbits/sec 0 1.05 MBytes > [ 5] 9.00-10.00 sec 1.62 GBytes 13.9 Gbits/sec 0 1.05 MBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 15.9 GBytes 13.7 Gbits/sec 0 sender > [ 5] 0.00-10.04 sec 15.9 GBytes 13.6 Gbits/sec > receiver > > > > > > >