Date: Mon, 13 Jul 2020 00:44:02 -0700 From: Mark Millard <marklmi@yahoo.com> To: John-Mark Gurney <jmg@funkthat.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: slow USB 3.0 on -current Message-ID: <1F4704D0-DD42-4DD2-A15C-D89FEF2FA382@yahoo.com> In-Reply-To: <20200713045149.GL4213@funkthat.com> References: <9D8F806C-2F11-4338-9905-E91BBCDEFC01.ref@yahoo.com> <9D8F806C-2F11-4338-9905-E91BBCDEFC01@yahoo.com> <20200713045149.GL4213@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2020-Jul-12, at 21:51, John-Mark Gurney <jmg at funkthat.com> wrote: > Mark Millard wrote this message on Sun, Jul 12, 2020 at 18:26 -0700: >> John-Mark Gurney jmg at funkthat.com wrote on >> Sat Jul 11 22:44:36 UTC 2020 : >>=20 >>> I'm having issues getting good ethernet performance from a USB = ethernet >>> adapter (ure) under FreeBSD on an HP EliteDesk 705 G2 Mini[1]. It's = an >>> AMD PRO A10-8700B based system using the AMD A78 FCH chipset. >>>=20 >>> Under FreeBSD -current (r362596), 12.1-R and 11.4-R, the RealTek USB >>> adapter only gets around 10MB/sec performance. During the transfer, >>> the CPU usage is only around 3-5%, so it's definitely not CPU bound. >>>=20 >>> I have tested Windows 10 and NetBSD 9.0 performance, and both = provide >>> 100MB/sec+ w/o troubles. >>>=20 >>> I have attached dmesg from both FreeBSD -current and NetBSD 9.0. >>>=20 >>> Any hints on how to fix this? >>>=20 >>> This may be related, but I'm also having issues w/ booting when I = have >>> both a SD USB 2.0 card reader AND the ure plugged into USB 3.0 = ports. >>>=20 >>> If I move the SD card reader to USB 2.0, the umass device will = attach >>> and work. I have also attached a clip of the dmesg from that >>> happening. >>>=20 >>> Has anyone else seen this issue? Ideas or thoughts on how to = resolve >>> the performance issues? >>=20 >> It might prove useful to use iperf3 with >>=20 >> # iperf3 -s >>=20 >> on one machine and doing >>=20 >> # iperf3 -c ADDR >> . . . >> # iperf3 -R -c ADDR >> . . . >>=20 >> on the other. (That last swaps the >> sender/receiver status.) >>=20 >> All 3 commands will have output. The >> -s one will produce output for each of >> the -c ones. >>=20 >> The outputs for the sender(s) will include Cwnd >> (congestion window size) information that may >> be relevant. It will report bit rate and >> retry count sampling (and overall figures). >=20 > Here is the results for FreeBSD w/ USB3 ure. .80 is the USB3 > adapter side: > gold,pts,/home/jmg,502$iperf3 -c 192.168.0.80 > Connecting to host 192.168.0.80, port 5201 > [ 5] local 192.168.0.2 port 50042 connected to 192.168.0.80 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 8.94 MBytes 75.0 Mbits/sec 931 15.5 = KBytes > [ 5] 1.00-2.00 sec 9.98 MBytes 83.7 Mbits/sec 919 27.3 = KBytes > [ 5] 2.00-3.00 sec 9.95 MBytes 83.5 Mbits/sec 954 5.71 = KBytes > [ 5] 3.00-4.00 sec 9.97 MBytes 83.7 Mbits/sec 939 28.7 = KBytes > [ 5] 4.00-5.00 sec 9.97 MBytes 83.6 Mbits/sec 951 17.3 = KBytes > [ 5] 5.00-6.00 sec 9.99 MBytes 83.8 Mbits/sec 913 31.5 = KBytes > [ 5] 6.00-7.00 sec 9.96 MBytes 83.5 Mbits/sec 956 20.1 = KBytes > [ 5] 7.00-8.00 sec 10.0 MBytes 83.9 Mbits/sec 913 33.0 = KBytes > [ 5] 8.00-9.00 sec 9.97 MBytes 83.6 Mbits/sec 945 24.4 = KBytes > [ 5] 9.00-10.00 sec 9.99 MBytes 83.8 Mbits/sec 916 34.4 = KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 98.7 MBytes 82.8 Mbits/sec 9337 = sender > [ 5] 0.00-10.25 sec 98.7 MBytes 80.8 Mbits/sec = receiver >=20 > iperf Done. > gold,pts,/home/jmg,503$iperf3 -R -c 192.168.0.80 > Connecting to host 192.168.0.80, port 5201 > Reverse mode, remote host 192.168.0.80 is sending > [ 5] local 192.168.0.2 port 51024 connected to 192.168.0.80 port 5201 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 9.69 MBytes 81.3 Mbits/sec > [ 5] 1.00-2.00 sec 10.7 MBytes 89.8 Mbits/sec > [ 5] 2.00-3.00 sec 10.7 MBytes 89.8 Mbits/sec > [ 5] 3.00-4.00 sec 10.7 MBytes 89.8 Mbits/sec > [ 5] 4.00-5.00 sec 10.7 MBytes 89.8 Mbits/sec > [ 5] 5.00-6.00 sec 10.7 MBytes 89.8 Mbits/sec > [ 5] 6.00-7.00 sec 10.7 MBytes 89.8 Mbits/sec > [ 5] 7.00-8.00 sec 10.4 MBytes 87.6 Mbits/sec > [ 5] 8.00-9.00 sec 10.7 MBytes 89.9 Mbits/sec > [ 5] 9.00-10.00 sec 10.7 MBytes 89.8 Mbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 106 MBytes 88.9 Mbits/sec 1381 = sender > [ 5] 0.00-10.00 sec 106 MBytes 88.7 Mbits/sec = receiver >=20 > iperf Done. The "iperf3 -s" should have had output with the Cwnd figures for the "Reverse mode" case above (and the distribution for the 1381 Retr total). They might not match when the earlier figures that you did report for the non-Reverse mode. > As you can see, it matches what I measured earlier. >=20 > And just to prove that the machine CAN move 100MB/sec, I've run iperf3 > using the onboard wired ethernet... I need multiple interfaces, which = is > why I'm bothering trying to get USB3 ethernet working. >=20 > This is using the onboard bge interface. It's IP is .79: > gold,pts,/home/jmg,507$iperf3 -c 192.168.0.79 > Connecting to host 192.168.0.79, port 5201 > [ 5] local 192.168.0.2 port 61500 connected to 192.168.0.79 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 101 MBytes 850 Mbits/sec 0 488 = KBytes > [ 5] 1.00-2.00 sec 112 MBytes 940 Mbits/sec 0 488 = KBytes > [ 5] 2.00-3.00 sec 112 MBytes 943 Mbits/sec 0 731 = KBytes > [ 5] 3.00-4.00 sec 112 MBytes 940 Mbits/sec 0 731 = KBytes > [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 0 731 = KBytes > [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 731 = KBytes > [ 5] 6.00-7.00 sec 112 MBytes 940 Mbits/sec 0 731 = KBytes > [ 5] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 0 731 = KBytes > [ 5] 8.00-9.00 sec 112 MBytes 940 Mbits/sec 0 731 = KBytes > [ 5] 9.00-10.00 sec 112 MBytes 940 Mbits/sec 0 731 = KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 1.08 GBytes 932 Mbits/sec 0 = sender > [ 5] 0.00-10.01 sec 1.08 GBytes 931 Mbits/sec = receiver >=20 > iperf Done. > gold,pts,/home/jmg,508$iperf3 -R -c 192.168.0.79 > Connecting to host 192.168.0.79, port 5201 > Reverse mode, remote host 192.168.0.79 is sending > [ 5] local 192.168.0.2 port 61726 connected to 192.168.0.79 port 5201 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 97.0 MBytes 814 Mbits/sec > [ 5] 1.00-2.00 sec 109 MBytes 917 Mbits/sec > [ 5] 2.00-3.00 sec 110 MBytes 920 Mbits/sec > [ 5] 3.00-4.00 sec 109 MBytes 917 Mbits/sec > [ 5] 4.00-5.00 sec 110 MBytes 920 Mbits/sec > [ 5] 5.00-6.00 sec 109 MBytes 916 Mbits/sec > [ 5] 6.00-7.00 sec 110 MBytes 919 Mbits/sec > [ 5] 7.00-8.00 sec 109 MBytes 917 Mbits/sec > [ 5] 8.00-9.00 sec 110 MBytes 919 Mbits/sec > [ 5] 9.00-10.00 sec 109 MBytes 916 Mbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.03 sec 1.06 GBytes 905 Mbits/sec 2866 = sender > [ 5] 0.00-10.00 sec 1.06 GBytes 907 Mbits/sec = receiver >=20 > iperf Done. Similar "iperf3 -s" output point here. >> Comparing the output of using iperf3 under >> NetBSD 9.0 or Windows 10 could be instructive. >=20 > And NetBSD 9.0. Here, NetBSD got assigned .50, but it's also using > the USB3 adapter. >=20 > gold,pts,/home/jmg,505$iperf3 -c 192.168.0.50 > Connecting to host 192.168.0.50, port 5201 > [ 5] local 192.168.0.2 port 55535 connected to 192.168.0.50 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 96.7 MBytes 811 Mbits/sec 0 50.9 = KBytes > [ 5] 1.00-2.00 sec 112 MBytes 938 Mbits/sec 0 82.1 = KBytes > [ 5] 2.00-3.00 sec 112 MBytes 938 Mbits/sec 14 114 = KBytes > [ 5] 3.00-4.00 sec 110 MBytes 924 Mbits/sec 60 146 = KBytes > [ 5] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 107 178 = KBytes > [ 5] 5.00-6.00 sec 112 MBytes 939 Mbits/sec 122 193 = KBytes > [ 5] 6.00-7.00 sec 112 MBytes 938 Mbits/sec 144 193 = KBytes > [ 5] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 165 193 = KBytes > [ 5] 8.00-9.00 sec 112 MBytes 939 Mbits/sec 144 193 = KBytes > [ 5] 9.00-10.00 sec 112 MBytes 938 Mbits/sec 165 193 = KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 1.08 GBytes 924 Mbits/sec 921 = sender > [ 5] 0.00-10.00 sec 1.08 GBytes 924 Mbits/sec = receiver >=20 > iperf Done. > gold,pts,/home/jmg,506$iperf3 -R -c 192.168.0.50 > Connecting to host 192.168.0.50, port 5201 > Reverse mode, remote host 192.168.0.50 is sending > [ 5] local 192.168.0.2 port 55691 connected to 192.168.0.50 port 5201 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 78.3 MBytes 657 Mbits/sec > [ 5] 1.00-2.00 sec 88.4 MBytes 742 Mbits/sec > [ 5] 2.00-3.00 sec 88.5 MBytes 742 Mbits/sec > [ 5] 3.00-4.00 sec 88.4 MBytes 741 Mbits/sec > [ 5] 4.00-5.00 sec 88.7 MBytes 744 Mbits/sec > [ 5] 5.00-6.00 sec 88.2 MBytes 740 Mbits/sec > [ 5] 6.00-7.00 sec 87.8 MBytes 737 Mbits/sec > [ 5] 7.00-8.00 sec 88.5 MBytes 742 Mbits/sec > [ 5] 8.00-9.00 sec 89.0 MBytes 746 Mbits/sec > [ 5] 9.00-10.00 sec 88.8 MBytes 745 Mbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 875 MBytes 734 Mbits/sec 0 = sender > [ 5] 0.00-10.00 sec 874 MBytes 734 Mbits/sec = receiver >=20 > iperf Done. Similar "iperf3 -s" output point here. > As you can see, both match approximately what I measured other = methods, > so, it's definitely not the way I measured performance. >=20 >> My observation would be that neither type >> of USB3 Ethernet adapter that I've tried >=20 > What is the chipset that you tried? One of the earlier ones that I > tried was an axe iirc, and was limited to around 500Mbps or so... Hmm. I only seem to be able to find one type. Its been a while since I've used the other and I do not know where it is at. For what I found: ugen0.2: <ASIX Elec. AX88179> at usbus0 axge0 on uhub0 axge0: <NetworkInterface> on usbus0 miibus1: <MII bus> on axge0 rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 3 on = miibus1 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, = 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow (I have access to more than one instance of the above.) The iperf3 output that I reported was for using of of the above. Note that when the USB3 EtherNet was reciveing Cwnd was reported as 29.8 KBytes or smaller for the example run, much like your output reporting 34.4 KBytes or less for the example run. This may suggest some common constraint across various USB3 EtherNet devices. The Cwnd figures are probably too small to get near 900 Mbit/s+. But, even with a (smaller but) similar Cwnd figure my example was getting faster transfers than your example. I got smaller Retr figures as well. It leaves me wondering if there are packets being rejected in your context that are not in my context. If what I reported would still be too slow, there may be two (or more) points to be addressed to get things going fast enough for you. You might be able to avoid one of the points by using a type of device that already does somewhat better. May be ask for the fastest examples folks have observed? >> (different chipsets) get anywhere near >> 100 MByte/s when ifconfig reports >> 1000baseT <full-duplex>. The Cwnd figures >> are smaller than for the built-in Ethernets >> that manage much faster overall transfer >> rates. >=20 > [...] >=20 >> I'll note that between machines with built-in EtherNet >> that can sustain fast transfers overall, the Cwnd figures >> tend to vary but can reach 1 MBytes+. The Retr counts >> tend to still exist. >>=20 >> By contrast, when the USB3 EtherNet is receiving above, >> the maximum Cwnd reported above for the sender at the >> time was: 29.8 KBytes. >>=20 >> I have not tried NetBSD, Windows 10, or Linux comparisons. >=20 > As you can see above, NetBSD easily achieves around 8-10x the > speed using the exact same USB3 device as FreeBSD does, so the > hardware CAN push the speeds, just FreeBSD cannot. The small Cwnd figures like 34.4 KBytes suggest that a small receiver window (Rwnd) might be specified in the TCP header that the destination provided for that transfer direction. Cwnd can increase up to the Rwnd unless duplicate ACKs or timeouts occur, as I understand. (I'm no expert.) This is part of the reason I thought that posting the output (with Cwnd) for both transfer directions could be of use: it gives a hint about what is controlling the Cwnd if the two directions have widely different figures vs. if both directions get similarly small figures. Comparisons with the other OS's figures in both directions could possibly also be suggestive, or so I hoped. May be the comparison with the figures that I reported gives someone some hints about what might be going on in the two contexts. > Hence, my original post, what can I do to possibly get FreeBSD's > performance up to what the hardware can achieve? >=20 Hopefully my notes prove of some use --but I'm not likely to directly solve the problem. It would be handy for me if USB3 EtherNet performed significantly better than I have observed. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1F4704D0-DD42-4DD2-A15C-D89FEF2FA382>