From owner-freebsd-current@freebsd.org Mon Jul 13 04:51:54 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2FC67358831 for ; Mon, 13 Jul 2020 04:51:54 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B4rpc5sgFz4QD6 for ; Mon, 13 Jul 2020 04:51:52 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 06D4pnGZ080221 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 12 Jul 2020 21:51:49 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 06D4pnX5080220; Sun, 12 Jul 2020 21:51:49 -0700 (PDT) (envelope-from jmg) Date: Sun, 12 Jul 2020 21:51:49 -0700 From: John-Mark Gurney To: Mark Millard Cc: FreeBSD Current Subject: Re: slow USB 3.0 on -current Message-ID: <20200713045149.GL4213@funkthat.com> Mail-Followup-To: Mark Millard , FreeBSD Current References: <9D8F806C-2F11-4338-9905-E91BBCDEFC01.ref@yahoo.com> <9D8F806C-2F11-4338-9905-E91BBCDEFC01@yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9D8F806C-2F11-4338-9905-E91BBCDEFC01@yahoo.com> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Sun, 12 Jul 2020 21:51:49 -0700 (PDT) X-Rspamd-Queue-Id: 4B4rpc5sgFz4QD6 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [3.62 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.53)[0.534]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.97)[0.973]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.92)[0.918]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 04:51:54 -0000 Mark Millard wrote this message on Sun, Jul 12, 2020 at 18:26 -0700: > John-Mark Gurney jmg at funkthat.com wrote on > Sat Jul 11 22:44:36 UTC 2020 : > > > I'm having issues getting good ethernet performance from a USB ethernet > > adapter (ure) under FreeBSD on an HP EliteDesk 705 G2 Mini[1]. It's an > > AMD PRO A10-8700B based system using the AMD A78 FCH chipset. > > > > Under FreeBSD -current (r362596), 12.1-R and 11.4-R, the RealTek USB > > adapter only gets around 10MB/sec performance. During the transfer, > > the CPU usage is only around 3-5%, so it's definitely not CPU bound. > > > > I have tested Windows 10 and NetBSD 9.0 performance, and both provide > > 100MB/sec+ w/o troubles. > > > > I have attached dmesg from both FreeBSD -current and NetBSD 9.0. > > > > Any hints on how to fix this? > > > > This may be related, but I'm also having issues w/ booting when I have > > both a SD USB 2.0 card reader AND the ure plugged into USB 3.0 ports. > > > > If I move the SD card reader to USB 2.0, the umass device will attach > > and work. I have also attached a clip of the dmesg from that > > happening. > > > > Has anyone else seen this issue? Ideas or thoughts on how to resolve > > the performance issues? > > It might prove useful to use iperf3 with > > # iperf3 -s > > on one machine and doing > > # iperf3 -c ADDR > . . . > # iperf3 -R -c ADDR > . . . > > on the other. (That last swaps the > sender/receiver status.) > > All 3 commands will have output. The > -s one will produce output for each of > the -c ones. > > The outputs for the sender(s) will include Cwnd > (congestion window size) information that may > be relevant. It will report bit rate and > retry count sampling (and overall figures). Here is the results for FreeBSD w/ USB3 ure. .80 is the USB3 adapter side: gold,pts,/home/jmg,502$iperf3 -c 192.168.0.80 Connecting to host 192.168.0.80, port 5201 [ 5] local 192.168.0.2 port 50042 connected to 192.168.0.80 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 8.94 MBytes 75.0 Mbits/sec 931 15.5 KBytes [ 5] 1.00-2.00 sec 9.98 MBytes 83.7 Mbits/sec 919 27.3 KBytes [ 5] 2.00-3.00 sec 9.95 MBytes 83.5 Mbits/sec 954 5.71 KBytes [ 5] 3.00-4.00 sec 9.97 MBytes 83.7 Mbits/sec 939 28.7 KBytes [ 5] 4.00-5.00 sec 9.97 MBytes 83.6 Mbits/sec 951 17.3 KBytes [ 5] 5.00-6.00 sec 9.99 MBytes 83.8 Mbits/sec 913 31.5 KBytes [ 5] 6.00-7.00 sec 9.96 MBytes 83.5 Mbits/sec 956 20.1 KBytes [ 5] 7.00-8.00 sec 10.0 MBytes 83.9 Mbits/sec 913 33.0 KBytes [ 5] 8.00-9.00 sec 9.97 MBytes 83.6 Mbits/sec 945 24.4 KBytes [ 5] 9.00-10.00 sec 9.99 MBytes 83.8 Mbits/sec 916 34.4 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 98.7 MBytes 82.8 Mbits/sec 9337 sender [ 5] 0.00-10.25 sec 98.7 MBytes 80.8 Mbits/sec receiver iperf Done. gold,pts,/home/jmg,503$iperf3 -R -c 192.168.0.80 Connecting to host 192.168.0.80, port 5201 Reverse mode, remote host 192.168.0.80 is sending [ 5] local 192.168.0.2 port 51024 connected to 192.168.0.80 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 9.69 MBytes 81.3 Mbits/sec [ 5] 1.00-2.00 sec 10.7 MBytes 89.8 Mbits/sec [ 5] 2.00-3.00 sec 10.7 MBytes 89.8 Mbits/sec [ 5] 3.00-4.00 sec 10.7 MBytes 89.8 Mbits/sec [ 5] 4.00-5.00 sec 10.7 MBytes 89.8 Mbits/sec [ 5] 5.00-6.00 sec 10.7 MBytes 89.8 Mbits/sec [ 5] 6.00-7.00 sec 10.7 MBytes 89.8 Mbits/sec [ 5] 7.00-8.00 sec 10.4 MBytes 87.6 Mbits/sec [ 5] 8.00-9.00 sec 10.7 MBytes 89.9 Mbits/sec [ 5] 9.00-10.00 sec 10.7 MBytes 89.8 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 106 MBytes 88.9 Mbits/sec 1381 sender [ 5] 0.00-10.00 sec 106 MBytes 88.7 Mbits/sec receiver iperf Done. As you can see, it matches what I measured earlier. And just to prove that the machine CAN move 100MB/sec, I've run iperf3 using the onboard wired ethernet... I need multiple interfaces, which is why I'm bothering trying to get USB3 ethernet working. This is using the onboard bge interface. It's IP is .79: gold,pts,/home/jmg,507$iperf3 -c 192.168.0.79 Connecting to host 192.168.0.79, port 5201 [ 5] local 192.168.0.2 port 61500 connected to 192.168.0.79 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 101 MBytes 850 Mbits/sec 0 488 KBytes [ 5] 1.00-2.00 sec 112 MBytes 940 Mbits/sec 0 488 KBytes [ 5] 2.00-3.00 sec 112 MBytes 943 Mbits/sec 0 731 KBytes [ 5] 3.00-4.00 sec 112 MBytes 940 Mbits/sec 0 731 KBytes [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 0 731 KBytes [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 731 KBytes [ 5] 6.00-7.00 sec 112 MBytes 940 Mbits/sec 0 731 KBytes [ 5] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 0 731 KBytes [ 5] 8.00-9.00 sec 112 MBytes 940 Mbits/sec 0 731 KBytes [ 5] 9.00-10.00 sec 112 MBytes 940 Mbits/sec 0 731 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.08 GBytes 932 Mbits/sec 0 sender [ 5] 0.00-10.01 sec 1.08 GBytes 931 Mbits/sec receiver iperf Done. gold,pts,/home/jmg,508$iperf3 -R -c 192.168.0.79 Connecting to host 192.168.0.79, port 5201 Reverse mode, remote host 192.168.0.79 is sending [ 5] local 192.168.0.2 port 61726 connected to 192.168.0.79 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 97.0 MBytes 814 Mbits/sec [ 5] 1.00-2.00 sec 109 MBytes 917 Mbits/sec [ 5] 2.00-3.00 sec 110 MBytes 920 Mbits/sec [ 5] 3.00-4.00 sec 109 MBytes 917 Mbits/sec [ 5] 4.00-5.00 sec 110 MBytes 920 Mbits/sec [ 5] 5.00-6.00 sec 109 MBytes 916 Mbits/sec [ 5] 6.00-7.00 sec 110 MBytes 919 Mbits/sec [ 5] 7.00-8.00 sec 109 MBytes 917 Mbits/sec [ 5] 8.00-9.00 sec 110 MBytes 919 Mbits/sec [ 5] 9.00-10.00 sec 109 MBytes 916 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.03 sec 1.06 GBytes 905 Mbits/sec 2866 sender [ 5] 0.00-10.00 sec 1.06 GBytes 907 Mbits/sec receiver iperf Done. > Comparing the output of using iperf3 under > NetBSD 9.0 or Windows 10 could be instructive. And NetBSD 9.0. Here, NetBSD got assigned .50, but it's also using the USB3 adapter. gold,pts,/home/jmg,505$iperf3 -c 192.168.0.50 Connecting to host 192.168.0.50, port 5201 [ 5] local 192.168.0.2 port 55535 connected to 192.168.0.50 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 96.7 MBytes 811 Mbits/sec 0 50.9 KBytes [ 5] 1.00-2.00 sec 112 MBytes 938 Mbits/sec 0 82.1 KBytes [ 5] 2.00-3.00 sec 112 MBytes 938 Mbits/sec 14 114 KBytes [ 5] 3.00-4.00 sec 110 MBytes 924 Mbits/sec 60 146 KBytes [ 5] 4.00-5.00 sec 112 MBytes 938 Mbits/sec 107 178 KBytes [ 5] 5.00-6.00 sec 112 MBytes 939 Mbits/sec 122 193 KBytes [ 5] 6.00-7.00 sec 112 MBytes 938 Mbits/sec 144 193 KBytes [ 5] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 165 193 KBytes [ 5] 8.00-9.00 sec 112 MBytes 939 Mbits/sec 144 193 KBytes [ 5] 9.00-10.00 sec 112 MBytes 938 Mbits/sec 165 193 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.08 GBytes 924 Mbits/sec 921 sender [ 5] 0.00-10.00 sec 1.08 GBytes 924 Mbits/sec receiver iperf Done. gold,pts,/home/jmg,506$iperf3 -R -c 192.168.0.50 Connecting to host 192.168.0.50, port 5201 Reverse mode, remote host 192.168.0.50 is sending [ 5] local 192.168.0.2 port 55691 connected to 192.168.0.50 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 78.3 MBytes 657 Mbits/sec [ 5] 1.00-2.00 sec 88.4 MBytes 742 Mbits/sec [ 5] 2.00-3.00 sec 88.5 MBytes 742 Mbits/sec [ 5] 3.00-4.00 sec 88.4 MBytes 741 Mbits/sec [ 5] 4.00-5.00 sec 88.7 MBytes 744 Mbits/sec [ 5] 5.00-6.00 sec 88.2 MBytes 740 Mbits/sec [ 5] 6.00-7.00 sec 87.8 MBytes 737 Mbits/sec [ 5] 7.00-8.00 sec 88.5 MBytes 742 Mbits/sec [ 5] 8.00-9.00 sec 89.0 MBytes 746 Mbits/sec [ 5] 9.00-10.00 sec 88.8 MBytes 745 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 875 MBytes 734 Mbits/sec 0 sender [ 5] 0.00-10.00 sec 874 MBytes 734 Mbits/sec receiver iperf Done. As you can see, both match approximately what I measured other methods, so, it's definitely not the way I measured performance. > My observation would be that neither type > of USB3 Ethernet adapter that I've tried What is the chipset that you tried? One of the earlier ones that I tried was an axe iirc, and was limited to around 500Mbps or so... > (different chipsets) get anywhere near > 100 MByte/s when ifconfig reports > 1000baseT . The Cwnd figures > are smaller than for the built-in Ethernets > that manage much faster overall transfer > rates. [...] > I'll note that between machines with built-in EtherNet > that can sustain fast transfers overall, the Cwnd figures > tend to vary but can reach 1 MBytes+. The Retr counts > tend to still exist. > > By contrast, when the USB3 EtherNet is receiving above, > the maximum Cwnd reported above for the sender at the > time was: 29.8 KBytes. > > I have not tried NetBSD, Windows 10, or Linux comparisons. As you can see above, NetBSD easily achieves around 8-10x the speed using the exact same USB3 device as FreeBSD does, so the hardware CAN push the speeds, just FreeBSD cannot. Hence, my original post, what can I do to possibly get FreeBSD's performance up to what the hardware can achieve? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."