From owner-freebsd-arm@freebsd.org Wed Jul 8 07:27:26 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 34DC1359B92 for ; Wed, 8 Jul 2020 07:27:26 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-20.consmr.mail.gq1.yahoo.com (sonic302-20.consmr.mail.gq1.yahoo.com [98.137.68.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B1rVP0tHFz4dxh for ; Wed, 8 Jul 2020 07:27:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: gw22JSEVM1kpL9rQ2PsGvzpULrE4FohjFwP.fF_qoO7G7nDOVLYXSd_MPBcgf2L nb4sPEM_CzPis0pZZg3PKAKLF5lGp1XA9.FfsoXuPFS8YEKze1WpTPI9JHzEUbdVArgJhy5jSitW gDkjG.AaXZCggfopJ.kO9afGKPTW_AqHW2a4wF3SYRwz5d2WIboT2mfOTuHiQUOe3xLETG8J7Dt2 bEqpbopc2v2BoSz2ZYrRmEOEN2qJgksx9vvuWy_SfvPgl4Yjuha7RES3slroh7I3aO1QjOd.SjOQ MOm6VrFGYIhgPNqQpLAP2fkaXh0raN6JWAdipJrA4xYIGHSpDzzUYncHEUrBpqHmeYsJFgUpB53x zhmKZ1rfMwIkCs3tE6ysvSYkoYzsrAiyo.AYCvmwRpMlr5u3Z.ZrkwaiUKHGkFvVGm7h17LgU1.s 7heapJPpxVlz6TUvZMMBfYx1JVwmpW6tKMUwC_XRz7vsrcj3sEod7LGtFC3F0JQeYnGA367JUuAE tNca1VUnli37pzdCOMqIcokQRUQZuWQKxY7YzkLj.OMgSspY_x3x2d4ARlUD06TcbWrU4Q5AIvnI Zaytn0TAuw6GuD.l4jmJTLRErDh6DJ60mqOfOv8Up1NiZKasIM1iby9utUH6ak1WgOXVR8iyBoTd vzrQHjAWtmfWBEzXD.ISOpUi96flDcR7ZaPBaTcG_IVmAh49MbZiKprSfMaQ5NIfUjq_NSBE09Q4 CizJkMA8QcPKDWFQoPWyxbYuLYhnFfN.WLSDAG1suBep3FS9Nb_zURQosY1vHDrB4mysmrUAWqnZ aNriP_nMLID255nEZigVOfN6wrIoBCMqV6FblqC2QRLcROJXNsHWqwbr6xoYVEIXuwmswjF2mdqe DD9RqJkqWV3XdIL6WtI2uszXeD8LrBdj0d2x9yN13.DdCkCivgtIjHtXqpVRKCT8yR9Jp_AqKC56 .dstbGKq43DQ8TRw7eDrWQ7_TpU2Y5ZPY7.DdF1pnoPElAWviOFrvJO77VhqPsmj2fdWmyJONzEz yyWPj428bwDX8_4qxsXCejOrGDehgxzAp.JTD6iGcUfqdAl9RIiKr2GnWLgis7Z_cFc9dxSzQt15 3djXCN.FvwT_EKba_._k4Q1jNWsjAijJfPUm5ppm_wqySxcX4tzZqzv4K9rOrfCLKgmXGm7ljXV3 cQLShmbpse5Te4ujF0NthiobhiVe4jf2Uh.YqZ8haV9hFRGG0zX9jtWrKKhEy8OVB4hcl4rSjH5F p0AHIBlnmgTXF4A_DZBbNeqhUzB3kny2mcl46FtHDszZEvcosNkEBZgQrnLSSbAK_bkAalAY3rzQ fpeIvh52N4FHp7w3gy_yWPAi.OEJGPlhV3v9ugUC0tQePwy.cEO.vO1mhwiDhhPxv3u5ogKUXFN1 DeHsBdTXhhCpIhacl2bQRWMAai5vInvuErwbLD3OOvzgXFA6cHIlL.TfGGA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.gq1.yahoo.com with HTTP; Wed, 8 Jul 2020 07:27:23 +0000 Received: by smtp418.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID e832bf499e19fd17610ceb450509e4cd; Wed, 08 Jul 2020 07:27:20 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: freebsd-arm Digest, Vol 740, Issue 7 (Rock64 Ethernet testing) From: Mark Millard In-Reply-To: Date: Wed, 8 Jul 2020 00:27:19 -0700 Cc: Peter Jeremy , freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <4AA1D1FD-F9A4-400E-80FA-C1888F399602@yahoo.com> References: <1731fbded28.10a3342f0357159.8148813293316485882@fkardame.com> <20200706204707.GA94158@bluezbox.com> <0A2E974E-39D3-46C8-8791-3BD914EBE7E9@yahoo.com> <0C77695E-A9D0-410A-B105-5B69823E17E2@yahoo.com> <3289DA3D-03FB-43BD-9A6A-956AC0A03B59@yahoo.com> <20200708021226.GA77884@bluezbox.com> To: Oleksandr Tymoshenko X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Rspamd-Queue-Id: 4B1rVP0tHFz4dxh X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.32 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.146:from]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.002]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.146:from]; NEURAL_HAM_SHORT(-0.85)[-0.845]; NEURAL_HAM_MEDIUM(-0.97)[-0.969]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2020 07:27:26 -0000 On 2020-Jul-7, at 21:52, Mark Millard wrote: > On 2020-Jul-7, at 19:12, Oleksandr Tymoshenko = wrote: >=20 >> Mark Millard (marklmi at yahoo.com) wrote: >>> Any chance that the delays (or other parameters) depend >>> on the operating temperature(s) of some parts? >>>=20 >>> If yes, then some of the following about the Rock64 V2 >>> context that I have access to might be relevant to >>> explaining my already reported V2 results (not much >>> for Retr): >>>=20 >>> A) The Rock64 has a "case" that is really just a top >>> and a bottom with posts: open on all 4 sides. >>>=20 >>> B) It has a fan blowing down on the board from the >>> top. >>>=20 >>> C) It has a heat sink on the SOC, which the fan blows >>> on directly. >>>=20 >>> D) It has a heat sink on the RAM, which the fan also >>> blows on directly. >>>=20 >>> (I've not dealt with a more modern non-debug kernel >>> build yet. It still may be some time before I deal >>> with that.) >>=20 >> Temperature is not likely to be a factor in the delay values. >> Rock64 V2 has a known issue with Gigabit ethernet stability: >>=20 >> https://forum.pine64.org/showthread.php?tid=3D7545 >> https://sanisimov.com/2019/08/fixing-rock64-v2-gigabit-ethernet/ >>=20 >> Althought judging by description it's more like an almost complete >> network lock-up and not performance degradation. >>=20 >> I received another board with RK3328 today and will investigate >> the issue further. >=20 > Okay. >=20 > Looks like I should have copied iperf3 output from the server > side as well: somewhat different information. The output was > still available from the earlier runs so here it is . . . >=20 > The modern debug-kernel runs: >=20 > Accepted connection from 192.168.1.109, port 47111 > [ 5] local 192.168.1.122 port 5201 connected to 192.168.1.109 port = 17015 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 17.5 MBytes 147 Mbits/sec =20= > [ 5] 1.00-2.00 sec 45.3 MBytes 380 Mbits/sec =20= > [ 5] 2.00-3.00 sec 44.9 MBytes 376 Mbits/sec =20= > [ 5] 3.00-4.00 sec 45.2 MBytes 379 Mbits/sec =20= > [ 5] 4.00-5.00 sec 44.9 MBytes 377 Mbits/sec =20= > [ 5] 5.00-6.00 sec 45.1 MBytes 379 Mbits/sec =20= > [ 5] 6.00-7.00 sec 44.5 MBytes 373 Mbits/sec =20= > [ 5] 7.00-8.00 sec 45.0 MBytes 378 Mbits/sec =20= > [ 5] 8.00-9.00 sec 44.9 MBytes 377 Mbits/sec =20= > [ 5] 9.00-10.00 sec 44.5 MBytes 373 Mbits/sec =20= > [ 5] 10.00-10.62 sec 27.9 MBytes 379 Mbits/sec =20= > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.62 sec 450 MBytes 355 Mbits/sec = receiver >=20 > Accepted connection from 192.168.1.109, port 22375 > [ 5] local 192.168.1.122 port 5201 connected to 192.168.1.109 port = 54738 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 24.7 MBytes 207 Mbits/sec 0 265 = KBytes =20 > [ 5] 1.00-2.00 sec 61.6 MBytes 517 Mbits/sec 4 211 = KBytes =20 > [ 5] 2.00-3.00 sec 61.4 MBytes 515 Mbits/sec 1 352 = KBytes =20 > [ 5] 3.00-4.00 sec 61.3 MBytes 514 Mbits/sec 4 269 = KBytes =20 > [ 5] 4.00-5.00 sec 61.4 MBytes 515 Mbits/sec 2 355 = KBytes =20 > [ 5] 5.00-6.00 sec 61.3 MBytes 514 Mbits/sec 3 304 = KBytes =20 > [ 5] 6.00-7.00 sec 61.3 MBytes 514 Mbits/sec 2 327 = KBytes =20 > [ 5] 7.00-8.00 sec 61.4 MBytes 515 Mbits/sec 5 278 = KBytes =20 > [ 5] 8.00-9.00 sec 61.4 MBytes 515 Mbits/sec 2 393 = KBytes =20 > [ 5] 9.00-10.00 sec 61.4 MBytes 515 Mbits/sec 3 284 = KBytes =20 > [ 5] 10.00-10.61 sec 37.3 MBytes 514 Mbits/sec 2 282 = KBytes =20 > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.61 sec 614 MBytes 486 Mbits/sec 28 = sender >=20 > (So a fairly consistent Retr rate.) >=20 > The non-debug head -r360311 kernel runs: >=20 > Accepted connection from 192.168.1.109, port 46431 > [ 5] local 192.168.1.122 port 5201 connected to 192.168.1.109 port = 39541 > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.00 sec 50.3 MBytes 422 Mbits/sec =20= > [ 5] 1.00-2.00 sec 72.7 MBytes 610 Mbits/sec =20= > [ 5] 2.00-3.00 sec 72.9 MBytes 611 Mbits/sec =20= > [ 5] 3.00-4.00 sec 72.8 MBytes 611 Mbits/sec =20= > [ 5] 4.00-5.00 sec 72.9 MBytes 611 Mbits/sec =20= > [ 5] 5.00-6.00 sec 72.9 MBytes 611 Mbits/sec =20= > [ 5] 6.00-7.00 sec 72.8 MBytes 611 Mbits/sec =20= > [ 5] 7.00-8.00 sec 72.8 MBytes 610 Mbits/sec =20= > [ 5] 8.00-9.00 sec 72.8 MBytes 610 Mbits/sec =20= > [ 5] 9.00-10.00 sec 72.8 MBytes 610 Mbits/sec =20= > [ 5] 10.00-10.32 sec 23.4 MBytes 612 Mbits/sec =20= > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate > [ 5] 0.00-10.32 sec 729 MBytes 593 Mbits/sec = receiver >=20 > Accepted connection from 192.168.1.109, port 40223 > [ 5] local 192.168.1.122 port 5201 connected to 192.168.1.109 port = 50696 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 78.5 MBytes 659 Mbits/sec 0 480 = KBytes =20 > [ 5] 1.00-2.00 sec 113 MBytes 945 Mbits/sec 0 747 = KBytes =20 > [ 5] 2.00-3.00 sec 112 MBytes 941 Mbits/sec 0 940 = KBytes =20 > [ 5] 3.00-4.00 sec 84.5 MBytes 709 Mbits/sec 52 368 = KBytes =20 > [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 0 681 = KBytes =20 > [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 889 = KBytes =20 > [ 5] 6.00-7.00 sec 112 MBytes 942 Mbits/sec 0 1.03 = MBytes =20 > [ 5] 7.00-8.00 sec 112 MBytes 942 Mbits/sec 0 1.17 = MBytes =20 > [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 0 125 = KBytes =20 > [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 3 586 = KBytes =20 > [ 5] 10.00-10.31 sec 34.7 MBytes 942 Mbits/sec 0 667 = KBytes =20 > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.31 sec 1.07 GBytes 892 Mbits/sec 55 = sender >=20 > So: Larger total Retr than the modern debug kernel case > but not a fairly consistent rate of Retr values. >=20 >=20 > I've still not dealt with updating to a modern non-debug > environment to test it. >=20 Turns out that I get non-zero Retr values between any two machines that I try, even when both are not arm at all. For example, between a PowerMac G5 (2 sockets, 2 cores each, powerpc64) and a Threadripper 1950X (amd64), Both are Gbps=20 capable . . . FBSDG5L2# iperf3 -c 192.168.1.120 Connecting to host 192.168.1.120, port 5201 [ 5] local 192.168.1.7 port 17239 connected to 192.168.1.120 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 113 MBytes 946 Mbits/sec 0 489 KBytes = =20 [ 5] 1.00-2.00 sec 113 MBytes 944 Mbits/sec 0 730 KBytes = =20 [ 5] 2.00-3.00 sec 113 MBytes 945 Mbits/sec 48 1.03 MBytes = =20 [ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec 95 52.8 KBytes = =20 [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 95 21.4 KBytes = =20 [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 96 1.04 MBytes = =20 [ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 91 148 KBytes = =20 [ 5] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 92 388 KBytes = =20 [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 99 792 KBytes = =20 [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 93 1.07 MBytes = =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 943 Mbits/sec 709 = sender [ 5] 0.00-10.79 sec 1.10 GBytes 874 Mbits/sec = receiver Accepted connection from 192.168.1.120, port 24400 [ 5] local 192.168.1.7 port 5201 connected to 192.168.1.120 port 14839 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 109 MBytes 914 Mbits/sec 84 668 KBytes = =20 [ 5] 1.00-2.00 sec 112 MBytes 942 Mbits/sec 102 942 KBytes = =20 [ 5] 2.00-3.00 sec 112 MBytes 942 Mbits/sec 101 603 KBytes = =20 [ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec 95 868 KBytes = =20 [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 92 135 KBytes = =20 [ 5] 5.00-6.00 sec 112 MBytes 942 Mbits/sec 89 989 KBytes = =20 [ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 95 28.5 KBytes = =20 [ 5] 7.00-8.00 sec 112 MBytes 942 Mbits/sec 100 158 KBytes = =20 [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 102 20.0 KBytes = =20 [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 99 335 KBytes = =20 [ 5] 10.00-10.00 sec 7.50 KBytes 525 Mbits/sec 0 344 KBytes = =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec 959 = sender Accepted connection from 192.168.1.7, port 30451 [ 5] local 192.168.1.120 port 5201 connected to 192.168.1.7 port 28078 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 38.8 MBytes 326 Mbits/sec 36 1.61 MBytes = =20 [ 5] 1.00-2.01 sec 80.6 MBytes 667 Mbits/sec 471 817 KBytes = =20 [ 5] 2.01-3.00 sec 106 MBytes 901 Mbits/sec 114 688 KBytes = =20 [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 893 KBytes = =20 [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 2 523 KBytes = =20 [ 5] 5.00-6.00 sec 111 MBytes 932 Mbits/sec 0 774 KBytes = =20 [ 5] 6.00-7.00 sec 108 MBytes 903 Mbits/sec 361 551 KBytes = =20 [ 5] 7.00-8.00 sec 96.3 MBytes 808 Mbits/sec 180 473 KBytes = =20 [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 0 740 KBytes = =20 [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 0 934 KBytes = =20 [ 5] 10.00-10.67 sec 74.1 MBytes 933 Mbits/sec 1 456 KBytes = =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.67 sec 1.03 GBytes 833 Mbits/sec 1165 = sender Connecting to host 192.168.1.7, port 5201 [ 5] local 192.168.1.120 port 60431 connected to 192.168.1.7 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 97.8 MBytes 818 Mbits/sec 0 633 KBytes = =20 [ 5] 1.00-2.00 sec 111 MBytes 934 Mbits/sec 128 649 KBytes = =20 [ 5] 2.00-3.00 sec 112 MBytes 937 Mbits/sec 0 863 KBytes = =20 [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 2 394 KBytes = =20 [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 0 693 KBytes = =20 [ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec 0 897 KBytes = =20 [ 5] 6.00-7.00 sec 111 MBytes 932 Mbits/sec 2 480 KBytes = =20 [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 0 746 KBytes = =20 [ 5] 8.00-9.00 sec 62.3 MBytes 523 Mbits/sec 320 436 KBytes = =20 [ 5] 9.00-10.00 sec 111 MBytes 934 Mbits/sec 0 717 KBytes = =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.03 GBytes 881 Mbits/sec 452 = sender [ 5] 0.00-10.01 sec 1.02 GBytes 880 Mbits/sec = receiver I may need to establish a better context or just may be limited to Bitrate comparisons instead of looking for Retr staying zero. (Still not at a modern non-debug build for the Rock64.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)