From owner-freebsd-arm@freebsd.org Sun Jul 19 02:18:43 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 560D137AE91 for ; Sun, 19 Jul 2020 02:18:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B8T755Zsdz43Q9 for ; Sun, 19 Jul 2020 02:18:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: APplh04VM1l4K3r.GxcL2HXkEWPmCjxgJS92JHGffmos7Dw7O5TanLO7oOcgTRm gnzCHC_YPVLYWKISl9mMMD6mr3tCObAj3GBEgssn2.5TxM.ADDaFICY22sB5CT92ohFXUY54N2Xv QJa1Xw5WA6.CAONG58SLcOe.fkOFWhvLPK4v1RAJJ6nQsk1rSO4MmkjQxhCBwn1_UeosfGF.mpDg hPfcLMwxm97uLqQ7Zyyf2bilbM6kFxDWgzcb3e_7zlSB_KRymek6.82wjlRXHCOp0gRH3HpUJWcS QQJC2nrX26dBtE6A.twsKcNXt0ngUcNcLDzhMb2zqQ6.dN63IKuABgmv41e5.GgthMMyfVfiyxwg KyNAiDntMkrUP3S2wlSNCNwgW14GRS2chmt5Fcr7oOyk2zK_AO2WBOG_kPR8i5vO7rIZkfBRvAKk WBYmHrK79w18fZjXZd9bVtSflJnoiXrfhpkoJFEO4E0cltHTHYe2Xv1ozzJnKVhcWYdZ_pVHrVS2 zqincRG.U_fEO1OkNbJ9r0a.D4tTM_G9F_eWwx7rDGnif0s3zzLwLDAw1lbMrJRzV3BAHRs_ET4J .FJ9aMUQgk0sPAn2G5PsaKi2npBu_ymRPr7SAHdUZvwF1cODYYNc8BmR9MryrsD67gDPwRa919Ih 6hWONQtLVzVqCdFEv1DoZiqqVMMBbdk6IdDeBP0zSl6DGKpf2QafatsJO8Qvv4AXwO2IltWHogYp v90aZi_LiOz5DRb5QHz5vvjqpEzECWwpU2fbVvxRfqJrKQVoe3t2zyg3sHI4Roj4h4XKVLC5KeTi 70I3ltjJdZIJn6E3jl9rBAR53xvmi7naUU0Ln0duUq.QFbrxLhdi6ZXNg8ZSORM0j8rpvFcey07z MUCtaWa1e0nv_qNVScZFologFHLKxbdf2SvIboYTrJfMOYD97i8e9NldLBl_0LUrabgpWWhLkkp9 8Cd0U8b41mu18ZXhLG0.ipx3Nd4QASo2GGbCYzTPP9vc3jkoexXB9I1qhB2viKmzA.J.d.J.aPGR iKooi5OejtLrXy2sSa5FbNNgJ58ub4.dZ7l8Z9efhIssFDeqIIqjF_C7fTkn_hm71BGyyVpYgyu8 OIJjtI8mZ1f_SL7WpmjNIO7oZwWY_rZ.OaB.UMddf_T4XnEHtIpqPiUplh29vnHrskj632VRg6vX Wv69lOaViaxp9oFrYsJEKmQrEZgKSdWn1oaY4DaiLS2QB69RTghewjlNmrEOaZyCoJqFcHmmAUTs ZS36pfrsSnzwft16FofFWhOFpZBrHHtr4V2c61qnq6DLmAGiQLl2kv1RctewJc_mdZDDiTbDAw72 VQsHCI88pTx13NCGyjuINMs9p6Z7pgJjVv_DhX6Nv5Tu8RBVstSbHM.6pH7sxxElkJothgAdPsyI .tu11cfqLYwGM7rj3T_2p7eyntvwBJ5bd_BZVYDp5nZrI01BYhmSFP9_KJ8M- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sun, 19 Jul 2020 02:18:40 +0000 Received: by smtp428.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 28a90e4c2536469b905325cbc613bac4; Sun, 19 Jul 2020 02:18:38 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: FYI: RPi4 (8 GiByte) USB3 vs. head -r363123: still a no-go for booting a USB3 / in my experiments From: Mark Millard In-Reply-To: Date: Sat, 18 Jul 2020 19:18:37 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <64A12BF8-9E2D-441D-8765-52B6D3907A58@yahoo.com> References: To: Robert Crowston X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Rspamd-Queue-Id: 4B8T755Zsdz43Q9 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.14 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.62)[-0.617]; FREEMAIL_TO(0.00)[protonmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.030]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.995]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.84:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.84:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jul 2020 02:18:43 -0000 > On 2020-Jul-18, at 14:37, Robert Crowston = wrote: >=20 > I believe this differential (https://reviews.freebsd.org/D25261) would = resolve it, but I haven't got around to addressing the comments there = yet. >=20 > -- RHC. Yep, a -r363123 kernel with that also it booted / from the USB3 SSD. (The kernel came from the mmcsd0.) Notes for this u-boot-rpi4 based context: Unlike for uefi v1.17, genet0 and mmcsd0 show up. There is a big asymmetry for sending vs. receiving over genet0 (192.168.1.126 here): # iperf3 -c 192.168.1.120 --get-server-output -B 192.168.1.126 Connecting to host 192.168.1.120, port 5201 [ 5] local 192.168.1.126 port 56649 connected to 192.168.1.120 port = 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 22.7 MBytes 191 Mbits/sec 85 38.4 KBytes = =20 [ 5] 1.00-2.00 sec 23.4 MBytes 196 Mbits/sec 146 25.7 KBytes = =20 [ 5] 2.00-3.00 sec 23.2 MBytes 195 Mbits/sec 226 27.1 KBytes = =20 [ 5] 3.00-4.00 sec 23.3 MBytes 195 Mbits/sec 229 95.0 KBytes = =20 [ 5] 4.00-5.00 sec 23.6 MBytes 198 Mbits/sec 228 83.7 KBytes = =20 [ 5] 5.00-6.00 sec 23.7 MBytes 199 Mbits/sec 209 7.13 KBytes = =20 [ 5] 6.00-7.00 sec 23.7 MBytes 199 Mbits/sec 207 49.8 KBytes = =20 [ 5] 7.00-8.00 sec 23.6 MBytes 198 Mbits/sec 217 1.43 KBytes = =20 [ 5] 8.00-9.00 sec 23.6 MBytes 198 Mbits/sec 224 71.3 KBytes = =20 [ 5] 9.00-10.00 sec 23.5 MBytes 197 Mbits/sec 242 8.55 KBytes = =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 234 MBytes 197 Mbits/sec 2013 = sender [ 5] 0.00-10.27 sec 234 MBytes 191 Mbits/sec = receiver Server output: ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.1.126, port 57882 [ 5] local 192.168.1.120 port 5201 connected to 192.168.1.126 port = 56649 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 16.4 MBytes 138 Mbits/sec =20 [ 5] 1.00-2.00 sec 23.4 MBytes 196 Mbits/sec =20 [ 5] 2.00-3.00 sec 23.2 MBytes 195 Mbits/sec =20 [ 5] 3.00-4.00 sec 23.1 MBytes 194 Mbits/sec =20 [ 5] 4.00-5.00 sec 23.6 MBytes 198 Mbits/sec =20 [ 5] 5.00-6.00 sec 23.7 MBytes 199 Mbits/sec =20 [ 5] 6.00-7.00 sec 23.7 MBytes 199 Mbits/sec =20 [ 5] 7.00-8.00 sec 23.6 MBytes 198 Mbits/sec =20 [ 5] 8.00-9.00 sec 23.6 MBytes 198 Mbits/sec =20 [ 5] 9.00-10.00 sec 23.5 MBytes 197 Mbits/sec =20 [ 5] 10.00-10.27 sec 6.30 MBytes 197 Mbits/sec =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.27 sec 234 MBytes 191 Mbits/sec = receiver iperf Done. Rock64orRPi4# iperf3 -R -c 192.168.1.120 --get-server-output -B = 192.168.1.126 Connecting to host 192.168.1.120, port 5201 Reverse mode, remote host 192.168.1.120 is sending [ 5] local 192.168.1.126 port 25404 connected to 192.168.1.120 port = 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec =20 [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.26 sec 1.09 GBytes 910 Mbits/sec 923 = sender [ 5] 0.00-10.00 sec 1.09 GBytes 933 Mbits/sec = receiver Server output: ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 192.168.1.126, port 41483 [ 5] local 192.168.1.120 port 5201 connected to 192.168.1.126 port = 25404 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 83.7 MBytes 702 Mbits/sec 62 1.61 MBytes = =20 [ 5] 1.00-2.00 sec 111 MBytes 932 Mbits/sec 92 1.61 MBytes = =20 [ 5] 2.00-3.00 sec 111 MBytes 934 Mbits/sec 93 175 KBytes = =20 [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 95 88.4 KBytes = =20 [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 94 268 KBytes = =20 [ 5] 5.00-6.00 sec 111 MBytes 932 Mbits/sec 89 291 KBytes = =20 [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 92 4.28 KBytes = =20 [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 92 1.27 MBytes = =20 [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 94 1.61 MBytes = =20 [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 96 1.42 MBytes = =20 [ 5] 10.00-10.26 sec 29.3 MBytes 931 Mbits/sec 24 1.61 MBytes = =20 - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.26 sec 1.09 GBytes 910 Mbits/sec 923 = sender iperf Done. My attempt to duplicate an about 11 GiByte .tar file failed: # cp -aRx /usr/obj/clang-armv7-on-aarch64.tar /usr/obj/mmjnk.tar g_vfs_done():gpt/Rock64root[READ(offset=3D274911461376, = length=3D32768)]error =3D 5 c/usr/obj/clang-armv7-on-UFS /dev/gpt/Rock64root (/) cylinder checksum = failed: cg 270, cgp: 0x0 !=3D bp: 0xa023cccf aarch64.tar: Input/output error UFS /dev/gpt/Rock64root (/) cylinder checksum failed: cg 270, cgp: 0x0 = !=3D bp: 0xa023cccf pid 1123 (ntpd), jid 0, uid 0: exited on signal 6 (core dumped) pid 1240 (login), jid 0, uid 0: exited on signal 11 (core dumped) Same media as used for Rock64 experiments. I've had no evidence of problems there. However uefi based booting the RPi4 also has problems unless uefi is set to force 3 GiBytes of RAM (at most). The problem has usually been visible as the diff after the copy showing 4 KiByte or slightly less having differences according to diff or cmp or the like. There is evidence of the content and where changing without updates to the files, suggesting garbage read data. Again, same media for 3GiByte RAM or on the Rock64: no evidence of problems for such operations. My initial guess is that the handling of the RPi4 DMA limitations is not yet correct overall, apparently for both uefi and u-boot style booting. > =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 = Original Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2= =80=90 > On Wednesday, 15 July 2020 10:09, Mark Millard via freebsd-arm = wrote: >=20 >> I did the following experiment mostly just to observe >> the current status for sysutils/u-boot-rpi4 based >> booting of the RPi4 (with rather modern RPi4 firmware >> in use). (I normally use uefi/acpi instead of u-boot, >> uefi now at v1.17 . I was hoping to see if u-boot based >> also had a bug that uefi contexts have.) >>=20 >> With the kernel on the microsd card (and earlier stage >> materials), boot -v reported (before mounting / from >> USB3 became relevant): >>=20 >> pci1: on pcib1 >> pcib1: allocated bus range (1-1) for rid 0 of pci1 >> pci1: domain=3D0, physical bus=3D1 >> found-> vendor=3D0x1106, dev=3D0x3483, revid=3D0x01 >>=20 >> domain=3D0, bus=3D1, slot=3D0, func=3D0 >> class=3D0c-03-30, hdrtype=3D0x00, mfdev=3D0 >> cmdreg=3D0x0000, statreg=3D0x0010, cachelnsz=3D0 (dwords) >> lattimer=3D0x00 (0 ns), mingnt=3D0x00 (0 ns), maxlat=3D0x00 (0 = ns) >> intpin=3Da, irq=3D0 >> powerspec 3 supports D0 D3 current D0 >> MSI supports 4 messages, 64 bit >> map[10]: type Memory, range 64, base 0, size 12, memory = disabled >>=20 >>=20 >> pcib1: slot 0 INTA is routed to irq 82 >> xhci0: irq 82 at device 0.0 on = pci1 >> pcib1: allocated memory range (0xf8000000-0xf8000fff) for rid 10 of = xhci0 >> xhci0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xf8000000 >> xhci0: 32 bytes context size, 64-bit DMA >> xhci0: attempting to allocate 1 MSI vectors (4 supported) >> xhci0: using IRQ 83 for MSI >> xhci0: MSI enabled >> xhci0: Controller reset timeout. >> xhci0: XHCI halt/start/probe failed err=3D18 >> xhci0: Controller reset timeout. >> device_attach: xhci0 attach returned 6 >> . . . >> simplebus2: xhci@7e9c0000 mem 0x7e9c0000-0x7eabffff irq 78 disabled = compat generic-xhci (no driver attached) >>=20 >> So the USB3 ends up unavailable. >>=20 >> / would have been from a USB3 SSD if things had worked. >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)