From owner-freebsd-arm@freebsd.org Sat Sep 19 03:41:22 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E56063F90F3 for ; Sat, 19 Sep 2020 03:41:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Btc1s4cCrz4X5W for ; Sat, 19 Sep 2020 03:41:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: CkGtj5QVM1lqw5jDY6YJAZPofr5v0v7IMO.Eo2sSwggZ_uqpDqxEOWM3qHSpKII myL2AaBoW1b.byceaFz565abKcJBs2vB.zn9bXh0lfhJv7gxHC3.m9._ezy6hFPIteqdM.LvgGSU 3F8Xv15uufWiDReRmk_1AmeqWCbh1bAOMSzn1I0ICUI61B6BvGarswzDJmIfO0WBK3KBvymRBXAt Gqur4mHScfpQosN5X1trxFa2WHs25v_37gCLqcoYnzpdiz9_NNbicIo89snkCS8DjFyfvf6x.fcR Y1zHnT8bpSpD5SldRwymN904PoRjKzaWXkButmZhTO0VUsgcZTWUWQjDoFdjE9y.LE30v4iuu4uj QwdPa7qtBOtr5qJ3L.W.dAWqA6hBaBWy1H9K0xbDx2T0TSk2Aka3W.fiu4kg9x1EhRRdsgAhgaPG Qb3diYZpwwd2mPmV0he8zYAfXKndrzbpVpiw1AOS1QnX4zNFCu6maP5NHh_M0HnosziI_I6H4JXv xccRiwySv4QEzW9UbdwBE4Z3J5v_fplgGFvi8CjxVXhWdXWFU.gHrOkkq7R_8IxTz2YHwvsofs1F .Im_u7X9dg_yjQpsvdk3gRzb.9Utu6W0Bsqr9uQJ.vmFDs.muKWUEax0AKIUF3BIRPfqnbWD1VwH qYniIiCunPwpp.3Y6tXoYuzn3iMKj_mBfoRYTRcwtxuqDdLGx7oj3rYY2nDJTBcOO32uLd6QCl7g mzVwHTxjUmDr_JWmh9tK6vcBXbM41fr5PNu3dXHPTofSExdJaC6ppW0ZDerGrOqASMeWGXcTTump KLAbjz.BxISlYmfgZryblE2RnRGZwSDI68gKCVBALm3wRV2_yxNpP2vxFESIGX.IRDEZPzByW6Sx luOLUp7O7xSZTQ3B6w_xb6D3lQpoMlE2WUZLgsskKAV5.IMtBTCtazxZNGuZ9AcOCmmPT7PBO5QL njtYJuNr9tfk3QBDldixmZFBGpydSU_xR_.Rhd4NzS.G.xhP4hBYxo5jTk_Waeyz7ry28wwE60sB mNqNi5RME88j7mTG.5iIwu9BBm.y4kYuI.hi1OHikdE1.7t18zXxipGv8XGGs_s14I.lCd88sYy5 5zXj29zeDUoRFwOVjIOgu2JaCcSxBGGcjMjfsXztxVEHuGBw2YJJXwhr2REDQnpXGCE5Fjlx8R.z _K4ReJ1LCor3GE5y6evKB1zOO6ZOZRKMTi_S7W693xi_VVvWz2UrnItOyHIp0ETEhi6eOao28LdD dYUrqvcfUWr.8vnOuz_t5J1qGG1gyOH6bnQklu5e4HsigIQO3dvKb17PUN1qeTciKakYL7dg0b9g CNJwDid.Uys7CZzkfLqq3ouXTqVCb81QCwG55e24PX7zvW0gxFg.NjnUKPPb9YCGvseISRbIF6sY xLUcj9TrZKZmtnkbjX5zwfLSaHUuXHZkCHPVctcoWcg6D0XzURz7V3fbE5D2F_KzDjHRed1ej0EI GuRqHtRE6pc8o5Ff_fBI6zTAeVoG2u9.DJDj2TUL3OX6HKipFsaHFUI_RvW0mQL7HTrnkQxqLls0 TbGBY_yWAKtyGvw-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Sep 2020 03:41:19 +0000 Received: by smtp420.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b42748159b4324e5795b5165d20adb1f; Sat, 19 Sep 2020 03:41:14 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Subject: Comment #135 for bugzilla 237666 : a USB3-handling problem with a investigatory fix for a cortex-a72 context Message-Id: <6E618C3D-12DF-429E-A249-5BAB90FC6B15@yahoo.com> Date: Fri, 18 Sep 2020 20:41:14 -0700 To: freebsd-arm X-Mailer: Apple Mail (2.3608.120.23.2.1) References: <6E618C3D-12DF-429E-A249-5BAB90FC6B15.ref@yahoo.com> X-Rspamd-Queue-Id: 4Btc1s4cCrz4X5W X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.46 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.96)[-0.959]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.980]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.016]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.32:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.32:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Sep 2020 03:41:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237666 is about USB3 problems that folks have been having for over a year. Bjoern A. Zeeb comment #125 that looked like something I'd seen on a cortex-a72 when I tried a kernel that had been built with -mcpu=cortex-a72 : an indefinite looping that involved "Resetting controller" for xhci0. (This prevented booting from the USB3 device.) Well, I got the A72 to boot and comment #135 indicates how but I'll paste the information below. All that was involved was adding examples of "dsb st" and one "dsb ld". Comment 135's content: A cortex-a72 success! (In the form of investigatory code, not code to check-in as is.) Just by adding some "dsb st" commands and a "dsb ld" command in places in the xhci code (just for __arach64__), I've gotten the cortext-A72 to boot from the USB3 SSD via a -mcpu=cortex-a72 based kernel build. No more indefinitely repeating "Resetting controller" problem. I do not claim the additions are the minimal ones. I know one place is required for sure: the last one added enabled the boot (given the others were already present). Prior to that I still had the indefinitely repeating "Resetting controller" problem. There could be more places that should have dsb st or dsb ld for overall correctness. This evidence may suggest somewhat analogous problems for other platforms than aarch64 that are seeing problems. For the aarch64 context, the evidence is (the changes are) as follows. The first "dsb st" textually is the last one I added. # svnlite diff /usr/src/sys/dev/usb/controller/xhci.c Index: /usr/src/sys/dev/usb/controller/xhci.c =================================================================== --- /usr/src/sys/dev/usb/controller/xhci.c (revision 363590) +++ /usr/src/sys/dev/usb/controller/xhci.c (working copy) @@ -431,6 +431,9 @@ phwr->hwr_ring_seg[0].qwEvrsTablePtr = htole64(addr); phwr->hwr_ring_seg[0].dwEvrsTableSize = htole32(XHCI_MAX_EVENTS); +#ifdef __aarch64__ +__asm __volatile("dsb st" : : : "memory"); +#endif DPRINTF("ERDP(0)=0x%016llx\n", (unsigned long long)addr); @@ -1098,6 +1101,9 @@ while (1) { +#ifdef __aarch64__ +__asm __volatile("dsb ld" : : : "memory"); +#endif temp = le32toh(phwr->hwr_events[i].dwTrb3); k = (temp & XHCI_TRB_3_CYCLE_BIT) ? 1 : 0; @@ -1107,6 +1113,9 @@ event = XHCI_TRB_3_TYPE_GET(temp); +#ifdef __aarch64__ +__asm __volatile("dsb ld" : : : "memory"); +#endif DPRINTFN(10, "event[%u] = %u (0x%016llx 0x%08lx 0x%08lx)\n", i, event, (long long)le64toh(phwr->hwr_events[i].qwTrb0), (long)le32toh(phwr->hwr_events[i].dwTrb2), @@ -1239,6 +1248,9 @@ sc->sc_command_idx = i; sc->sc_command_ccs = j; +#ifdef __aarch64__ +__asm __volatile("dsb st" : : : "memory"); +#endif XWRITE4(sc, door, XHCI_DOORBELL(0), 0); err = cv_timedwait(&sc->sc_cmd_cv, &sc->sc_bus.bus_mtx, @@ -2855,6 +2867,9 @@ index = xfer->xroot->udev->controller_slot_id; if (xfer->xroot->udev->flags.self_suspended == 0) { +#ifdef __aarch64__ +__asm __volatile("dsb st" : : : "memory"); +#endif XWRITE4(sc, door, XHCI_DOORBELL(index), epno | XHCI_DB_SID_SET(xfer->stream_id)); } @@ -4180,6 +4195,9 @@ for (n = 1; n != XHCI_MAX_ENDPOINTS; n++) { for (p = 0; p != XHCI_MAX_STREAMS; p++) { +#ifdef __aarch64__ +__asm __volatile("dsb st" : : : "memory"); +#endif XWRITE4(sc, door, XHCI_DOORBELL(index), n | XHCI_DB_SID_SET(p)); } I'm clearly just "evidence hacking" here. I've no clue what a good design for allowing aarch64 build to supply having any of those "dsb st"s or the "dsb ld". Hopefully someone that knows what they are doing can figure out if any of the other examples on other platforms are analogous --and ,if they are, provide some hook for platform to contribute such code to the xhci code. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)