From owner-freebsd-arm@freebsd.org Wed Feb 12 02:18:08 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 16317249EEF for ; Wed, 12 Feb 2020 02:18:08 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-25.consmr.mail.gq1.yahoo.com (sonic311-25.consmr.mail.gq1.yahoo.com [98.137.65.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48HNbL67q7z3F9d for ; Wed, 12 Feb 2020 02:18:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: fGxmRzMVM1nkU7Sjk_4GnP8bMho.DbXT6TcwXdevn2rLzWFJmgPKkczEiUkyx72 YpEe8vVlzudH.LeLL6L_6rae_koNrRWj41gulFSri.9N_CEAgOCOwg_QCxgLJN.XMqzWeVfOW7p2 .Oh3s7AzB6hgz22eBOyJEDlITV6pxfibzgD5q89ABhsFM7D925x4HNRwY35nSVvjp0Zv8PbY0lJQ jz8HLa9hIplLbZGugvwR5bbOlAxQYZNRNV_BAh2uAMRINF.U_Bi1mpbsA9ZkhidIVpJw2riu9Yez OiQjl423k_T_pw_YLBn73KbOoBhChMeAPkyWd00MzP_YQI5ahJ1BfyAT.UI4DK70gqh1Y5n9MDET OAuGX3hdJjLGULt49navWZBiCk63TDn2imnjG_TgnmLECsJ7lQk30htK2gRnmU1HkavpBsyyexUw UvvaKJosRxKiPskhPZUQzOWPhhJMLUXpIrYE3M6Letk2SGdWqTd230Y0mG30m.GD5QGO28CiNNJT 8HEXLa2LfjKVYqgH36QedtGrmWI10pwUg7sYEJbcr6NtIm6gAPqkf2yG_JrQIxUi_7F3WCJOGSLq LcYy8U2xIz54DUAkQRGj_V2VYH3n8L6z6rdKDT85omueBiJE7_DtyJwXkOA2LdKGS6GwrCWPr4cq IQFbiIRB8hx.kXgPLcUouYjpueUjOcZSclisvcdTZmim0sRKJD5DQMI05HDze5uHwlSNqUPDNABg k1GNcJCuGyyWTtdx0tct.Y3GFgJ_rDBc2AbJ5QnnMs4oaDmUI4W5WvPtfcvZUbcsosxILyD2amGd ZFF.MRnpFCfI6Eo2xuFrI.4mfoyb1BZ4dDTlpvL4tHY6G_HNFFRYwtPulJ.AZv5s48I2BJG07Nwj todv_F5DE2dgQ0MN..4gMTqDr9rNFbh5AcgLXNicooO1lx3r5rpZs7ViGYMDKyqtfM94s3HQOPLC sKILWLmxvr7EuHx1uJC3kk30B2FBrtfxpGkh7GEgYuoxxN6Z62Hny7mj.D4rasBTtZxENgT9ZBdn g18eSI7WOpG5RaDxLq_18eL7iQhGz2ZbrfMxlL1n79jhyCQQ_Yp8lMiOhNFgY8eFuhEhlx3xK3YT sNBWGs8V6esgaINXG3rjCnoFuMoMClp6iSJmccCCdK0lBzY8EBNXtWLDg_23p24C8sevd8TASycm j.RjuS6uybAa4517tI84BLIQkg_.6f3IWY7HwftDcS533FZtgvLBVizrs5vSiRoAUc3y1hUn9G22 RfNJCK5KJ1WKdv4eOpuzODKM1gKjYmGC6e9x.aN_RR3vML.9iHHHbFyuKExc1z7cqG9JFgaecgUO OYGaGV4sFaQ.ki98P6QntJGFqxEHxwczIcYv_JVfx7oMjDvewwPCrz_n9K8qFjrjhO17LD9RFg._ safcz5hvBg7dl3xLLpf_eqiUh4bc_KgUSQRmrZBZ8bE81pCgekmeKYh_GLgsyqQYOwVtVLMJFjQ- - Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Wed, 12 Feb 2020 02:18:04 +0000 Received: by smtp404.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID a3c56076a7c923d8c7a5300c175298eb; Wed, 12 Feb 2020 02:17:59 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: FreeBSD's armstub8-gic.bin or armstub8.bin is bigger than the reserve area is it loaded to; later part ends up overwritten Message-Id: Date: Tue, 11 Feb 2020 18:17:58 -0800 Cc: Andrew Turner , Emmanuel Vadot To: freebsd-arm , Kyle Evans X-Mailer: Apple Mail (2.3608.60.0.2.5) References: X-Rspamd-Queue-Id: 48HNbL67q7z3F9d X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[206.65.137.98.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ip: (-5.26), ipnet: 98.137.64.0/21(0.83), asn: 36647(0.66), country: US(-0.05)]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Feb 2020 02:18:08 -0000 Using FreeBSD's armstub8-gic.bin as the example that applies to my test context: It has something like 0x15bc bytes to the end of its last instruction. (I looked at the .elf from a build that I did.) Doing an experiment to make the PSCI version request also report where it sees its _start, el3_exception_vectors, and loop addresses showed (my summary): _start at 0x0 el3_exception_vectors at 0x800 loop at 0x100c (given my code additions that made the size 0x15c8 instead of 0x15bc). So: loaded to Address 0x0. I confirmed in ddb that I see the code sequence starting at 0x0. In my context boot -v was reporting the below. Compare and contrast the reserved size starting at 0x0 to the size of armstub8-gic's code span: Type Physical Virtual #Pages Attr Reserved 000000000000 0 00000001 WB=20 ConventionalMemory 000000001000 1000 00007ef1 WB=20 . . . Physical memory chunk(s): 0x00001000 - 0x39f48fff, 927 MB ( 237384 pages) 0x39f50000 - 0x39f50fff, 0 MB ( 1 pages) . . . Excluded memory regions: 0x00000000 - 0x00000fff, 0 MB ( 1 pages) NoAlloc=20 0x32000000 - 0x33773fff, 23 MB ( 6004 pages) NoAlloc=20 Using ddb I confirmed that starting at 0x1000 armstub8-gic.bin 's content had been replaced. That includes swap32, memmove, and fixup_dt_blob. It looks to me like fixup_dt_blob needs to take into account the span of the armstub8* material and reserve at least 1 more page (past the page holding its self-observed start address) than is now reserved. (Wording avoids assuming that all aarch64 RPi*'s would use address 0x0.) I do not claim to know that such a change would be sufficient. But the above seems to be a fragile structure. Detailed evidence techniques: I adjusted a copy of sysutils/rpi-firmware 's armstub8 code to have the pSCI version request also fill in the address of _start, el3_exception_vectors, and loop: psci_version: /* Return v0.2 */ adr x3, loop adr x2, el3_exception_vectors adr x1, _start mov x0, #0x00000002 eret I then added a direct use of arm_smccc_smc in a place early enough that I'd never seen the resulting version number to be wrong: uma_startup2(); =20 //HACK!!! #ifdef __aarch64__ struct arm_smccc_res result; int psci_version=3D = arm_smccc_smc(0x84000000,0,0,0,0,0,0,0,&result); printf("psci_version+: %x %lx %lx %lx %lx\n", psci_version, = result.a0, result.a1, result.a2, result.a3); #endif } I added the printf in order find what address range over which the armstub80-gic code would see itself span for my RPi4B based test context. The result was: psci_version+: 2 2 0 800 100c So: (minor) version 2 (twice) _start at 0x0 el3_exception_vectors at 0x800 loop at 0x100c (given my code additions). I'll note that this investigative hack was enough of a change that the RPi4B made it to a login prompt and I logged in. There were problems, but I could do some stuff. The failure mechanism for the "extra" cores (APs) was different, reporting very early on: Starting CPU 1 (1) Failed to start CPU 1 (1) Starting CPU 2 (2) Failed to start CPU 2 (2) Starting CPU 3 (3) Failed to start CPU 3 (3) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)