From owner-freebsd-arm@freebsd.org Thu Feb 13 02:00:47 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DC7C8249E94 for ; Thu, 13 Feb 2020 02:00:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48J08v042Qz4KF0 for ; Thu, 13 Feb 2020 02:00:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: Nk.dtsgVM1kBv0.Iv928C0UEJhS.sxuEkScFjlO_guNJw_blrAJoUpybpHMwHgA TsTQMtPtYVokGuygh9.xgToBbjE7iudP7hcI0rBEGH7VCgBh3B_JQwb.mtpQcsdfxKW3AKLxdZCJ 0cunkcPMlGKSrLEd7UfV3PtvDgdEvHopK3SrB4escejd6MPfkQBl.liea1Mrph5oHkz_Wsra6uIw U4SLl2mukDKGy5EPP5abLfIm0hHBMlmKYYckrhIBUTsXMH_ehJ9tGWL3L0kUz2L7_8W4tgM9Fcos ..AYwhPNg5ksObXoK5lhT8yfzhJruoR5D_wM0JdOecjEdMFy.Jx.nmezPQs2xaayG28ijOEH.Zqb HZiDhkUNzVpSBY5big4dNdqsbTYU.kjFytLRKLH5c1cd_lJMoOuSQ4_2RdlhsJVhmKl3rbS4tT3T pFRdnVlaA3RCsoEfY8.iApMBz0.HfU.0pFo8d2RPU.M7PM_0XYh8C8Itdpi5ZIdsblOjyIqvvZTr tXPfI_F6Cjv6nSQjSsX7Xikjiylt8GQKWcadIiIrth6OuMIDhzIAAxSL1ZVK03V795D3DCKKUWMw dvhGkETqa8566C4pYvLD6F9vu0XiT97Lv_OGayCFscR18EUJPqr5emiIklxTdNeaKMIQPcff0Pe1 qlgzPHh0uAUkAqcgXTHw0H7doAb6RO3EbBKHWemZZ6GMPUqi.76NHVmr8kEVolB.NMdCmeQyLtuw SKjjhPQliMPQIoJ91T1aAEMSpKZhEjTmvqLg7ztg4ZsYn00zfvfVX2JQmpwFUdsMzftSIe1RrHU4 urW1GxyXxiiSuPwLKJcxe5zJh7XXAIByy4gNDOi6nt05VmLcvzpY07jgI166pWRhAymFLQ0e8o2. qi6Z0AfbdiyZLCyhZ5LXzSfS2EEEx175wZEnjx3zUGE206zLCLPuZwPFM.Q2qn0wPCRlKIM8SRzd RTJjiPjJNizg8r6GAZRgRVtORDpzZoZOab5xgLMjVO1KSWeDIZTPeYqIDTri2bQ1IibjnGPKtt0G rFR.rJEXI30E.lZ10uo0URYxLkm2eJT8a2AEe.8oAd4n2GCeLhagRlQePDLn2YNxR.Li7rbIKVRr y8evfd9UACAlmQk9y8FYEbSGBxGDajg4SV2Suhm9RRxtpsVxJJZ4e741BcSWGI.W5PFfZMr5KrFO UoPRM.UJXoY.Ym9yARFh1T1a5rqbJ55m2FLMVOXMDztUc3dNA5KviL7Ueki5rcY4J7ZuzgiNMLxe fviVsypX7uGsaeIsOs4SIbLv_ZHzIMPfhQOj_9JNlHagtclY2JH7afamch15lTGxF0ULdRv7McmJ dlZG8ekUmeIas54rQXLf7CenevNJTLwNP5vaVDms.zTxCij0IdOJ4qFMrmVPIwSRP0PpXTDNkvwL EVHHj0umjM7vbHzqkusMaTdEbIyhLjOevQy6KxGhLFJUOMaILZI0Mxu7s4u32fcrboiiozhwoD7y YXQ-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Thu, 13 Feb 2020 02:00:44 +0000 Received: by smtp406.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 34ad7830a4137743b585d067bc313c2f; Thu, 13 Feb 2020 02:00:43 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: A investigative hack that makes (for example) head -r356529 boot and operate normally an RPi4B (finally!): protect all armstub8-gic.bin's loaded content from replacement by the kernel Message-Id: <7E7605DC-021D-448A-8459-8EC26BA9836D@yahoo.com> Date: Wed, 12 Feb 2020 18:00:41 -0800 Cc: Andrew Turner , Emmanuel Vadot To: freebsd-arm , Kyle Evans , gonzo@freebsd.org X-Mailer: Apple Mail (2.3608.60.0.2.5) References: <7E7605DC-021D-448A-8459-8EC26BA9836D.ref@yahoo.com> X-Rspamd-Queue-Id: 48J08v042Qz4KF0 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.02 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[yahoo.com:+]; SUBJECT_HAS_EXCLAIM(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.71)[-0.713,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.81)[-0.811,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (5.02), ipnet: 98.137.64.0/21(0.82), asn: 36647(0.66), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[84.64.137.98.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Feb 2020 02:00:47 -0000 [head -r356529 was just handy in my context. I expect this works for head -r356776 and later and probably before the problem was exposed as well.] The technique here is a quick hack to give evidence for what correct, general code needs to do that is not being done ( given the head -r356776 changes exposing an assumption no longer true and the use of the FreeBSD specific armstub8-gic.bin in my test context but armstub8.bin in some other contexts). I put in code to add a reserved memory region spanning the 2 pages at the beginning of the address space. This is enough to span all the armstub8-gic.bin content (that is loaded to address 0x0 in my test context). Note the first listed memory region in: Excluded memory regions: 0x00000000 - 0x00001fff, 0 MB ( 2 pages) NoAlloc NoDump 0x00000000 - 0x00000fff, 0 MB ( 1 pages) NoAlloc . . . My hack added that 2 page range as if it was in memreserve in the live DTB. I already had the failover code for fdt_get_reserved_mem to call fdt_get_reserved_regions (to try for memreserve when /reserved-memory is not found). So here is what enabled the "boots and operates normally" status for the RPi4B with a -r356776 or later version of head (yes, all 4 cores operating): # svnlite diff /usr/src/sys/dev/fdt/fdt_common.c Index: /usr/src/sys/dev/fdt/fdt_common.c =================================================================== --- /usr/src/sys/dev/fdt/fdt_common.c (revision 357529) +++ /usr/src/sys/dev/fdt/fdt_common.c (working copy) @@ -485,7 +485,18 @@ tuples = res_len / tuple_size; reservep = (pcell_t *)&reserve; +#ifdef __aarch64__ + //HACK!!! + // Reserve the first few pages, for example to + // preserve armstub8-gic.bin or armstub.bin + // content. + mr[0].mr_start= 0; + mr[0].mr_size= 2*4096; + tuples++; + for (i = 1; i < tuples; i++) { +#else for (i = 0; i < tuples; i++) { +#endif rv = fdt_data_to_res(reservep, addr_cells, size_cells, (u_long *)&mr[i].mr_start, (u_long *)&mr[i].mr_size); @@ -512,6 +523,11 @@ root = OF_finddevice("/reserved-memory"); if (root == -1) { + // Fail over to checking for and handling memreserve, + // such as for a RPi4B. + if (0 == fdt_get_reserved_regions(reserved,mreserved)) + return (0); + return (ENXIO); } I had done other investigative work earlier to find for sure where armstub8-gic.bin was being loaded in my example context: address 0x0. I'm not trying to imply that assuming that load address is appropriate. I simply do not know how general address 0x0 is. I'm only trying to imply that the page range that ends up containing the armstub8*.bin content should be excluded from what the kernel will use for allocations. I'm not trying to imply that DTB memreserve content should be the involved. The memreserve related code just happened to be where I choose to do the "exclude more" hack. I expect that causing the pages holding armstub*.bin content to be excluded from FreeBSD writing to any of those pages would also enable RPi3's and the like to boot and operate as aarch64 FreeBSD examples for head -r356776 (and later, other than other bugs). It did for the RPi4B 4 GiByte test context I used. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)