From owner-freebsd-ppc@FreeBSD.ORG Sat Feb 28 15:17:42 2015 Return-Path: Delivered-To: freebsd-ppc@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85F37CA6 for ; Sat, 28 Feb 2015 15:17:42 +0000 (UTC) Received: from asp.reflexion.net (outbound-242.asp.reflexion.net [69.84.129.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 428E495 for ; Sat, 28 Feb 2015 15:17:42 +0000 (UTC) Received: (qmail 14793 invoked from network); 28 Feb 2015 15:11:01 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 28 Feb 2015 15:11:01 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v7.40.1) with SMTP; Sat, 28 Feb 2015 10:11:01 -0500 (EST) Received: (qmail 32754 invoked from network); 28 Feb 2015 15:11:01 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (DHE-RSA-AES256-SHA encrypted) SMTP; 28 Feb 2015 15:11:01 -0000 X-No-Relay: not in my network X-No-Relay: not in my network Received: from [192.168.1.8] (c-67-189-19-145.hsd1.or.comcast.net [67.189.19.145]) by iron2.pdx.net (Postfix) with ESMTPSA id 4E6051C43A6; Sat, 28 Feb 2015 07:10:58 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Large RAM Powermac G5 powerpc64 openfirmware entry point use: I seem to have removed the major/frequent boot problem(s) Message-Id: Date: Sat, 28 Feb 2015 07:10:59 -0800 To: FreeBSD PowerPC ML , Nathan Whitehorn Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) X-Mailer: Apple Mail (2.2070.6) X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 15:17:42 -0000 I have removed my re-try logic that I had in ofwcall because other = changes I've made got rid of the %r1/%r3 corruption issue that I used to = have so frequently when booting PowerMac G5s that have lots of RAM (8G, = 12G, or 16G). Also I'm no longer getting the specific .got area corruption that = authnone code was failing on when it used its %r2 to access various = things when I built with VERBOSE_SYSINIT. (It is hard to have evidence = that no corruption is happening anywhere.) I've done multiple "make -j 8 buildworld kernel && make installworked" = examples under the changes and have updated to 10.1-STABLE -r279201 = under these changes. I've done similarly for with and without = VERBOSE_SYSINIT and the like. I've done a half dozen build/install = reboot and use sequences. I've booted and used both a PowerMac11,2 and a Powermac7,2 from the same = SSD. (I do not have access to other PowerMac G5 variants.) (I've done no = investigation of analogous points yet for powerpc PowerMacs: only = powerpc64 usage of G5s.) [I would not claim that this addresses the less frequent boot failure = tracebacks that I've reported in the past: just the most frequent one = and the recent .got corruption evidence.] So what did I do? It was limited to openfirmware_core, ofw_sprg_prepare, = and ofwcall. The bias was to simplify the context involved in my = investigation, not to produce production FreeBSD code = structured/partitioned for general use. The below is not in the order of = investigation but is a description of the net result. I do not claim = that all the eliminations involved were necessary but I had less code to = worry about this way. 0) I've looked at some of the openfirmware entry point code to see what = appeared to be needed vs. not needed. 1) I've looked at some of the BootX code and what it compiles to see an = example of what was sufficient in that context. For sys/powerpc/ofw/ofwcall64.S... 2) Despite working in/with/on 10.1-STABLE I started from ofwcall64.S = from a recent 11.0-CURRENT, from after the powerpc64 kernel-relocation = changes. (And I used the matching sys/powerpc/include/asm.h update so = ofwcall64.S would compile in 10.1-STABLE.) 3) I removed things that I found to not be necessary: ofwstk and its use = (openfirmware does its own storage management for such things), use of = ofwmsr storage (msr being just locally in registers for my code, = SPRG's not directly used), saving and restoring unused non-volatile = registers (openfirmware does 64-bit save/restore of the non-volatile = registers that it uses). BootX also does nothing special for %r1 when = calling openfirmware. 4) I added/kept what was needed (possibly expressed differently): = forcing 32-bit mode before and putting msr back to its original status = after, putting back %r2's TOC value (openfirmware uses Darwin's rule = that %r2 is volatile/non-dedicated --no TOC use-- and it replaces r2's = value), sign extend %r3. 5) In ofwcall I followed all the ABI rules that I've learned of, both = FreeBSD powerpc64 and Darwin ABI rules (e.g., the %r12 indirect call = address rule). This includes covering %r1, cr, lr, %r14, %r15 = saves/restores back to caller's status. (%r14 and %r15 were used in my = ofwcall.) 6) My ofwcall does increment a count stored in memory if it sees a = changed %r1 when openfirmware returns. This allows checking if it ever = happened via kgdb. It also records the most recent such new %r1 value. = This part of the code I consider temporary a evidence gathering hack in = case problems eventually happen. For sys/powerpc/ofw/ofw_machdep.c ... 7) In ofw_sprg_prepare I eliminated the unnecessary SPRG register = handling, leaving only SPRG0 (for later restoring required FreeBSD = context) and SPRG3 (for restoring required Openfirmware context). 8) In/for openfirmware_core I eliminated all the trap vector = save/restore calls. If openfirmware needs any of its exception vector = code it seems to patch/unpatch as necessary. BootX also does not context = switch the vectors but just leaves its own in place. I also eliminated = the extern save_trap_init declaration and the save_trap_of storage. = Because ofwcall no longer had ofwmsr involved, I removed ofwmsr's extern = status so it would provide the definition for the link. (I did not = change the size or the positions assigned for the used parts of ofwmsr.) = Much of this was commenting-out activity. (Note: I choose to ignore that the trap vector save/restore calls have = code to disable the activity based on ofw_real_mode's value and make it = locally obvious that no such code was in use for my testing.) 9) I did not eliminate the now-otherwise-unused save_trap_init storage = or the ofw_save_trap_vec call from powerpc_init in = sys/powerpc/aim/machdep.c. But I could have --at which point = ofw_save_trap_vec could also have been eliminated. Also I'm not = reporting on temporary evidence/investigation code that I had at various = times but have removed over time. The result was the frequent boot crashes disappeared and I've no = evidence so far of %r1/%r3 corruption or the .got corruption that I'd = reported on earlier. My openfirmware_core, ofw_sprg_prepare, and ofwcall are too PowerMac G5 = context-specific for general FreeBSD use. But they should well point to = a now-known way to improve booting for PowerMac G5's. Other/supporting notes: I finally figured out that the ddb code for x/i is incomplete for = Powerpc64. For example DDB's x/i calls mtmsrd an Illegal Instruction. One can see this if you have it decode ofwcall from memory and compare = to the source code. Because of this the decoding of the first instructions for the = openfirmware entry point (0xff846d78 starting address on the G5 = Quad-Core) is: or r2,r0,r2, addis r2,r0,-0x49 ori r2,r2,0xf000 /* so %r2:=3D0xFFB7F000: fixed address (32-bit = mode). */ std r1,r2,0x8, /* %r1 saved to have a special, separate copy */ std r0,r2,0x10, /* more saves to fixed locations */ mfspr r0,lr std r0,r2,0x120, /* more saves to fixed locations: return address = */ mfmsr r1 std r1,r2,0x108, /* more saves to fixed locations: msr */ rldicl r1,r1,0,1 /* clears the most significant %r1 bit */ 0x7c200164 (actually mtmsrd %r1) /* forces 32-bit mode */ isync Unfortunately for 64-bit mode at the start: %r2=3D0xFF...FFB7F000 and = std r1,r2,0x8, ends up rejecting the effective address. Apple does not = force 32-bit mode until a little later in the above. Thus ofwcall does = need to force 32-bit mode and return it back to normal, despite the msr = and other save/restore code that openfirmware has.=20 (I've not explored trying to set up a mapping for the involved 64-bit = effective address range in order to allow the translation of the 64-bit = addresses in openfirmware's %r2 above. With that it might be that 64-bit = mode could be left in place.) As far as I can tell this %r2 value and its use is the only reason that = FreeBSD needs to establish 32-bit mode before the call into = openfirmware. %r2 needs to be restored after openfirmware returns since openfirmware = is using the Darwin ABI where %r2 is non-volatile and non-dedicated (no = use of TOCs), as can be seen above. =3D=3D=3D Mark Millard markmi at dsl-only.net