From owner-freebsd-ppc@freebsd.org Mon Nov 21 11:23:07 2016 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33B6BC4C8B5 for ; Mon, 21 Nov 2016 11:23:07 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-29.reflexion.net [208.70.210.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDF04B6D for ; Mon, 21 Nov 2016 11:23:05 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 22025 invoked from network); 21 Nov 2016 11:23:34 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 21 Nov 2016 11:23:34 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.20.0) with SMTP; Mon, 21 Nov 2016 06:22:44 -0500 (EST) Received: (qmail 15564 invoked from network); 21 Nov 2016 11:22:44 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 21 Nov 2016 11:22:44 -0000 Received: from [192.168.1.106] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id A673CEC770D; Mon, 21 Nov 2016 03:22:58 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) Subject: Re: I think I found why the iMac G3 that I have access to has not booted FreeBSD vintages: 2015-Mar+ . . . [Yep: booted!] Date: Mon, 21 Nov 2016 03:22:58 -0800 References: <8915A2F8-0C75-4B8B-BCC1-7252EA02D292@dsl-only.net> To: Nathan Whitehorn , Justin Hibbits , FreeBSD PowerPC ML In-Reply-To: <8915A2F8-0C75-4B8B-BCC1-7252EA02D292@dsl-only.net> Message-Id: <53724219-2378-45E0-B521-8F3EFA85CE41@dsl-only.net> X-Mailer: Apple Mail (2.3251) X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2016 11:23:07 -0000 [Top post of operational confirmation.] I'm now logged in on the iMac G3 under a variant of head -r308874 . (Finally after about 1.8 years.) It is currently running a variation of head -r308874 with a debug kernel that has the two isync's added around moea_activate's mtsrin (and KTR turned back off). With no such isync's (or other such "context-synchronizations") the iMac G3 does not boot. (The below likely does not preserve tabs.) # svnlite diff /usr/src/sys/powerpc/aim/mmu_oea.c=20 Index: /usr/src/sys/powerpc/aim/mmu_oea.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/aim/mmu_oea.c (revision 308874) +++ /usr/src/sys/powerpc/aim/mmu_oea.c (working copy) @@ -991,7 +991,9 @@ CPU_SET(PCPU_GET(cpuid), &pm->pm_active); PCPU_SET(curpmap, pmr); =20 + isync(); mtsrin(USER_SR << ADDR_SR_SHFT, = td->td_pcb->pcb_cpu.aim.usr_vsid); + isync(); } stable/11 should also get such a change, not just head. It would be nice if releng/11 eventually picked up such a change so that some release/11.0.? booted on iMac G3's as well. Otherwise it waits for release/11.1.0 . I wonder if there might be intermittent problems for TARGET_ARCH=3Dpowerpc systems that are (usually) booting for release/11.0.x currently. (I only have access to one iMac G3 to test and no access to any other kinds of G3's. I have access to a few types of PowerMac G4's and 2 types of PowerMac G5's. All the PowerPc family machines that I have access to are Apple machines.) Note: stable/10 still has the old powerpc/swtch32.S code and so is fine for this issue. Part of the context from back in early 2015 was that I switched from 10 to 11 as part of getting ready to investigate projects/clang380-import for powerpc and powerpc64 use. I did not revert back to 10.x despite the iMac G3 not booting. =3D=3D=3D Mark Millard markmi at dsl-only.net On 2016-Nov-21, at 2:10 AM, Mark Millard wrote: > First I report my understanding of the PowerPc background information > involved: > (then later the code that has that background involved) >=20 > For reference: >=20 > 82 mtsrin(vm_offset_t va, register_t value) > 83 { > 84=20 > 85 __asm __volatile ("mtsrin %0,%1" :: "r"(value), "r"(va)); > 86 } >=20 > PowerPC requirements: >=20 > mtsr(instruction access): no synchronization required before; > context synchronization required after > mtsrin(instruction access): no synchronization required before; > context synchronization required after >=20 > So the same criteria. isync, sc, or rfi would be=20 > "context-synchronizing". >=20 > mtsr(data access): context synchronization required before; > context synchronization required after > mtsrin(data access): context synchronization required before; > context synchronization required after >=20 > So even more required for this context: before and after. > Again isync would be "context-synchronizing". >=20 >=20 > Now the code that has that background involved. . . >=20 > aim/mmu_oea.c's moea_activate does mtsrin without any explicit > "context-synchronizing" before or after it --and it replaced > code that did have the "context-synchronizing". >=20 > The modern (2015-Mar-4+) code: >=20 > /* > * Activate a user pmap. The pmap must be activated before it's = address > * space can be accessed in any way. > */ > void > moea_activate(mmu_t mmu, struct thread *td) > { > pmap_t pm, pmr; >=20 > /* > * Load all the data we need up front to encourage the compiler = to > * not issue any loads while we have interrupts disabled below. > */ > pm =3D &td->td_proc->p_vmspace->vm_pmap; > pmr =3D pm->pmap_phys; >=20 > CPU_SET(PCPU_GET(cpuid), &pm->pm_active); > PCPU_SET(curpmap, pmr); >=20 > mtsrin(USER_SR << ADDR_SR_SHFT, = td->td_pcb->pcb_cpu.aim.usr_vsid); > } >=20 > I expect that two isync's are missing. >=20 > At the assembler level of detail the modern code in my example > build is: >=20 > 0080e3fc stwu r1,-32(r1) > 0080e400 stw r31,24(r1) > 0080e404 mr r31,r1 > 0080e408 lwz r9,4(r4) > 0080e40c lwz r10,308(r9) > 0080e410 lwz r8,312(r10) > 0080e414 mfsprg r9,0 > 0080e418 lwz r9,36(r9) > 0080e41c rlwinm r9,r9,27,5,31 > 0080e420 mfsprg r11,0 > 0080e424 rlwinm r9,r9,2,0,29 > 0080e428 add r9,r9,r10 > 0080e42c addi r9,r9,256 > 0080e430 lwz r11,36(r11) > 0080e434 clrlwi r11,r11,27 > 0080e438 li r0,1 > 0080e43c slw r0,r0,r11 > 0080e440 lwz r11,24(r9) > 0080e444 or r0,r0,r11 > 0080e448 stw r0,24(r9) > 0080e44c mfsprg r9,0 > 0080e450 stw r8,304(r9) > 0080e454 lwz r9,644(r4) > 0080e458 lwz r9,1176(r9) > 0080e45c lis r0,-16384 > 0080e460 mtsrin r9,r0 <<<<<<<<=3D=3D=3D=3D=3D=3D= =3D "Context-synchronization(s)"? > 0080e464 lwz r11,0(r1) > 0080e468 lwz r31,-8(r11) > 0080e46c mr r1,r11 > 0080e470 blr >=20 >=20 > But the old, historical code that this replaced did have > explicit "context-synchornizing" in powerpc/swtch32.S for > AIM ( -r279594 replaced the below on 2015-Mar-4 ): >=20 > -#ifdef AIM > - lwz %r5,PCB_AIM_USR_VSID(%r3) /* Load the USER_SR segment = reg */ > - isync > - mtsr USER_SR,%r5 > - isync > -#endif >=20 > (It was part of setting the user pmap during thread switching.) >=20 > This replacement happened shortly before I discovered that > the iMac G3 could no longer boot. It stops in pmap_activate > in the lwz r11,0(r1) just after moea_activate returns from > its bctrl-based call (modern code again below): >=20 > . . . > 00847954 beq- cr7,00847960 = > 00847958 lwz r3,1024(r11) > 0084795c bl 004fdfac > 00847960 lwz r3,4(r3) > 00847964 mtctr r3 > 00847968 mr r3,r29 > 0084796c mr r4,r28 > 00847970 bctrl <<<<<<<<<<=3D=3D=3D=3D=3D=3D=3D=3D=3D= Calls moea_activate. > 00847974 lwz r11,0(r1) <<<<<<<=3D=3D=3D=3D=3D = This fails. >=20 > I end up at the db> prompt (too early for interactive input). > (I have ddb automatically execute a compiled-in script.) >=20 > I've confirmed with show registers that ctl has the address of = moea_activate > (0x80e3fc in my example build of head -r308874) and srr0 has = pmap_activate+0xb4's > value (matching lr). srr1 has the value 0x1032. dar has the value = 0xe43f6a50 > (matching r1 and r31). dsisr has the value 0x40000000. >=20 > I also have confirmed with verbose KTR reporting that: >=20 > cpu0 mi_switch: old thread 100022 (td_sched 0x2x0e6a8, pid 12, irq0: = pcm0) >=20 > happens just before it stops at pmap_activate+0xb4, at least > as far as visible messages go. >=20 > Again, I expect that two isync's are missing in moea_activate: > one before the mtsrin and one after it, matching the old mtsr > handling from before the change. >=20 > =3D=3D=3D > Mark Millard > markmi at dsl-only.net