From owner-freebsd-ppc@freebsd.org  Thu Feb 14 03:45:33 2019
Return-Path: <owner-freebsd-ppc@freebsd.org>
Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3376D14E6466
 for <freebsd-ppc@mailman.ysv.freebsd.org>;
 Thu, 14 Feb 2019 03:45:33 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic309-22.consmr.mail.gq1.yahoo.com
 (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 3CD078C57F
 for <freebsd-ppc@freebsd.org>; Thu, 14 Feb 2019 03:45:31 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: o.eS9N4VM1lri9KL8p7AiQ13MMBm1JZBQ7pYr4JU7YoLFGeBSPhlnRXHF_23oA4
 eVfLAPN_t4GfVlMiYBxztGnEBdevRCqwQe_drXjHBm21b5j8eKf6otp5mVWKw_2G40njvhmtGfOQ
 dMwtO9.9QPoGRCJVnzQdE.3pMZkAMO50YWgAihZytacQpUaaX3.Afe7892lLvNQ94saJEV0ibBNA
 qI_K1DRmV2UYt7550643Ktx3VGe31CikFcSdXSe_KxzV1I5Ms3LWc_dLIrCa7ISbCRf3oAbQC6jj
 nPkMUhkLcBUOPeiVok3.nnwAjYQUa6GDpxFHHmmSUoI.bX8OoNXdO_iEKNJE9XKEHIMi.y3XBQi3
 0M6RoGhpti48NPEdgt8izyYnmcQhCI1TDqd6Fa3cnQtP9WqPtLMLJ4cJ7cqXD5UPc_1vYX_vWKD_
 cD_n2oTNoc6OnGU7BxShh9dDi.9EsWuvR.FuJRTJaEzWf4lqZwwPNQGCuTHntikQec5dRKACVKY9
 Sb0LFZpscYvdGE0Stz6XBvL0Ni6IM9O7Cqy2DLjZTSC_P6xl6xLGfka2ut7Y8x5490Tk7tkxw68F
 ClY.yCh2ZUbs00GAcm.OZTqJQcZShfEojXlVMEfXa9Ao.Acv.BFpI_2ZA0BFaqKE5mdr23eL.SIw
 h_IV7BQg_pGLVRgb_EduyhmRnJjJ_5wGQNCXOp3O22awRoCKEKTyymPxhf5FP0aqtftoy0bmpd_q
 Ej8_qv9kes_Znqvs_Kov9QBAfsagQwHjnx_xx0.8jE7JuzMimu_zer2aS5BaLhvrzT3lBAbbbhxW
 atM68z9w0y06hwcE_wrrtcpSVRHoxs4JO6NraBTMXYv9s5H9yWkygabRUqWovgwtCCpAZt3rrfK4
 XVXK2Sncht_9yMk2eY7olu3QIp6vkeJxrKBwVQe0y4HJoREjxCza5UoQCA_ZB8r8BBqXfu2XuJBn
 VKRKpU6M3IMp0nVyRFMwmxOk3ReThVzHEiTMWMBlzQ6SAVFsbXmX0mdn9Klc7uGSWXmgAS.gmxHp
 C3v5MnsFGFadUhJEEqh7.DILWo1a3ZFL91AEMuj__H.GxaAGqNg--
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic309.consmr.mail.gq1.yahoo.com with HTTP; Thu, 14 Feb 2019 03:45:21 +0000
Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113])
 ([67.170.167.181])
 by smtp431.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
 d60a4c4b29c7061f2fdea3a4957147ff for <freebsd-ppc@freebsd.org>;
 Thu, 14 Feb 2019 03:35:14 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\))
Subject: Some evidence about the PowerMac G5 multiprocessor boot hang ups with
 the modern VM_MAX_KERNEL_ADDRESS value
Message-Id: <11680D15-D43D-4115-AF4F-5F6E4E0022C9@yahoo.com>
Date: Wed, 13 Feb 2019 19:35:13 -0800
To: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>
X-Mailer: Apple Mail (2.3445.102.3)
X-Rspamd-Queue-Id: 3CD078C57F
X-Spamd-Bar: -
X-Spamd-Result: default: False [-1.44 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[];
 FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3];
 TO_DN_ALL(0.00)[];
 MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net];
 DKIM_TRACE(0.00)[yahoo.com:+];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[];
 MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.930,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048];
 URIBL_BLOCKED(0.00)[dsl-only.net.multi.uribl.com];
 FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[];
 NEURAL_HAM_LONG(-0.99)[-0.993,0]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org];
 RCPT_COUNT_ONE(0.00)[1];
 IP_SCORE(0.24)[ipnet: 98.137.64.0/21(0.71), asn: 36647(0.57), country:
 US(-0.07)]; NEURAL_SPAM_SHORT(0.75)[0.755,0];
 RCVD_IN_DNSWL_NONE(0.00)[148.65.137.98.list.dnswl.org : 127.0.5.0]
X-BeenThere: freebsd-ppc@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Porting FreeBSD to the PowerPC <freebsd-ppc.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-ppc>,
 <mailto:freebsd-ppc-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ppc/>
List-Post: <mailto:freebsd-ppc@freebsd.org>
List-Help: <mailto:freebsd-ppc-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-ppc>,
 <mailto:freebsd-ppc-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Feb 2019 03:45:33 -0000

I added some bootverbose messages to:

static int
powermac_smp_start_cpu(platform_t plat, struct pcpu *pc)
(in /usr/src/sys/powerpc/powermac/platform_powermac.c )

and:

void
machdep_ap_bootstrap(void)
( /usr/src/sys/powerpc/powerpc/mp_machdep.c )

and:

static void
cpu_mp_unleash(void *dummy)
( /usr/src/sys/powerpc/powerpc/mp_machdep.c )



The result was (typed from a image and
showing normal messages too):

Adding CPU 0, hwref=3Dcd38, awkae=3D1
Waling up CPU 3 (dev=3Dc480)
Before reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0
After reset 4&0 for CPU3, hwref=3Dc480, awake=3D0
After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0
cpu_mp_unleash after platform_smp_start_cpu and waiting: CPU3, =
hwref=3Dx480, awake=3D0
cpu_mp_unleash adding pc_cpuid to stopped_cpus: CPU 3 (dev=3Dc480)
Waking up CPU 2 (dev=3Dc768)
Before reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0

There is no more text.

Part of the information is what messages were not displayed.

Some notes:

Only CPU 0 gots a: smp_cpus++ (resulting in 1).

There is no solid evidence for if machdep_ap_bootstrap was
ever used for CPU 3 or CPU 2.

The wait loop after the platform_smp_start_cpu call in
cpu_mp_unleash for CPU 3 is not used. The code in
machdep_ap_bootstrap:

        PCPU_SET(awake, 1);
        __asm __volatile("msync; isync");

did not have a visible effect on the pc->pc_awake value in
platform_smp_start_cpu and cpu_mp_unleash. I have no
unique evidence that it was executed at all.

My messages added to machdep_ap_bootstrap were not displayed.

platform_smp_start_cpu did not return for CPU 2.
cpu_mp_unleash makes no more progress. In fact the
2 resets do not complete overall for CPU 2 (but did
for CPU 3, even if it is unclear what code was
executed):

        *rstvec =3D 4;
        powerpc_sync();
        (void)(*rstvec);
        powerpc_sync();
        DELAY(1);
        *rstvec =3D 0;
        powerpc_sync();
        (void)(*rstvec);
        powerpc_sync();

I'll see about getting more information about the resets used.



For reference for the added messages:

# svnlite diff /usr/src/sys/powerpc/powermac/platform_powermac.c =
/usr/src/sys/powerpc/powerpc/mp_machdep.c
Index: /usr/src/sys/powerpc/powermac/platform_powermac.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- /usr/src/sys/powerpc/powermac/platform_powermac.c	(revision =
344018)
+++ /usr/src/sys/powerpc/powermac/platform_powermac.c	(working copy)
@@ -371,6 +371,11 @@
 		rstvec_virtbase =3D pmap_mapdev(0x80000000, PAGE_SIZE);
=20
 	rstvec =3D rstvec_virtbase + reset;
+=20
+        if (bootverbose) // HACK!!!
+                printf("Before reset 4&0 for CPU %d, hwref=3D%jx, =
awake=3D%x\n",
+                    pc->pc_cpuid, (uintmax_t)pc->pc_hwref,
+                    pc->pc_awake);
=20
 	*rstvec =3D 4;
 	powerpc_sync();
@@ -382,10 +387,20 @@
 	(void)(*rstvec);
 	powerpc_sync();
=20
+        if (bootverbose) // HACK!!!
+                printf("After reset 4&0 for CPU %d, hwref=3D%jx, =
awake=3D%x\n",
+                    pc->pc_cpuid, (uintmax_t)pc->pc_hwref,
+                    pc->pc_awake);
+
 	timeout =3D 10000;
 	while (!pc->pc_awake && timeout--)
 		DELAY(100);
=20
+        if (bootverbose) // HACK!!!
+                printf("After attempted wait for awake CPU %d, =
hwref=3D%jx, awake=3D%x\n",
+                    pc->pc_cpuid, (uintmax_t)pc->pc_hwref,
+                    pc->pc_awake);
+
 	return ((pc->pc_awake) ? 0 : EBUSY);
 #else
 	/* No SMP support */
Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- /usr/src/sys/powerpc/powerpc/mp_machdep.c	(revision 344018)
+++ /usr/src/sys/powerpc/powerpc/mp_machdep.c	(working copy)
@@ -97,6 +97,9 @@
 	/* Initialize decrementer */
 	decr_ap_init();
=20
+        if (bootverbose) // HACK!!!
+                printf("machdep_ap_bootstrap before ap_boot_mtx lock: =
AP CPU #%d launched\n", PCPU_GET(cpuid));
+
 	/* Serialize console output and AP count increment */
 	mtx_lock_spin(&ap_boot_mtx);
 	ap_awake++;
@@ -109,6 +112,8 @@
=20
 	while(smp_started =3D=3D 0)
 		;
+        if (bootverbose) // HACK!!!
+                printf("machdep_ap_bootstrap after smp_started!=3D0: AP =
CPU #%d launched\n", PCPU_GET(cpuid));
=20
 	/* Start per-CPU event timers. */
 	cpu_initclocks_ap();
@@ -238,10 +243,19 @@
=20
 			ret =3D platform_smp_start_cpu(pc);
 			if (ret =3D=3D 0) {
+                                if (bootverbose) // HACK!!!
+                                        printf("cpu_mp_unleash =
attempting to wait for pc_awake: CPU %d, hwref=3D%jx, awake=3D%x\n",
+                                            pc->pc_cpuid, =
(uintmax_t)pc->pc_hwref,
+                                            pc->pc_awake);
+
 				timeout =3D 2000;	/* wait 2sec for =
the AP */
 				while (!pc->pc_awake && --timeout > 0)
 					DELAY(1000);
 			}
+                        if (bootverbose) // HACK!!!
+                                printf("cpu_mp_unleash after =
platform_smp_start_cpu and waiting: CPU %d, hwref=3D%jx, awake=3D%x\n",
+                                    pc->pc_cpuid, =
(uintmax_t)pc->pc_hwref,
+                                    pc->pc_awake);
 		} else {
 			pc->pc_awake =3D 1;
 		}
@@ -252,7 +266,13 @@
 				    pc->pc_awake);
 			smp_cpus++;
 		} else
+                { // HACK!!!
+                        if (bootverbose) // HACK!!!
+                                printf("cpu_mp_unleash adding pc_cpuid =
to stopped_cpus: CPU %d (dev=3D%x)\n",
+                                    pc->pc_cpuid, (int)pc->pc_hwref);
 			CPU_SET(pc->pc_cpuid, &stopped_cpus);
+                } // HACK!!!
+
 	}
=20
 	ap_awake =3D 1;


=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)