From owner-freebsd-ppc@freebsd.org Thu Feb 14 03:45:33 2019 Return-Path: <owner-freebsd-ppc@freebsd.org> Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3376D14E6466 for <freebsd-ppc@mailman.ysv.freebsd.org>; Thu, 14 Feb 2019 03:45:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3CD078C57F for <freebsd-ppc@freebsd.org>; Thu, 14 Feb 2019 03:45:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: o.eS9N4VM1lri9KL8p7AiQ13MMBm1JZBQ7pYr4JU7YoLFGeBSPhlnRXHF_23oA4 eVfLAPN_t4GfVlMiYBxztGnEBdevRCqwQe_drXjHBm21b5j8eKf6otp5mVWKw_2G40njvhmtGfOQ dMwtO9.9QPoGRCJVnzQdE.3pMZkAMO50YWgAihZytacQpUaaX3.Afe7892lLvNQ94saJEV0ibBNA qI_K1DRmV2UYt7550643Ktx3VGe31CikFcSdXSe_KxzV1I5Ms3LWc_dLIrCa7ISbCRf3oAbQC6jj nPkMUhkLcBUOPeiVok3.nnwAjYQUa6GDpxFHHmmSUoI.bX8OoNXdO_iEKNJE9XKEHIMi.y3XBQi3 0M6RoGhpti48NPEdgt8izyYnmcQhCI1TDqd6Fa3cnQtP9WqPtLMLJ4cJ7cqXD5UPc_1vYX_vWKD_ cD_n2oTNoc6OnGU7BxShh9dDi.9EsWuvR.FuJRTJaEzWf4lqZwwPNQGCuTHntikQec5dRKACVKY9 Sb0LFZpscYvdGE0Stz6XBvL0Ni6IM9O7Cqy2DLjZTSC_P6xl6xLGfka2ut7Y8x5490Tk7tkxw68F ClY.yCh2ZUbs00GAcm.OZTqJQcZShfEojXlVMEfXa9Ao.Acv.BFpI_2ZA0BFaqKE5mdr23eL.SIw h_IV7BQg_pGLVRgb_EduyhmRnJjJ_5wGQNCXOp3O22awRoCKEKTyymPxhf5FP0aqtftoy0bmpd_q Ej8_qv9kes_Znqvs_Kov9QBAfsagQwHjnx_xx0.8jE7JuzMimu_zer2aS5BaLhvrzT3lBAbbbhxW atM68z9w0y06hwcE_wrrtcpSVRHoxs4JO6NraBTMXYv9s5H9yWkygabRUqWovgwtCCpAZt3rrfK4 XVXK2Sncht_9yMk2eY7olu3QIp6vkeJxrKBwVQe0y4HJoREjxCza5UoQCA_ZB8r8BBqXfu2XuJBn VKRKpU6M3IMp0nVyRFMwmxOk3ReThVzHEiTMWMBlzQ6SAVFsbXmX0mdn9Klc7uGSWXmgAS.gmxHp C3v5MnsFGFadUhJEEqh7.DILWo1a3ZFL91AEMuj__H.GxaAGqNg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Thu, 14 Feb 2019 03:45:21 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp431.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID d60a4c4b29c7061f2fdea3a4957147ff for <freebsd-ppc@freebsd.org>; Thu, 14 Feb 2019 03:35:14 +0000 (UTC) From: Mark Millard <marklmi@yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Some evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value Message-Id: <11680D15-D43D-4115-AF4F-5F6E4E0022C9@yahoo.com> Date: Wed, 13 Feb 2019 19:35:13 -0800 To: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 3CD078C57F X-Spamd-Bar: - X-Spamd-Result: default: False [-1.44 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.930,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; URIBL_BLOCKED(0.00)[dsl-only.net.multi.uribl.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.993,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.24)[ipnet: 98.137.64.0/21(0.71), asn: 36647(0.57), country: US(-0.07)]; NEURAL_SPAM_SHORT(0.75)[0.755,0]; RCVD_IN_DNSWL_NONE(0.00)[148.65.137.98.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC <freebsd-ppc.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-ppc>, <mailto:freebsd-ppc-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ppc/> List-Post: <mailto:freebsd-ppc@freebsd.org> List-Help: <mailto:freebsd-ppc-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-ppc>, <mailto:freebsd-ppc-request@freebsd.org?subject=subscribe> X-List-Received-Date: Thu, 14 Feb 2019 03:45:33 -0000 I added some bootverbose messages to: static int powermac_smp_start_cpu(platform_t plat, struct pcpu *pc) (in /usr/src/sys/powerpc/powermac/platform_powermac.c ) and: void machdep_ap_bootstrap(void) ( /usr/src/sys/powerpc/powerpc/mp_machdep.c ) and: static void cpu_mp_unleash(void *dummy) ( /usr/src/sys/powerpc/powerpc/mp_machdep.c ) The result was (typed from a image and showing normal messages too): Adding CPU 0, hwref=3Dcd38, awkae=3D1 Waling up CPU 3 (dev=3Dc480) Before reset 4&0 for CPU 3, hwref=3Dc480, awake=3D0 After reset 4&0 for CPU3, hwref=3Dc480, awake=3D0 After attempted wait for awake CPU 3, hwref=3Dc480, awake=3D0 cpu_mp_unleash after platform_smp_start_cpu and waiting: CPU3, = hwref=3Dx480, awake=3D0 cpu_mp_unleash adding pc_cpuid to stopped_cpus: CPU 3 (dev=3Dc480) Waking up CPU 2 (dev=3Dc768) Before reset 4&0 for CPU 2, hwref=3Dc768, awake=3D0 There is no more text. Part of the information is what messages were not displayed. Some notes: Only CPU 0 gots a: smp_cpus++ (resulting in 1). There is no solid evidence for if machdep_ap_bootstrap was ever used for CPU 3 or CPU 2. The wait loop after the platform_smp_start_cpu call in cpu_mp_unleash for CPU 3 is not used. The code in machdep_ap_bootstrap: PCPU_SET(awake, 1); __asm __volatile("msync; isync"); did not have a visible effect on the pc->pc_awake value in platform_smp_start_cpu and cpu_mp_unleash. I have no unique evidence that it was executed at all. My messages added to machdep_ap_bootstrap were not displayed. platform_smp_start_cpu did not return for CPU 2. cpu_mp_unleash makes no more progress. In fact the 2 resets do not complete overall for CPU 2 (but did for CPU 3, even if it is unclear what code was executed): *rstvec =3D 4; powerpc_sync(); (void)(*rstvec); powerpc_sync(); DELAY(1); *rstvec =3D 0; powerpc_sync(); (void)(*rstvec); powerpc_sync(); I'll see about getting more information about the resets used. For reference for the added messages: # svnlite diff /usr/src/sys/powerpc/powermac/platform_powermac.c = /usr/src/sys/powerpc/powerpc/mp_machdep.c Index: /usr/src/sys/powerpc/powermac/platform_powermac.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powermac/platform_powermac.c (revision = 344018) +++ /usr/src/sys/powerpc/powermac/platform_powermac.c (working copy) @@ -371,6 +371,11 @@ rstvec_virtbase =3D pmap_mapdev(0x80000000, PAGE_SIZE); =20 rstvec =3D rstvec_virtbase + reset; +=20 + if (bootverbose) // HACK!!! + printf("Before reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake); =20 *rstvec =3D 4; powerpc_sync(); @@ -382,10 +387,20 @@ (void)(*rstvec); powerpc_sync(); =20 + if (bootverbose) // HACK!!! + printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake); + timeout =3D 10000; while (!pc->pc_awake && timeout--) DELAY(100); =20 + if (bootverbose) // HACK!!! + printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake); + return ((pc->pc_awake) ? 0 : EBUSY); #else /* No SMP support */ Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/mp_machdep.c (revision 344018) +++ /usr/src/sys/powerpc/powerpc/mp_machdep.c (working copy) @@ -97,6 +97,9 @@ /* Initialize decrementer */ decr_ap_init(); =20 + if (bootverbose) // HACK!!! + printf("machdep_ap_bootstrap before ap_boot_mtx lock: = AP CPU #%d launched\n", PCPU_GET(cpuid)); + /* Serialize console output and AP count increment */ mtx_lock_spin(&ap_boot_mtx); ap_awake++; @@ -109,6 +112,8 @@ =20 while(smp_started =3D=3D 0) ; + if (bootverbose) // HACK!!! + printf("machdep_ap_bootstrap after smp_started!=3D0: AP = CPU #%d launched\n", PCPU_GET(cpuid)); =20 /* Start per-CPU event timers. */ cpu_initclocks_ap(); @@ -238,10 +243,19 @@ =20 ret =3D platform_smp_start_cpu(pc); if (ret =3D=3D 0) { + if (bootverbose) // HACK!!! + printf("cpu_mp_unleash = attempting to wait for pc_awake: CPU %d, hwref=3D%jx, awake=3D%x\n", + pc->pc_cpuid, = (uintmax_t)pc->pc_hwref, + pc->pc_awake); + timeout =3D 2000; /* wait 2sec for = the AP */ while (!pc->pc_awake && --timeout > 0) DELAY(1000); } + if (bootverbose) // HACK!!! + printf("cpu_mp_unleash after = platform_smp_start_cpu and waiting: CPU %d, hwref=3D%jx, awake=3D%x\n", + pc->pc_cpuid, = (uintmax_t)pc->pc_hwref, + pc->pc_awake); } else { pc->pc_awake =3D 1; } @@ -252,7 +266,13 @@ pc->pc_awake); smp_cpus++; } else + { // HACK!!! + if (bootverbose) // HACK!!! + printf("cpu_mp_unleash adding pc_cpuid = to stopped_cpus: CPU %d (dev=3D%x)\n", + pc->pc_cpuid, (int)pc->pc_hwref); CPU_SET(pc->pc_cpuid, &stopped_cpus); + } // HACK!!! + } =20 ap_awake =3D 1; =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)