From owner-freebsd-ppc@freebsd.org Thu Feb 14 23:27:07 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DD8C14E638D for ; Thu, 14 Feb 2019 23:27:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-4.consmr.mail.bf2.yahoo.com (sonic307-4.consmr.mail.bf2.yahoo.com [74.6.134.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 41C5F6B4D4 for ; Thu, 14 Feb 2019 23:27:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: fN52r4kVM1k7miSZpz633cVXZCwLkf.xlcGB6vhD36DIcQegMBTBCIKFzjXw5tK 4ID3c4fpmB30mjEg_ZfC25vLKqKMhbjWoeKrJYgz8xjEwiq7mXDwsH0Op4F4mvGe8lU6e6XGhmI0 HyDvteBike9DFKdcbxoj0qEZZ.QutqUP7VyLIwc7sOMPTU6St4DzI.RCz4aztEdeOLyY.E9ASvPd P0E1Lrxnh_IJ4EF5TLyMCh2uXQSodxASNhOikpwj63fklyC0N9MKYl3DAmxFIQOmHQsRDZsWFtVN nsndhtQccEi8mPZVHPGrJQC.QkR3ZqweJCr4Tw9MW3CMWQ_FD4rWUAeWViBAQU9c7ifwI.rOLeqv _lwWJQl0GYKnO1eudgiWPB22oQy8htxFUpDBtdRhHRdqPtmYpDkF_Q_cYynnOOpLEHm0TdwZgRn1 ZWWZoMdFCS86KloRAIWY1ohF7kgfftYwKZIGP90wVpEpd4nj1YZUREY8XD1pMtV2tv7jXxaKvlNy j4GN7dJTwuVbDMAHMdgjp2XmENt9dzJdNTARQYG6PiFfWzIBCZXzQQI6_Li9pXf25J.0YeAYadtc PGTkgSbF.W.KD5OAN95klBKtQQKUji7yY1uyDS7Z2EQ6gr7vKpY.7Xuf3hg7ZTJWWHtR774z7shS 94p5P9PQo646osworMqR.r62ZOma0PD0rc3bqiQyaZzDFwPJQAoK.5EARCNlAbPxv6eE2ab155Eg tINDI3ez2q.lvOI1ajm9uXmO_iDuOaJtwTEJnKQSpzErthw6jqjHHDvO1oPJCJ0mI9Fc5vAfktl2 uhhI.FPUB6QnvaC7QMb4CnFTXM0VSkV0exE.BzKIy6gbynIu6dCWH68e0SEBQYuzHOeGw4TfQj2h 83x0lzxzZ3gn8.uyJJlMWEfQYawjLNCypR4XCSjhCqL1O8ueZCUqnvfRNGLrNBaqwo3C18a64of1 geD05o.P6boFf76ofOEXMRhuiqwNmHZT3hnvPgUWKWp7dNY_Dpo4fYlvIV9z5ht2YS2osKX5PXkj 0pIM9I.bYCyGhlwStLeH.ZuUAQC9nlK.XWPbnm7hQWzeqZ9IE4oRgUS0y2NdEbA-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.bf2.yahoo.com with HTTP; Thu, 14 Feb 2019 23:27:04 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp406.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 70c9c53440250f58ffce88c08747e2c7; Thu, 14 Feb 2019 23:16:55 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Some evidence about the PowerMac G5 multiprocessor boot hang ups with the modern VM_MAX_KERNEL_ADDRESS value From: Mark Millard In-Reply-To: Date: Thu, 14 Feb 2019 15:16:53 -0800 Cc: freebsd-ppc@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <11680D15-D43D-4115-AF4F-5F6E4E0022C9@yahoo.com> To: Dennis Clarke X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 41C5F6B4D4 X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.21 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; URIBL_BLOCKED(0.00)[dsl-only.net.multi.uribl.com]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.87)[0.867,0]; NEURAL_HAM_LONG(-0.10)[-0.096,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.23)[ip: (3.79), ipnet: 74.6.128.0/21(1.36), asn: 26101(1.09), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.72)[0.717,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[43.134.6.74.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 23:27:07 -0000 On 2019-Feb-14, at 12:56, Dennis Clarke = wrote: > On 2/13/19 10:35 PM, Mark Millard via freebsd-ppc wrote: >> I added some bootverbose messages to: >=20 > I clearly need to apply your "HACK" verbosity because I am seeing odd > behavior where I now need to set kern.smp.disabled=3D1 to get a boot = AND > then I get all four cores online. >=20 > Here is the /var/run/dmesg.boot with some 'malloc' lines snipped out. >=20 > Everything comes to a screetching halt at vt_upgrade(&vt_consdev) = unless I set kern.smp.disabled=3D1 and then I get a normal boot. >=20 > . . . Even without my additions, using boot -v at the loader prompt would echo more information, including during: subsystem f000000 cpu_mp_unleash(0)... done. (where I get a hangup much of the time). boot -v is independent of debug.verbose_sysinit=3D1 : they can be together or separately. Looks like I did not include the svnlite diff output as of my adding the address information and such. But I only added to the cpu_mp_unleash related activity overall. My updates svnlite diff is as follows: # svnlite diff /usr/src/sys/powerpc/powermac/platform_powermac.c = /usr/src/sys/powerpc/powerpc/mp_machdep.c = Index: = /usr/src/sys/powerpc/powermac/platform_powermac.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powermac/platform_powermac.c (revision = 344018) +++ /usr/src/sys/powerpc/powermac/platform_powermac.c (working copy) @@ -344,6 +344,10 @@ =20 cpu =3D pc->pc_hwref; res =3D OF_getprop(cpu, "soft-reset", &reset, sizeof(reset)); + if (bootverbose) // HACK!!! + printf("powermac_smp_start_cpu 's OF_getprop for CPU = %d, hwref=3D%jx, awake=3D%x: res=3D%d, reset=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake, res, reset); if (res < 0) { reset =3D 0x58; =20 @@ -363,14 +367,33 @@ default: return (ENXIO); } + if (bootverbose) // HACK!!! + printf("powermac_smp_start_cpu for res<0 for CPU = %d, hwref=3D%jx, awake=3D%x: res=3D%d, reset=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake, res, reset); } =20 ap_pcpu =3D pc; =20 if (rstvec_virtbase =3D=3D NULL) + { // HACK!!! rstvec_virtbase =3D pmap_mapdev(0x80000000, PAGE_SIZE); + if (bootverbose) // HACK!!! + printf("powermac_smp_start_cpu for CPU %d, = hwref=3D%jx, awake=3D%x: rstvec_virtbase=3D%p\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake, rstvec_virtbase); + } // HACK!!! =20 rstvec =3D rstvec_virtbase + reset; + if (bootverbose) // HACK!!! + printf("powermac_smp_start_cpu for CPU %d, hwref=3D%jx, = awake=3D%x: rstvec=3D%p\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake, rstvec); +=20 + if (bootverbose) // HACK!!! + printf("Before reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake); =20 *rstvec =3D 4; powerpc_sync(); @@ -382,10 +405,20 @@ (void)(*rstvec); powerpc_sync(); =20 + if (bootverbose) // HACK!!! + printf("After reset 4&0 for CPU %d, hwref=3D%jx, = awake=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake); + timeout =3D 10000; while (!pc->pc_awake && timeout--) DELAY(100); =20 + if (bootverbose) // HACK!!! + printf("After attempted wait for awake CPU %d, = hwref=3D%jx, awake=3D%x\n", + pc->pc_cpuid, (uintmax_t)pc->pc_hwref, + pc->pc_awake); + return ((pc->pc_awake) ? 0 : EBUSY); #else /* No SMP support */ Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/powerpc/powerpc/mp_machdep.c (revision 344018) +++ /usr/src/sys/powerpc/powerpc/mp_machdep.c (working copy) @@ -97,6 +97,9 @@ /* Initialize decrementer */ decr_ap_init(); =20 + if (bootverbose) // HACK!!! + printf("machdep_ap_bootstrap before ap_boot_mtx lock: = AP CPU #%d launched\n", PCPU_GET(cpuid)); + /* Serialize console output and AP count increment */ mtx_lock_spin(&ap_boot_mtx); ap_awake++; @@ -109,6 +112,8 @@ =20 while(smp_started =3D=3D 0) ; + if (bootverbose) // HACK!!! + printf("machdep_ap_bootstrap after smp_started!=3D0: AP = CPU #%d launched\n", PCPU_GET(cpuid)); =20 /* Start per-CPU event timers. */ cpu_initclocks_ap(); @@ -238,10 +243,19 @@ =20 ret =3D platform_smp_start_cpu(pc); if (ret =3D=3D 0) { + if (bootverbose) // HACK!!! + printf("cpu_mp_unleash = attempting to wait for pc_awake: CPU %d, hwref=3D%jx, awake=3D%x\n", + pc->pc_cpuid, = (uintmax_t)pc->pc_hwref, + pc->pc_awake); + timeout =3D 2000; /* wait 2sec for = the AP */ while (!pc->pc_awake && --timeout > 0) DELAY(1000); } + if (bootverbose) // HACK!!! + printf("cpu_mp_unleash after = platform_smp_start_cpu and waiting: CPU %d, hwref=3D%jx, awake=3D%x\n", + pc->pc_cpuid, = (uintmax_t)pc->pc_hwref, + pc->pc_awake); } else { pc->pc_awake =3D 1; } @@ -252,7 +266,13 @@ pc->pc_awake); smp_cpus++; } else + { // HACK!!! + if (bootverbose) // HACK!!! + printf("cpu_mp_unleash adding pc_cpuid = to stopped_cpus: CPU %d (dev=3D%x)\n", + pc->pc_cpuid, (int)pc->pc_hwref); CPU_SET(pc->pc_cpuid, &stopped_cpus); + } // HACK!!! + } =20 ap_awake =3D 1; (The above may not preserve all whitespace detail.) I'll note that I did some experimenting with a system-clang based kernel build, used with devel/powerpc64-binutils , avoiding loading kernel modules dynamically. I did not have any boot problems but I'd not done enough activity to infer too much. My normal builds are based on devel/powerpc64-xoolchain-gcc related tools, including devel/powerpc64-gcc . =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)