From owner-freebsd-ppc@freebsd.org Sun Feb 24 21:50:32 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 075B8150E4FE for ; Sun, 24 Feb 2019 21:50:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-22.consmr.mail.ne1.yahoo.com (sonic305-22.consmr.mail.ne1.yahoo.com [66.163.185.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F29898A365 for ; Sun, 24 Feb 2019 21:50:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 3UfgHdEVM1lMEREpk9SCDMvWf.VrVBe9PSSTCcstSwzvnHAyLML3EC5ftFOLuSQ Q_2SZsKEvUW96VAQv4tECpAwsd1x_pNjnpJjKlil.653kHGMdrGkUKeYDIfVkAHJRtK5qYJpiEZ1 FvH.yamJl8pCXR.eJCI0CZoo2WPh_OSmn3Rpj9SsKI7HzLEVuUUZvdcO6rwp1CUtiAM_YVkMQnwU .TN.uQCtT6IME1KTOJ3VXt8S0isk_ba0s.T_7eV1ihsNXGGm8XIX7bxenqtonbyzdsJVpHK7FLT0 ZEJzLxjbKDwYdesVqVmg2pLJt3S_RgPIzYrNc8D5eWiyd3LavsUyIjusl8SJnDKsoxQpBGqyV2iX sOj2kv65tMPiiahg_xrn5X.Thxl1UOzeZm4YfjdQK3yU6poSuNLdMRTb0VuPePFMM9ZN8Zd0kDVm TqEYX3aA9UXCo.ApAHXyxEor.wzKwHiQdId7R7uo4p7q4QD2kT5WjQxpdKClYDVWu7eI8n053I08 95lGbPFgRhYuHEKfJ1fSWqdQi7dyqeYF1Mb_96N3ujSSDJ3mkxUjh2y9eC5Hw3EkyxLW9qZyHsCo WJJ7yuI2JFS9tnQQrrv6FRKJ8q8kVOzizARJWiy8KCpJxBczcgarC6CBn6hyOb1.nIUrDMWOamsd DriEGTdR65MLRo49gReW6DekdOks_Wko3BjVXpHH2vjWxMToL5SgzUC5gsyd3MVnmyRh60q22phi voldyw9bHZZXCaw7g_SmEKLqgHEqnOMlLVmVcnn1sn1exJpiTLwm5GemuGF7WZzECFZJgdlXtAGX G_.XrsNKSnovP_0iRIdnffm76ENEK9AGQ4nZhfDNf_6gewr0fpGQ6aoJB9NlA0XCTiBwVF6m5qVv pcdNLEhHAHN28tIHJanTMN7_zWWakQuCoRwRdxADo2Yp4efdtgkHtXZtBssKe29uaULxvSmne_Ly te6rbhNdH__vuu8aDDDj9wwwcPGvE2Vbj6hVaA6rp3ap9AL7C9tKdyQ2g9NJ4GFOswseCY.WSk14 FHp7IfffeUZ0TwPaKjWF1PNTWuMdUEdzPnfH4CzB3B9RGsh96qCz1yc4GnM4ML.s- Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.ne1.yahoo.com with HTTP; Sun, 24 Feb 2019 21:50:22 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp413.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 6e72ba45aad1c5b1c3a4c1c2479e17e0; Sun, 24 Feb 2019 21:50:19 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: An experimental hack that appears to allow old PowerMacG5 4-core (system total) system to boot reliably (head -r343884 based context) From: Mark Millard In-Reply-To: Date: Sun, 24 Feb 2019 13:50:17 -0800 Cc: FreeBSD PowerPC ML , Dennis Clarke Content-Transfer-Encoding: quoted-printable Message-Id: <466B6E08-5631-41FB-A1FD-263C27519F65@yahoo.com> References: To: Justin Hibbits X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: F29898A365 X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.13 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.99)[0.993,0]; NEURAL_HAM_LONG(-0.61)[-0.610,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.43)[ip: (4.89), ipnet: 66.163.184.0/21(1.28), asn: 36646(1.03), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.83)[0.829,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[148.185.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Feb 2019 21:50:32 -0000 On 2019-Feb-24, at 13:07, Justin Hibbits = wrote: > On Sat, Feb 23, 2019 at 1:36 PM Mark Millard = wrote: >>=20 >> For sys/powerpc/aim/mp_cpudep.c 's cpudep_ap_bootstrap I added as = shown below: >>=20 >> +extern void hack_into_slb_if_needed(void* vap); // HACK!!! >> + >> uintptr_t >> cpudep_ap_bootstrap(void) >> { >> . . . >> + hack_into_slb_if_needed(pcpup->pc_curpcb); // HACK!!! >> + >> sp =3D pcpup->pc_curpcb->pcb_sp; >>=20 >> and in src/sys/powerpc/aim/slb.c I added an implementation: >>=20 >> +void hack_into_slb_if_needed(void* vap); // HACK!!! >> +void hack_into_slb_if_needed(void* vap) // HACK!!! >> +{ // HACK!!! >> + struct slb *cache=3D PCPU_GET(aim.slb); >> + vm_offset_t va=3D (vm_offset_t)vap; >> + uint64_t slbv=3D kernel_va_to_slbv(va); >> + uint64_t esid=3D va>>ADDR_SR_SHFT; >> + uint64_t slbe=3D (esid<> + int i; >> + >> + for (i =3D 0; i < n_slbs; i++) { >> + if (i =3D=3D USER_SLB_SLOT) >> + continue; >> + if (cache[i].slbe =3D=3D (slbe | i)) >> + break; >> + } >> + >> + if (i=3D=3Dn_slbs) >> + slb_insert_kernel(slbe,slbv); >> +} // HACK!!! >> + >>=20 >> So far I've not had any boot hang-ups after this. >>=20 >> Given the random nature of the hang-ups it will be a >> while before I conclude for sure how reliable this >> change makes booting, but so far so good. >>=20 >> (I recognize that the "break" could be "return" >> and then then the "if (i=3D=3Dn_slbs)" would not be >> needed.) >>=20 >>=20 >> Other issues not fixed by this: >>=20 >> This does not change the buf*daemon* randomly getting >> hung up (and so timing out on shutdown). This appears >> to be the same issue that leads to the fans sometimes >> starting to run full-rate because of pmac_thermal >> being hun -up. >>=20 >> For buf*daemon* "top -SHIopid" before shutdown shows >> just the ones that will not hang-up. The same goes for >> seeing before hand for pmac_thermal vs. the fans. >>=20 >> =3D=3D=3D >> Mark Millard >=20 > Hi Mark, >=20 > Fantastic work tracking this down! So the problem is we now can fault > when accessing KVA space. I think we should allow this, otherwise we > can hamper performance with reduced KVA size. I'll have to think > about how best to do this. Would you be willing to test patches I > come up with? I'll try to test whatever updates you want but there may be some issues with timeliness. The reason for the "sometimes" boot-failure is that the entry in the slb for the PCB/stack for the CPU being added has sometimes been replaced already before the CPU the pcb is for has sufficiently configured to allow automatic handling --and other times has not yet been replaced: the random slb replacement mechanism. There already is code to handle slb entry replacements but it does not work for a CPU still being set up (at the stage of the sometimes failure). At least that is what I expect for: # grep -r "handle_kernel_slb_spill" /usr/src/sys/powerpc/ /usr/src/sys/powerpc/aim/trap_subr64.S: bl handle_kernel_slb_spill /usr/src/sys/powerpc/powerpc/trap.c: void = handle_kernel_slb_spill(int, register_t, register_t); /usr/src/sys/powerpc/powerpc/trap.c:handle_kernel_slb_spill(int type, = register_t dar, register_t srr0) So my hack was to separately do the potential replacement in that early time frame to allow the configuration for the CPU to get far enough along for the existing mechanism to work. (At least that is what I expect that I did.) So far I've had no boot failures of any kind with the hack. I've removed the hacks for reporting information and things still work. But I've not tried anything extensive after booting because things like buf*daemon* threads and pmac_thermal are randomly hanging up in/at: mi_switch+0x134 sleepq_switch+0x2ec sleepq_timedwait+0x48 _sleep+0x41c (mi_swtich seems to have called sched_switch based on the "+0x134" and the code in that area --but ched_switch is not listed) I've no clue what is safe when one or more buf*daeomon* threads make no progress. For shutdown that frequently leads to timeouts for stopping some buf*deamon* threads (when all 8 time out it takes about 8 minutes). The buf*deamon* that fail are the ones that "top -SHIopid" no longer shows. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)