From owner-freebsd-ppc@freebsd.org Thu May 2 00:23:06 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E0F415824A9 for ; Thu, 2 May 2019 00:23:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-33.consmr.mail.ne1.yahoo.com (sonic317-33.consmr.mail.ne1.yahoo.com [66.163.184.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E46466CEE3 for ; Thu, 2 May 2019 00:23:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: C7Yn6B4VM1kpllsz14q_JSzXJK9TAg.eLotK4EZZ26X6XSh97CDbCp8jDmWab5P fB0_MEqM3rKvIsuu7hxMtZuY4KSVwtB2sMe_E_AVntxA8WVOF4dQVpHPPu0Tx6tEJcaPr4.31OFx uYS1.TzKqfWB19_jC8AFyStt_5GcHxFSr10t3guzcFa_4Qyh3nNwOd1fs1c786criPDBelwKSm.x Se0j33BrdsJYOE5ti5cUFBe14C5NQOvm1l6lItrr1SLsrJZmubsWCm9LSExfOXu5V39btO0C5Z8Z gxE72ZjQr126OVWUvm06HcsyBB_Lsjc5QA0AEjWruluKs8ovi2xdv2X43wUDgWSIORd4hyXtVHDt 27XfXYBp7BLd4brVIM2dcnWhYoaA0QXDeARMWBqnfoUu8dNKR0js6agDBhRV2RBc42A5TQa.SXJq FmzwHALwYWxmIMAIWBV8lWUpY1g8pxjxJSoPq4BykuTy0Aba_iK4SHKg2CCmaE1E8Irh_1hZbM0H nR9cXWfFh24eSCSB7mZ1HNWxeVzz.ZQy5oXIWWcXyBWsF9sLyDX7MwUjpa5b8HBpUZ7O8e4VaaJg 2QQ.F60SlnQtH6w1T0ODvflQA7YukAqknHHVkcRmlEy1mJrGQn9yzAybyxjhtxLWaM.SCYmWTizb 3vKzpvJvAqET0bDOoX42xPUXhc95iI5Nb7cw9vojKftjVECw6YW_Ne4nDrMumf33KDhLLAeAR3ch Ze4ncP3pHCncrIRV33QNzcstPFPSbdVeJw4eifoSUgLMeeaYBDITES2CDgZUXvIpG0lVKrPvTu4b BoBvrd8_3U_mtL1ErP_JAjCDoOi0w56.X1SmggmmhByndD3nrOxHsZ3uGCD0_gY52XCM57Eb1_de QM9EJ2Y.P8Y8Iiprz.2.wRnX2E3SAUrs7xlOjb_AvJOudKZJ2P1uSGyCEZwUQFtHmHrsoxkzWGz9 sNsfmeO6.IAkLuoEg__CsTsL.8enp8QfhK8Uj2dLHrRQakb0_nsmjyYHIoYPveIjQOixLXvEm23. vXMH4JeDCLx0WYHafYo2.TJesL9V7y2TACh5YUoM0oQz8_XFscNUy.bTdVvcC1w9MkdGrE.5R7Ic Jrvgz6GsPgq1ltcjQAfEEmKu.iKJvC4eNzZ59kzXbiiXsE0iasZz5QagMu6aV5EIUARM6PxFAvS0 2yt9MwYF92So1 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Thu, 2 May 2019 00:23:02 +0000 Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.103]) ([76.115.7.162]) by smtp426.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID ba1aa267bb27d4dcd0c621f84e0efefb; Thu, 02 May 2019 00:22:58 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: How many segments does it take to span from VM_MIN_KERNEL_ADDRESS through VM_MAX_SAFE_KERNEL_ADDRESS? 128 in moea64_late_bootstrap From: Mark Millard In-Reply-To: <20190501165403.7d8d1f8f@titan.knownspace> Date: Wed, 1 May 2019 17:22:56 -0700 Cc: FreeBSD PowerPC ML Content-Transfer-Encoding: 7bit Message-Id: <1B8116F2-9749-4331-95BD-D528AA52A771@yahoo.com> References: <3C69CF7C-7F33-4C79-92C0-3493A1294996@yahoo.com> <6159F4A6-9431-4B99-AA62-451B8DF08A6E@yahoo.com> <20190501094029.542c5f46@titan.knownspace> <212E50E5-7EB1-4381-A662-D5EACB1E5D46@yahoo.com> <20190501165403.7d8d1f8f@titan.knownspace> To: Justin Hibbits X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: E46466CEE3 X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.74 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; NEURAL_SPAM_SHORT(0.57)[0.571,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(2.26)[ip: (8.88), ipnet: 66.163.184.0/21(1.38), asn: 36646(1.11), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.82)[0.817,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.60)[0.602,0]; RCVD_IN_DNSWL_NONE(0.00)[44.184.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 00:23:06 -0000 On 2019-May-1, at 14:54, Justin Hibbits wrote: > On Wed, 1 May 2019 14:35:56 -0700 > Mark Millard wrote: > >>>> What happens if you revert all your patches, >>> >>> Most of the patches in Bugzilla 233863 are not for this >>> issue at all and are not tied to starting the non-bsp >>> cpus. (The one for improving how close the Time Base >>> registers are is tied to starting these cpus.) Only the >>> aim/mp_cpudep.c and aim/slb.c changes seem relevant. >>> >>> Are you worried about some form of interaction that means >>> I need to avoid patches for other issues? >>> >>> Note: for now I'm staying at using head -r345758 as the >>> basis for my experiments. >>> >>>> and change this loop to >>>> stop at n_slb? So something more akin to: >>>> >>>> int i = 0; >>>> >>>> for (va = virtual_avail; va < virtual_end && i < n_slb - >>>> 1; va += SEGMENT_LENGTH, i++); >>>> ... >>>> >>>> If it reliably boots with that, then that's fine. We can prefault >>>> as much as we can and leave the rest for on-demand. >>> >>> I'm happy to experiment with this loop without my hack >>> for forcing the slb entry to exist in cpudep_ap_bootstrap. >>> >>> But, it seems to presume that the pc_curpcb's will >>> all always point into the lower address range spanned >>> when cpudep_ap_bootstrap is executing on the cpu. >>> Does some known property limit the pc_curpcb-> >>> references to such? Only that would be sure to >>> avoid an slb-miss at that stage. Or is this just an >>> alternate hack or a means of getting evidence, not a >>> proposed solution? >>> >>> (Again, I'm happy to disable my hack that forces the >>> slb entry and to try the loop suggested.) > ... >> And the patch for the loop looks like: >> >> virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; >> >> /* >> - * Map the entire KVA range into the SLB. We must not fault >> there. >> + * Map the lower-address part of the KVA range into the SLB. >> We must not fault there. */ >> #ifdef __powerpc64__ >> - for (va = virtual_avail; va < virtual_end; va += >> SEGMENT_LENGTH) >> + i = 0; >> + for (va = virtual_avail; va < virtual_end && i> += SEGMENT_LENGTH, i++) moea64_bootstrap_slb_prefault(va, 0); >> #endif >> > > Yep, that's the patch I was going for. > >> >> So I've built, installed, and have tested some: it did not go well >> overall. >> >> Using: >> >> OK set debug.verbose_sysinit=1 >> >> to show better context about where the hangs occur, shows: >> (Typed from a screen picture.) >> >> subsystem a800000 >> boot_run_interrupt_driven_config_hooks(0)... >> . . . (omitted) . . . >> done. >> vt_upgrade(&vt_consdev). . . >> >> The "vt_upgrade(&vt_consdev). . ." never says done when booting >> hangs with the above changes. >> >> Trying to boot a bunch of times did produce one >> completed boot, all 4 cpus working. Otherwise I'm >> using kernel.old to manage to complete a boot. >> >> I'll note that "vt_upgrade(&vt_consdev). . ." is where >> Dennis Clarke reported for the hangups that he was >> seeing, without any of my patches being available back >> then: 2019-Feb-14. > > Maybe try the commit that caused the problem back in July? r334498. > I'd already started down the path of getting materials from: https://artifact.ci.freebsd.org/snapshot/head/r347003/powerpc/powerpc64/ and putting them on a separate SSD that I sometimes use for artifact.ci or snapshot experiments. Also: checking out matching svn sources for -r347003 and then doing a buildworld buildkernel with a bootstrap gcc 4.2.1 compiler used. I'm verifying that I can build it before making the source changes for the kernel. The build is of a debug kernel (GENERIC64). The test buildworld is still in process. Let me know if this is insufficient for your purposes. I could revert to: https://artifact.ci.freebsd.org/snapshot/head/r334594/powerpc/powerpc64/ (There is no head/r334498/ and the first after that with a powerpc64/ is head/r334594/ .) For either head/r347003/ or head/r334594/ : Use of artifact materials allows using officially built files for every file but some specific file(s) that I replace. It also allows comparison/contrast of the behavior of the official files vs. when adjusted ones are substituted. Use of artifact-version materials also means that I know I'm using a vintage that actually built --and so I hope to avoid other problems getting in the way. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)