Date: Wed, 1 May 2019 15:10:52 -0500 From: Justin Hibbits <chmeeedalf@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: How many segments does it take to span from VM_MIN_KERNEL_ADDRESS through VM_MAX_SAFE_KERNEL_ADDRESS? 128 in moea64_late_bootstrap Message-ID: <20190501151052.6bc834b0@titan.knownspace> In-Reply-To: <212E50E5-7EB1-4381-A662-D5EACB1E5D46@yahoo.com> References: <3C69CF7C-7F33-4C79-92C0-3493A1294996@yahoo.com> <6159F4A6-9431-4B99-AA62-451B8DF08A6E@yahoo.com> <20190501094029.542c5f46@titan.knownspace> <212E50E5-7EB1-4381-A662-D5EACB1E5D46@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 1 May 2019 11:51:44 -0700 Mark Millard <marklmi@yahoo.com> wrote: > On 2019-May-1, at 07:40, Justin Hibbits <chmeeedalf at gmail.com> > wrote: > > > On Tue, 30 Apr 2019 21:45:00 -0700 > > Mark Millard <marklmi@yahoo.com> wrote: > > > >> [I realized another implication about a another point of > >> potential slb-misses in cpudep_ap_bootstrap: the > >> address in sprg0 on the cpu might end up not able to be > >> dereferenced.] > >> > >> On 2019-Apr-30, at 20:58, Mark Millard <marklmi at yahoo.com> > >> wrote: > >>> [At the end this note shows why the old VM_MAX_KERNEL_ADDRESS > >>> lead to no slb-miss exceptions in cpudep_ap_bootstrap.] > >>> > >>> There is code in moea64_late_bootstrap that looks like: > >>> > >>> virtual_avail = VM_MIN_KERNEL_ADDRESS; > >>> virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; > >>> > >>> /* > >>> * Map the entire KVA range into the SLB. We must not fault > >>> there. */ > >>> #ifdef __powerpc64__ > >>> for (va = virtual_avail; va < virtual_end; va += > >>> SEGMENT_LENGTH) moea64_bootstrap_slb_prefault(va, 0); > >>> #endif > > > > What happens if you revert all your patches, > > Most of the patches in Bugzilla 233863 are not for this > issue at all and are not tied to starting the non-bsp > cpus. (The one for improving how close the Time Base > registers are is tied to starting these cpus.) Only the > aim/mp_cpudep.c and aim/slb.c changes seem relevant. > > Are you worried about some form of interaction that means > I need to avoid patches for other issues? > > Note: for now I'm staying at using head -r345758 as the > basis for my experiments. > > > and change this loop to > > stop at n_slb? So something more akin to: > > > > int i = 0; > > > > for (va = virtual_avail; va < virtual_end && i < n_slb - 1; > > va += SEGMENT_LENGTH, i++); > > ... > > > > If it reliably boots with that, then that's fine. We can prefault > > as much as we can and leave the rest for on-demand. > > I'm happy to experiment with this loop without my hack > for forcing the slb entry to exist in cpudep_ap_bootstrap. > > But, it seems to presume that the pc_curpcb's will > all always point into the lower address range spanned > when cpudep_ap_bootstrap is executing on the cpu. > Does some known property limit the pc_curpcb-> > references to such? Only that would be sure to > avoid an slb-miss at that stage. Or is this just an > alternate hack or a means of getting evidence, not a > proposed solution? > > (Again, I'm happy to disable my hack that forces the > slb entry and to try the loop suggested.) The idea with this is if you can test with stock -CURRENT (or post-VM_KERNEL_MAXADDR change), to eliminate any other variables. This is *only* for testing that it brings up the APs, not that they're properly synced. That will happen with other changes. This is a proposed solution. From my understanding, we typically allocate from low to high for KVA allocations, so keeping the low addresses in memory long enough to bring up the APs to sanity is the goal, so the commit would be along the lines of "Prefault as much of KVA as we can fit into the SLB". Thanks for lending your G5(s) for this testing. I do want to get the G5 working correctly again, since they're so plentiful and inexpensive. - Justin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190501151052.6bc834b0>