Date: Sat, 4 May 2019 01:35:46 -0700 From: Mark Millard <marklmi@yahoo.com> To: Justin Hibbits <chmeeedalf@gmail.com>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: The first 2 handle_kernel_slb_spill calls on the 2-socket/2-cores-each G5 example context: as expected? (short) Message-ID: <9B5E32D1-1A91-432C-A9B5-1A2CF628A8B0@yahoo.com> In-Reply-To: <47414D54-99F2-490C-AFFC-43503556FA4A@yahoo.com> References: <9D9A51A9-C8A6-475F-B241-0A3C3546D3D6.ref@yahoo.com> <9D9A51A9-C8A6-475F-B241-0A3C3546D3D6@yahoo.com> <2388C664-2D47-4851-95AF-A125CE48C282@yahoo.com> <47414D54-99F2-490C-AFFC-43503556FA4A@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[I'm just showing the code that got the handle_kernel_slb_spill
reports.]
On 2019-May-4, at 00:03, Mark Millard <marklmi at yahoo.com> wrote:
> [I forgot to show where I always stop the enable of the
> reporting.]
>=20
> On 2019-May-3, at 23:52, Mark Millard <marklmi at yahoo.com> wrote:
>=20
>> [A correction --and interesting information from a somewhat later
>> time frame.]
>>=20
>> On 2019-May-3, at 20:22, Mark Millard <marklmi at yahoo.com> wrote:
>>=20
>>> [This is from the -r347003 experiment context, not my
>>> normal environment.]
>>>=20
>>> I stuck a printf in handle_kernel_slb_spill, reporting the type,
>>> dar, and srr0 values. The resultant build does not get far
>>> booting but does report the first 2 calls. Typed from a screen
>>> picture:
>>>=20
>>> KDB: debugger backends: ddb
>>> KDB: current backend: ddb
>>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc
>>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x10000000 srr0=3D0xa869bc=
>>>=20
>>> That is as far as it gets, as far as output goes, with that
>>> unconditional printf in place.
>>>=20
>>> (I was not sure I'd get anything from this experiment.)
>>>=20
>>> This suggests that the slb is partially(?) populated in the
>>> hardware before the (adjusted) loop that I've been testing with
>>> tries to establish coverage of part of the KVA space. The two
>>> examples reported are from neither the Direct-Map space nor the
>>> Kernel-Virtual-Address space.
>>>=20
>>> Are these expected? Is their presence handled?
>>>=20
>>=20
>> I made the printf in handle_kernel_slb_spill conditional
>> on a global so I could control when it would try to
>> print.
>>=20
>> I learned that I guessed the ordering wrong on the initial
>> report:
>>=20
>> QUOTE
>> #ifdef __powerpc64__
>> i =3D 0;
>> for (va =3D virtual_avail; va < virtual_end && i<(n_slbs-1)/2; =
va +=3D SEGMENT_LENGTH, i++)
>> moea64_bootstrap_slb_prefault(va, 0);
>> #endif
>> enable_handle_kernel_slb_spill_reporting=3D 1;
>> END QUOTE
>>=20
>> gets the lines I originally showed:
>>=20
>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc
>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x10000000 srr0=3D0xa869bc
>>=20
>> So these were after then loop, not before.
>>=20
>> Note: So far those messages always have displayed and
>> then things were hung-up for this enable placement.
>>=20
>>=20
>> I then commented that enable out and added a
>> printf:
>>=20
>> pa =3D moea64_bootstrap_alloc(kstack_pages * PAGE_SIZE, =
PAGE_SIZE);
>> va =3D virtual_avail + KSTACK_GUARD_PAGES * PAGE_SIZE;
>> virtual_avail =3D va + kstack_pages * PAGE_SIZE;
>> CTR2(KTR_PMAP, "moea64_bootstrap: kstack0 at %#x (%#x)", pa, =
va);
>> printf("moea64_bootstrap: kstack0 at %#x (%#x)\n", pa, va);
>>=20
>> and also set up an enable just before dpcpu_init's=20
>> all:
>>=20
>> enable_handle_kernel_slb_spill_reporting=3D 1;
>> dpcpu_init(dpcpu, curcpu);
>>=20
>> The result, when it did not boot, was as below,
>> again showing a couple of handle_kernel_slb_spill
>> lines for a not very large addresses and no more
>> lines after that:
>>=20
>> KDB: debugger backends: ddb
>> KDB: current backend: ddb
>> moea64_bootstrap: kstack0 at 0x3000 (0x1000)
>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x22ef8 srr0=3D0xa86690
>> handle_kernel_slb_spill: type=3D0x480 dar=3D0x22ef8 srr0=3D0xa86690
>>=20
>> It is the same addresses but two distinct types. It
>> also would seem to be the same segment as for:
>>=20
>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc
>> (from the earlier placement)
>>=20
>>=20
>> By contrast, interestingly, it did sometimes boot for
>> this later enable placement, and, when it did boot,
>> there were no handle_kernel_slb_spill lines output:
>>=20
>> KDB: debugger backends: ddb
>> KDB: current backend: ddb
>> moea64_bootstrap: kstack0 at 0x3000 (0x1000)
>> ---<<BOOT>>---
>>=20
>> (and so on.)
>>=20
>>=20
>> This means that the type=3D0x?80 dar=3D0x22ef8 srr0=3D0xa86690
>> slb-misses are intermittent for this testing context.
>>=20
>>=20
>> Of course, with more testing I might see more variability.
>=20
>=20
> I forgot to show that I used:
>=20
> /* Bring up virtual memory */
> moea64_late_bootstrap(mmup, kernelstart, kernelend);
> enable_handle_kernel_slb_spill_reporting=3D 0; // hangs without printf =
display first when this late
> }
>=20
> It did no good the enable it this late so I set
> it as a disable point instead. Trying to use the
> handle_kernel_slb_spill printf after this point
> seem to just result in silently hanging-up.
>=20
> So this disable was involved in the cases that
> booted for enabling just before dpcpu_init .
> (It is not clear just how far the non-booting
> cases got internally.)
For:
handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc
handle_kernel_slb_spill: type=3D0x380 dar=3D0x10000000 srr0=3D0xa869bc
both seem to involve the stbx instruction in:
0000000000a869bc <.memset+0x20> stbx r4,r9,r3
0000000000a869c0 <.memset+0x24> addi r9,r9,1
0000000000a869c4 <.memset+0x28> bdnz 0000000000a869bc <.memset+0x20>
For:
handle_kernel_slb_spill: type=3D0x380 dar=3D0x22ef8 srr0=3D0xa86690
handle_kernel_slb_spill: type=3D0x480 dar=3D0x22ef8 srr0=3D0xa86690
both seem to involve the stdu instruction in:
0000000000a8668c <.memcpy+0x140> ldu r0,-8(r9)
0000000000a86690 <.memcpy+0x144> stdu r0,-8(r11)
0000000000a86694 <.memcpy+0x148> bdnz 0000000000a8668c =
<.memcpy+0x140>
although the first is for a EXC_DSE (Data Segment Exception)
and the second for a EXC_ISE (Instruction Segment Exception).
The effective addresses reported for srr0 seem to match
what objdump shows for the kernel file.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9B5E32D1-1A91-432C-A9B5-1A2CF628A8B0>