From owner-freebsd-ppc@freebsd.org Sat May 4 08:35:53 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E93815867A9 for ; Sat, 4 May 2019 08:35:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-10.consmr.mail.ne1.yahoo.com (sonic313-10.consmr.mail.ne1.yahoo.com [66.163.185.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 04AD077086 for ; Sat, 4 May 2019 08:35:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: zkhuTtIVM1mRV70APczORwGusWFmrWePS3TusR.iO3ORrw6_BzacQxC.SRKSn1F jBIhirpz58Mc52CXNvsmxt9O9AeA5E9EbktTTu.gUbgpMnsWta_qKdTkpSaKAxyigZJO6zJoJTMM SiQ4b4uy3YcgjtQ_sq7HV0hfAK236.9GFj1a7ERD9KwsKqoR3bKe6MIetpPAGnfEFYz1w9wHm6pH 6nJoxI075y_8ssUItqZxhoiIxH8rhIh2B_FQUXAsnPthcuEt7QWyrSCjc9xpIkQMY4EdW5SqLw06 kog6s1Ck5Pn9VwVZmmGm02faIxUt9js9oO76EQ0zoMMz1YxGdZ553DKY9lpWqS1Hn46agTHI5P3Y zOuYQNHne8TMs6d0nu5GilymVR03uKZxa9tFXMUJXbEjzaavjDLEVQXIiXWjT4DHmQExgp4hnoCr 0VbzhhAyTvjpw.FJCh52mztQ6_dDGifVLIr.82WbcpOaAdHMcWPQH4Y85kS_LhCxDv9ymum2FH4c pBGkmJBejswr66U82yZjtg_YN5bSZ8ChUM4vwqf6FQTOe_Fq1e5MZhJa.5G9KVmPlV5fuRs5Llif KjsekYG4Wb52yjQEZweQjWFS7DUSb69zzCJg3jgJq6XQAcXhVze8XG9m3wQTT..q.CBu2sj.k3rQ .wwsgb6v9d4MZ.je6gS07eyacRVs98dR4zRCV9hJeFWGjmCTVvvJ4L_ZVu1xCXTOPswlVR0MNawn RYTVlDI16oe78JTrsTWhWwiR2oQu8L0YZ6YZ5_SDYEhozVvrsjvidpmZufE0GKtksgNBfa_Oa43X 35IEKxH.TF7NVzC47mzAaBJaaSDOIZ6oFJgqCt2aBUFQsP8rouq_tQerqkNDJOyioIed2oMvGOkf QKCQyd3SqAy43PFF1HjYt4VzLKk7PlRQ6gKhih9icbyDoeIl_m2LmaoC9T6KST7rIUhM0Tt0b9vq hLHGd8O0bwgoGDOSXltphtSwxaqWlrwX1LXF8bADZyfy8AX0QCdIP7nMmt2z.t6QGRKERFknLinf tAFDpI3tvrsJEmK3saTGmoP5O_xkCk5KEPPAEoJa9ouNXKwyOWiO_SnpxfUG_VGvTXcLvQSa02K1 0ISzKbnjwXKksEAwL9lxfScN3V15gtk5EkjwEdXd77ln2IY0tvx2KbC7ebUVy8V5CjIrLESSl8.D 7s1lTTuVOnQ-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.ne1.yahoo.com with HTTP; Sat, 4 May 2019 08:35:50 +0000 Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.103]) ([76.115.7.162]) by smtp432.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 60650b50fd7b6de97bd5cdaa0958ee19; Sat, 04 May 2019 08:35:48 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: The first 2 handle_kernel_slb_spill calls on the 2-socket/2-cores-each G5 example context: as expected? (short) Date: Sat, 4 May 2019 01:35:46 -0700 References: <9D9A51A9-C8A6-475F-B241-0A3C3546D3D6.ref@yahoo.com> <9D9A51A9-C8A6-475F-B241-0A3C3546D3D6@yahoo.com> <2388C664-2D47-4851-95AF-A125CE48C282@yahoo.com> <47414D54-99F2-490C-AFFC-43503556FA4A@yahoo.com> To: Justin Hibbits , FreeBSD PowerPC ML In-Reply-To: <47414D54-99F2-490C-AFFC-43503556FA4A@yahoo.com> Message-Id: <9B5E32D1-1A91-432C-A9B5-1A2CF628A8B0@yahoo.com> X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 04AD077086 X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.66 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; NEURAL_SPAM_SHORT(0.71)[0.705,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.42)[ip: (4.70), ipnet: 66.163.184.0/21(1.37), asn: 36646(1.10), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.75)[0.751,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.30)[0.297,0]; RCVD_IN_DNSWL_NONE(0.00)[33.185.163.66.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[33.185.163.66.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 08:35:53 -0000 [I'm just showing the code that got the handle_kernel_slb_spill reports.] On 2019-May-4, at 00:03, Mark Millard wrote: > [I forgot to show where I always stop the enable of the > reporting.] >=20 > On 2019-May-3, at 23:52, Mark Millard wrote: >=20 >> [A correction --and interesting information from a somewhat later >> time frame.] >>=20 >> On 2019-May-3, at 20:22, Mark Millard wrote: >>=20 >>> [This is from the -r347003 experiment context, not my >>> normal environment.] >>>=20 >>> I stuck a printf in handle_kernel_slb_spill, reporting the type, >>> dar, and srr0 values. The resultant build does not get far >>> booting but does report the first 2 calls. Typed from a screen >>> picture: >>>=20 >>> KDB: debugger backends: ddb >>> KDB: current backend: ddb >>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc >>> handle_kernel_slb_spill: type=3D0x380 dar=3D0x10000000 srr0=3D0xa869bc= >>>=20 >>> That is as far as it gets, as far as output goes, with that >>> unconditional printf in place. >>>=20 >>> (I was not sure I'd get anything from this experiment.) >>>=20 >>> This suggests that the slb is partially(?) populated in the >>> hardware before the (adjusted) loop that I've been testing with >>> tries to establish coverage of part of the KVA space. The two >>> examples reported are from neither the Direct-Map space nor the >>> Kernel-Virtual-Address space. >>>=20 >>> Are these expected? Is their presence handled? >>>=20 >>=20 >> I made the printf in handle_kernel_slb_spill conditional >> on a global so I could control when it would try to >> print. >>=20 >> I learned that I guessed the ordering wrong on the initial >> report: >>=20 >> QUOTE >> #ifdef __powerpc64__ >> i =3D 0; >> for (va =3D virtual_avail; va < virtual_end && i<(n_slbs-1)/2; = va +=3D SEGMENT_LENGTH, i++) >> moea64_bootstrap_slb_prefault(va, 0); >> #endif >> enable_handle_kernel_slb_spill_reporting=3D 1; >> END QUOTE >>=20 >> gets the lines I originally showed: >>=20 >> handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc >> handle_kernel_slb_spill: type=3D0x380 dar=3D0x10000000 srr0=3D0xa869bc >>=20 >> So these were after then loop, not before. >>=20 >> Note: So far those messages always have displayed and >> then things were hung-up for this enable placement. >>=20 >>=20 >> I then commented that enable out and added a >> printf: >>=20 >> pa =3D moea64_bootstrap_alloc(kstack_pages * PAGE_SIZE, = PAGE_SIZE); >> va =3D virtual_avail + KSTACK_GUARD_PAGES * PAGE_SIZE; >> virtual_avail =3D va + kstack_pages * PAGE_SIZE; >> CTR2(KTR_PMAP, "moea64_bootstrap: kstack0 at %#x (%#x)", pa, = va); >> printf("moea64_bootstrap: kstack0 at %#x (%#x)\n", pa, va); >>=20 >> and also set up an enable just before dpcpu_init's=20 >> all: >>=20 >> enable_handle_kernel_slb_spill_reporting=3D 1; >> dpcpu_init(dpcpu, curcpu); >>=20 >> The result, when it did not boot, was as below, >> again showing a couple of handle_kernel_slb_spill >> lines for a not very large addresses and no more >> lines after that: >>=20 >> KDB: debugger backends: ddb >> KDB: current backend: ddb >> moea64_bootstrap: kstack0 at 0x3000 (0x1000) >> handle_kernel_slb_spill: type=3D0x380 dar=3D0x22ef8 srr0=3D0xa86690 >> handle_kernel_slb_spill: type=3D0x480 dar=3D0x22ef8 srr0=3D0xa86690 >>=20 >> It is the same addresses but two distinct types. It >> also would seem to be the same segment as for: >>=20 >> handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc >> (from the earlier placement) >>=20 >>=20 >> By contrast, interestingly, it did sometimes boot for >> this later enable placement, and, when it did boot, >> there were no handle_kernel_slb_spill lines output: >>=20 >> KDB: debugger backends: ddb >> KDB: current backend: ddb >> moea64_bootstrap: kstack0 at 0x3000 (0x1000) >> ---<>--- >>=20 >> (and so on.) >>=20 >>=20 >> This means that the type=3D0x?80 dar=3D0x22ef8 srr0=3D0xa86690 >> slb-misses are intermittent for this testing context. >>=20 >>=20 >> Of course, with more testing I might see more variability. >=20 >=20 > I forgot to show that I used: >=20 > /* Bring up virtual memory */ > moea64_late_bootstrap(mmup, kernelstart, kernelend); > enable_handle_kernel_slb_spill_reporting=3D 0; // hangs without printf = display first when this late > } >=20 > It did no good the enable it this late so I set > it as a disable point instead. Trying to use the > handle_kernel_slb_spill printf after this point > seem to just result in silently hanging-up. >=20 > So this disable was involved in the cases that > booted for enabling just before dpcpu_init . > (It is not clear just how far the non-booting > cases got internally.) For: handle_kernel_slb_spill: type=3D0x380 dar=3D0x3d99348 srr0=3D0xa869bc handle_kernel_slb_spill: type=3D0x380 dar=3D0x10000000 srr0=3D0xa869bc both seem to involve the stbx instruction in: 0000000000a869bc <.memset+0x20> stbx r4,r9,r3 0000000000a869c0 <.memset+0x24> addi r9,r9,1 0000000000a869c4 <.memset+0x28> bdnz 0000000000a869bc <.memset+0x20> For: handle_kernel_slb_spill: type=3D0x380 dar=3D0x22ef8 srr0=3D0xa86690 handle_kernel_slb_spill: type=3D0x480 dar=3D0x22ef8 srr0=3D0xa86690 both seem to involve the stdu instruction in: 0000000000a8668c <.memcpy+0x140> ldu r0,-8(r9) 0000000000a86690 <.memcpy+0x144> stdu r0,-8(r11) 0000000000a86694 <.memcpy+0x148> bdnz 0000000000a8668c = <.memcpy+0x140> although the first is for a EXC_DSE (Data Segment Exception) and the second for a EXC_ISE (Instruction Segment Exception). The effective addresses reported for srr0 seem to match what objdump shows for the kernel file. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)