From owner-freebsd-ppc@freebsd.org Sun May 5 01:24:31 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C001159E4C4 for ; Sun, 5 May 2019 01:24:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 683B480AAC for ; Sun, 5 May 2019 01:24:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: y0IUGM0VM1lJij5QHQyT.RLt8uZd.39C8HgnSRpBnU0U_O7GqmzSkwJ.i6nprZb BNqgmtlpsnIXeH0P_f7HEHaP7lhwJn9_5KbGj6i6oAk1JBN3kdz.BrvYb77jnUdBAC0MSDn_R5EU sk0SD3GKAMOBpyAkfmtB8o4ewPQWvVv_MUV3sm2ui241fsgQehLrwlEzM81IkLAIS.rqTz15tjE. x5iH907plsy17bMmBwzV7xlOsoNWBX6lSgQjprqE0HtWMAj1tvKgU3wLVA28Vqj1LZiivzOONI4z Gtd2bRx_FiO4yDoI2K01VgSk0CIStFuJ.yJhTJ0pJSmvz_kOJjMsAOVgUBiLTahJ.yGJ2TXsrDIN u_.qnGGmxmxBRpUb0bh6kp3Eif9v0Ycoo73WMhF40Mj7NtdQxdqMsnVhRXFf1Z00.0xYv9Bvxvry wWKTbC0eBya5cci1EcHM59o.JzkA8iD9NIEVg3h8XldfSsMQfj8DbUnS1rMJdBUdQNrlRWFdrORt Rmn_5D5opAmuMIxjXu3n2Xz_pyMCfVO74itVU_9Wyly8Y_qPxwiunU0Ggc5hWlh1uspG8x7Aojgu z3YuSZ3sAZ_gtjt3srmQzxDhx5uwkyfiZblAs7z5KRY9S2HImyRW5dwR90wlopMYCVf746mNRLnU ZFGL__0gx1ujXccMFZKB5UCLyQIVleA2EdMrQjbUPvbXptMCJpVYbhfrb4hmbib6qMzlZgKXScIT bE657AqCG.vj_o7Z1qq.Ow0pwOhnR8r0LxGTfl5XFsIaD9O_8_0T0JPOjtKdSurpBR_H7l6kZvrZ ZkRN17ZR7VN_UPvURXYhaH0oLjiPYAlWstH6z0a22Nk2OJDUJ35XdFdLXFmtBLCmW5pvp4TScKXi 1ITZ19LNPeBdnPrrePvOvkYKlpU2CJgPBUR5l5y0Fm5J4.qgF_qsuM3NEaie4oSOi1wVKhBOED2g 0.a3TKL2K0Dka5YGsxWxWr3ZGdETUU9JvW0qjHL_lSzt7XCNQYwpUsS3i4YNcYtw.xz.qeIFmeYH aj9bCmWtXKsF7YXARXr59cWNbWtwcuLXi6LYlLfHhaz2eHncR6nkDrIXbsevbBhQr0fHIWTciatM zc17WjywwpZ4f_6DyTf5Po4MSoN2pgzeDSNpBicD0ej7NFFDecwNrx1vWWyJQoIhduWc2zeuFS4x r8.jGIzld0Id9h.SYHbDCBOuvx5_Lv2mMsSI0BTxC42qkEWs8TKM- Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Sun, 5 May 2019 01:24:23 +0000 Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.103]) ([76.115.7.162]) by smtp401.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 9a82a4fe1b480ba324818b06f8d3fa68; Sun, 05 May 2019 01:24:20 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Examples of early boot variability in handle_kernel_slb_spill happening on 2-socket/2-core-each G5 PowerMac11,2 Message-Id: <331C36C9-3884-402B-BB1A-1E2C9A2BB511@yahoo.com> Date: Sat, 4 May 2019 18:24:18 -0700 To: Justin Hibbits , FreeBSD PowerPC ML X-Mailer: Apple Mail (2.3445.104.8) References: <331C36C9-3884-402B-BB1A-1E2C9A2BB511.ref@yahoo.com> X-Rspamd-Queue-Id: 683B480AAC X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.57)[0.573,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.85)[ip: (7.73), ipnet: 98.137.64.0/21(0.87), asn: 36647(0.70), country: US(-0.06)]; NEURAL_SPAM_MEDIUM(0.83)[0.833,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.76)[0.755,0]; RCVD_IN_DNSWL_NONE(0.00)[84.64.137.98.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 May 2019 01:24:31 -0000 I've switched to keeping counts of handle_kernel_slb_spill usage for my experiments with the slb-miss behavior: #if defined(__powerpc64__) && defined(AIM) unsigned int at_or_after_KVA_START_slb_spill_srr0_count=3D 0; unsigned int from_DMAP_START_upto_KVA_START_slb_spill_srr0_count=3D 0; unsigned int below_DMAP_START_slb_spill_srr0_count=3D 0; unsigned int at_or_after_KVA_START_slb_spill_dar_count=3D 0; unsigned int from_DMAP_START_upto_KVA_START_slb_spill_dar_count=3D 0; unsigned int below_DMAP_START_slb_spill_dar_count=3D 0; /* Handle kernel SLB faults -- runs in real mode, all seat belts off */ void handle_kernel_slb_spill(int type, register_t dar, register_t srr0) { struct slb *slbcache; uint64_t slbe, slbv; uint64_t esid, addr; int i; if (EXC_ISE=3D=3Dtype) { if (0xe000000000000000u<=3Dsrr0) at_or_after_KVA_START_slb_spill_srr0_count++; else if (0xc000000000000000u<=3Dsrr0) from_DMAP_START_upto_KVA_START_slb_spill_srr0_count++; else below_DMAP_START_slb_spill_srr0_count++; } else { if (0xe000000000000000u<=3Ddar) at_or_after_KVA_START_slb_spill_dar_count++; else if (0xc000000000000000u<=3Ddar) from_DMAP_START_upto_KVA_START_slb_spill_dar_count++; else below_DMAP_START_slb_spill_dar_count++; } . . . That is only designed to deal with the "only bsp" time frame. The examples below are for my use of prefaulting about half of the 64-1 kernel entries: i =3D 0; for (va =3D virtual_avail; va < virtual_end && i<(n_slbs-1)/2; = va +=3D SEGMENT_LENGTH, i++) moea64_bootstrap_slb_prefault(va, 0); (Some of the numbers checked are from before this loop as well.) The boots were a mix of: The original power-on shutdown -r now shutdown -p now then power-on The summary of the pictures of the samplings of the counts goes like . . . The moea64_bootstrap_slb_prefault loop had examples of: (counts from just-before vs. just-after checked for increases to non-zero) A) No handle_kernel_slb_spill use in the boot. B) dar