From owner-freebsd-hackers@freebsd.org Fri Apr 5 21:31:52 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06CE5155F397 for ; Fri, 5 Apr 2019 21:31:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic302-3.consmr.mail.bf2.yahoo.com (sonic302-3.consmr.mail.bf2.yahoo.com [74.6.135.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9443F762FD for ; Fri, 5 Apr 2019 21:31:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: m3n.7wIVM1kc6473I5483ksjM9s0yR8uGzlSWg3L9xpBxZIAEq2e2TeMKoLHYKk 7NKnJVbk8QF3DI37BqEt5tSU9MMVpK3jaRGQDEFAshAa.2WUSG2D3aTuIcC99upH.KxqzdIlbG23 ngKmuMl0UsK2IphWhUuK0IGKPrfHIYZkNspo7WWi0QonLAGn3WkiEaBFH6k5wIKyr5Zbu7JOaqr_ 15OhnGjVKgxms6hxZK5_q3Q4M1i1TwiQ2RBhDx2oLr6e8roMYvz9WOlnTAo.pyr5EIKYzK7ijbL0 F7rKwsZiANSgwsRoEeu16nbrcFsjLbWxUYV35.zbSs6gVPKxDcn49P_o7udmpczqX2zjd.8YgDj4 a2RVOQtWBvxmF8Nxz8yqLuE5rzuPkitj9lxuE5VafuuG6YMOPN546RnmzzA_g2rS6ouDyH9bq1xU _PEJadq1zBmuJtIs0mX1ejPl9mUiwEKKPBy9cyS9aUaYOS6YLv_WCFK26_SW6Rj2yLo9zSugxj1c mQONqVhHD_k6G4K3k0gIbtLhYrTT91CrvhNBQSv6kT_cJDMyTdQrmLRCe6dXKV5.tNTF8x7jB7t. ItXNVgrgBKwn0zr.N_YXPJs140tsiHmBAu1yJhYRxDq7J6tfrT6dJRTMGg0pGcNu_p.SaVTeuerx C.7GhqK5xv9lVhmDjLAacK5rYIh69wcTAkzl3071VzjNPXh939c2KZiFbWaoBkTJfdJRwzXpJQD3 tmWsCjZb1LNhC8NzLbQevSs7r3xwoM19FBJRIEvuaUgDJ05WXpV6WpwocPkYjp18cQ7gb5QMcA62 r7WoOaB7kXabWjJWfw.L.KYvApEPQGKsDLf.O5WcIwwJibt6.cE6XTzWIxL4AYXSC.P3shpiZc0D 702ZzBc9OvDp5GnIkq0KbK7y2nY2QnwNERYxAetyLcRED0vINsu.qJ5WXaYkUo3ge_lpbCWGKq5G pjD6SI4OmU7ayibz2A._z4Go4LqX5ImS3cM.HpdK7yAqj8McxPfhvmawukXAuuOXwUAIxaSmtSkT hustoDi2P_tLfQLDVG6XuaeNweK9IEDmjc7h5E3xUGhtYWd8ATl_FT3coIalTQc.BlaWIXbPp Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.bf2.yahoo.com with HTTP; Fri, 5 Apr 2019 21:31:50 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp401.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 7418801b5e1fee9bf1fae3b3fb8f9af6; Fri, 05 Apr 2019 21:31:46 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <20190405113504.GA1923@kib.kiev.ua> Date: Fri, 5 Apr 2019 14:31:43 -0700 Cc: freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: <55604EF6-CB81-4300-8E9E-E3A94504D0B5@yahoo.com> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> <20190405201055.I2396@besplex.bde.org> <20190405113504.GA1923@kib.kiev.ua> To: Konstantin Belousov , Bruce Evans X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 9443F762FD X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 21:31:52 -0000 [I found a document covering PLLs for the 970MP's in the old PowerMac G5 involved.] > On 2019-Apr-5, at 04:35, Konstantin Belousov = wrote: >=20 > On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: >> On Fri, 5 Apr 2019, Mark Millard wrote: >>=20 >>> On 2019-Apr-4, at 21:38, Bruce Evans = wrote: >>>=20 >>>> On Thu, 4 Apr 2019, Mark Millard wrote: >>>>> ... >>>>> Unfortunately, all the multi-socket contexts that I sometimes have >>>>> access to are old PowerMacs. And, currently, the only such context >>>>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >>>>> not been able to set up other types of examples to see if problems >>>>> repeat. >>>>>=20 >>>>> I do not have access to a single-socket powerpc64 for contrast in >>>>> that direction. >>>>=20 >>>> Testing 1 socket is time-consuming enough. Do these old systems >>>> use the equivalent of an x86 TSC for the timecounter? With = multiple >>>> sockets, it isn't clear how even a hardware timer independent of = the >>>> CPUs can be distributed so as to appear to be monotonic on all = cors. >>>=20 >>> "The DEC frequency is based on the same implementation-dependent >>> frequency that drives the time base." The frequency may well be >>> fixed by the PowerMac G5 model implementation but is not fixed >>> by the powerpc64 architecture. >>>=20 >>> The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD = terms) >>> increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value >>> can be set (mttb instruction) and the boot sequence in FreeBSD does >>> attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is >>> used to read the 64-bit value. FreeBSD masks it down to 32-bits to >>> contribute to the time-counter. >>>=20 >>> (Is that description sufficient for what you were after? I've never >>> seen documentation of how the 33,333,333 MHz is produced.) >>=20 >> This seems to be equivalent to an x86 TSC. > It is equivalent from the interface PoV. I saw some references that > Powers do have per-core PLLs, the best I can find now is > = https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-co= mmon-in-modern-processors I finally found a document covering the 970MP in the old PowerMac G5 in question: http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf It has material about the PLL's, pinout, etc. : "This section will help in configuring the PLL and determining SYSCLK = input frequency for PowerPC 970MP systems." Also: 100 MHz to 700 MHz SYSCLK. "SYSCLK minimum frequency is for PLL mode. In PLL bypass mode (BYPASS = low), SYSCLK frequency may be as low as 10MHz." BUS_CFG[0:2], PLL_MULT, and PLL_RANGE[1:0] are involved for PLL mode. Ratio from BUS_CFG: 3:1, 6:1, 12:1 normally use PLL_MULT=3D0 for a PLL = multiplier of 12. Ratio from BUS_CFG: 2:1, 4:1, 8:1 normally use PLL_MULT=3D1 for a PLL = multiplier of 8. (It describes the PLL power supply filtering circuit as well.) So far I've not seen anything that is directly for the rate that the TBR and DEC registers change. > For Intel there is no much information, the best guess is that TSC is = in > uncore and effectively shared by all cores. See also > = https://software.intel.com/en-us/articles/best-timing-function-for-measuri= ng-ipp-api-timing/ >=20 >>=20 >> x86 P-state-invariant TSCs apparently use a 100 MHz clock which = corresponds >> even more closely to this, except for historical reasons this clock = is >> scaled and interpolated to a clock resembling the CPU cycle count at = a >> nominal frequency. > For Intel designs, there is indeed a single-source 100MHz signal which > is distributed over all consumers using clock fan-out buffers like = DB1900Z. >=20 >>=20 >>> As FreeBSD supports multi-socket, what are its criteria for a = sufficient >>> context for it to work with for supporting sbinuptime and the like? = Is >>> FreeBSD supposed to then make it appear that sbinputime and the like = are >>> weakly increasing, even as threads migrate between CPUs (cores, >>> hw-threads)? >>=20 >> CLOCK_MONOTONIC has to be monotonic, but it is only clear what this = means >> within a single thread: sequential clock_gettime() calls must occur = in >> program order and the results must be consistent with that order. = Across >> threads, I think it should mean that the results must be consistent = with >> any order established using any supported ordering methods. >>=20 >>> ... >>>>> One oddity is that the eventtimer's decrementer and timecounter >>>>> may change (nearly) together: both change at 33,333,333 Hz, as if >>>>> they are tied to the same clock (at least on one socket). >>>>=20 >>>> I think this is from a normal hardware implementation. On all of >>>> my x86 systems with a TSC, the TSC frequency is an exact fractional >>>> multiple of the i8254, the ACPI timer (if present) and the HPET (if >>>> present). Only the RTC has an independent frequency. The fraction = is >>>> changed by changing the nominal TSC frequency in the BIOS, but is = not >>>> changed by temperature variations. This must be because most = clocks are >>>> derived from a common clock using a PLL. I use this to calibrate = all >>>> clocks (except the RTC) by calibrating only 1. >>>=20 >>> I'm not aware of the OpenFirmware having any control over the >>> TBR-change frequency behavior. I've no evidence about any = variability >>> based on temperature. >>=20 >> Temperature changes usually affect the actual frequency but not the >> nominal frequency. >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)