Date: Fri, 5 Apr 2019 14:31:43 -0700 From: Mark Millard <marklmi@yahoo.com> To: Konstantin Belousov <kostikbel@gmail.com>, Bruce Evans <brde@optusnet.com.au> Cc: freebsd-hackers Hackers <freebsd-hackers@freebsd.org>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <55604EF6-CB81-4300-8E9E-E3A94504D0B5@yahoo.com> In-Reply-To: <20190405113504.GA1923@kib.kiev.ua> References: <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <E0785613-2B6E-4BB3-95CD-03DD96902CD8@fh-muenster.de> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <F22CCA2C-08BB-452E-B00C-A36CD4611540@yahoo.com> <20190405150236.A959@besplex.bde.org> <F3BBD355-5198-41A0-A461-8E5E12984BE7@yahoo.com> <20190405201055.I2396@besplex.bde.org> <20190405113504.GA1923@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
[I found a document covering PLLs for the 970MP's in the old PowerMac G5 involved.] > On 2019-Apr-5, at 04:35, Konstantin Belousov <kostikbel at gmail.com> = wrote: >=20 > On Fri, Apr 05, 2019 at 09:13:50PM +1100, Bruce Evans wrote: >> On Fri, 5 Apr 2019, Mark Millard wrote: >>=20 >>> On 2019-Apr-4, at 21:38, Bruce Evans <brde at optusnet.com.au> = wrote: >>>=20 >>>> On Thu, 4 Apr 2019, Mark Millard wrote: >>>>> ... >>>>> Unfortunately, all the multi-socket contexts that I sometimes have >>>>> access to are old PowerMacs. And, currently, the only such context >>>>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >>>>> not been able to set up other types of examples to see if problems >>>>> repeat. >>>>>=20 >>>>> I do not have access to a single-socket powerpc64 for contrast in >>>>> that direction. >>>>=20 >>>> Testing 1 socket is time-consuming enough. Do these old systems >>>> use the equivalent of an x86 TSC for the timecounter? With = multiple >>>> sockets, it isn't clear how even a hardware timer independent of = the >>>> CPUs can be distributed so as to appear to be monotonic on all = cors. >>>=20 >>> "The DEC frequency is based on the same implementation-dependent >>> frequency that drives the time base." The frequency may well be >>> fixed by the PowerMac G5 model implementation but is not fixed >>> by the powerpc64 architecture. >>>=20 >>> The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD = terms) >>> increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value >>> can be set (mttb instruction) and the boot sequence in FreeBSD does >>> attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is >>> used to read the 64-bit value. FreeBSD masks it down to 32-bits to >>> contribute to the time-counter. >>>=20 >>> (Is that description sufficient for what you were after? I've never >>> seen documentation of how the 33,333,333 MHz is produced.) >>=20 >> This seems to be equivalent to an x86 TSC. > It is equivalent from the interface PoV. I saw some references that > Powers do have per-core PLLs, the best I can find now is > = https://stackoverflow.com/questions/43844438/are-multiple-clock-domains-co= mmon-in-modern-processors I finally found a document covering the 970MP in the old PowerMac G5 in question: http://datasheets.chipdb.org/IBM/PowerPC/970/PowerPC-970MP.pdf It has material about the PLL's, pinout, etc. : "This section will help in configuring the PLL and determining SYSCLK = input frequency for PowerPC 970MP systems." Also: 100 MHz to 700 MHz SYSCLK. "SYSCLK minimum frequency is for PLL mode. In PLL bypass mode (BYPASS = low), SYSCLK frequency may be as low as 10MHz." BUS_CFG[0:2], PLL_MULT, and PLL_RANGE[1:0] are involved for PLL mode. Ratio from BUS_CFG: 3:1, 6:1, 12:1 normally use PLL_MULT=3D0 for a PLL = multiplier of 12. Ratio from BUS_CFG: 2:1, 4:1, 8:1 normally use PLL_MULT=3D1 for a PLL = multiplier of 8. (It describes the PLL power supply filtering circuit as well.) So far I've not seen anything that is directly for the rate that the TBR and DEC registers change. > For Intel there is no much information, the best guess is that TSC is = in > uncore and effectively shared by all cores. See also > = https://software.intel.com/en-us/articles/best-timing-function-for-measuri= ng-ipp-api-timing/ >=20 >>=20 >> x86 P-state-invariant TSCs apparently use a 100 MHz clock which = corresponds >> even more closely to this, except for historical reasons this clock = is >> scaled and interpolated to a clock resembling the CPU cycle count at = a >> nominal frequency. > For Intel designs, there is indeed a single-source 100MHz signal which > is distributed over all consumers using clock fan-out buffers like = DB1900Z. >=20 >>=20 >>> As FreeBSD supports multi-socket, what are its criteria for a = sufficient >>> context for it to work with for supporting sbinuptime and the like? = Is >>> FreeBSD supposed to then make it appear that sbinputime and the like = are >>> weakly increasing, even as threads migrate between CPUs (cores, >>> hw-threads)? >>=20 >> CLOCK_MONOTONIC has to be monotonic, but it is only clear what this = means >> within a single thread: sequential clock_gettime() calls must occur = in >> program order and the results must be consistent with that order. = Across >> threads, I think it should mean that the results must be consistent = with >> any order established using any supported ordering methods. >>=20 >>> ... >>>>> One oddity is that the eventtimer's decrementer and timecounter >>>>> may change (nearly) together: both change at 33,333,333 Hz, as if >>>>> they are tied to the same clock (at least on one socket). >>>>=20 >>>> I think this is from a normal hardware implementation. On all of >>>> my x86 systems with a TSC, the TSC frequency is an exact fractional >>>> multiple of the i8254, the ACPI timer (if present) and the HPET (if >>>> present). Only the RTC has an independent frequency. The fraction = is >>>> changed by changing the nominal TSC frequency in the BIOS, but is = not >>>> changed by temperature variations. This must be because most = clocks are >>>> derived from a common clock using a PLL. I use this to calibrate = all >>>> clocks (except the RTC) by calibrating only 1. >>>=20 >>> I'm not aware of the OpenFirmware having any control over the >>> TBR-change frequency behavior. I've no evidence about any = variability >>> based on temperature. >>=20 >> Temperature changes usually affect the actual frequency but not the >> nominal frequency. >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55604EF6-CB81-4300-8E9E-E3A94504D0B5>