From owner-freebsd-ppc@freebsd.org Fri Apr 5 10:13:58 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D115A1570B54; Fri, 5 Apr 2019 10:13:57 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 3646587EDA; Fri, 5 Apr 2019 10:13:54 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id 0D37343A785; Fri, 5 Apr 2019 21:13:51 +1100 (AEDT) Date: Fri, 5 Apr 2019 21:13:50 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Bruce Evans , Konstantin Belousov , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] In-Reply-To: Message-ID: <20190405201055.I2396@besplex.bde.org> References: <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405150236.A959@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=P6RKvmIu c=1 sm=1 tr=0 cx=a_idp_d a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=E-FLDMUcsM50LrUROKkA:9 a=CjuIK1q_8ugA:10 X-Rspamd-Queue-Id: 3646587EDA X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of brde@optusnet.com.au designates 211.29.132.246 as permitted sender) smtp.mailfrom=brde@optusnet.com.au X-Spamd-Result: default: False [-6.01 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:211.29.132.0/23]; FREEMAIL_FROM(0.00)[optusnet.com.au]; RCPT_COUNT_FIVE(0.00)[5]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: extmail.optusnet.com.au]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[246.132.29.211.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[optusnet.com.au]; ASN(0.00)[asn:4804, ipnet:211.28.0.0/14, country:AU]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[optusnet.com.au]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-2.72)[ip: (-7.39), ipnet: 211.28.0.0/14(-3.42), asn: 4804(-2.72), country: AU(-0.04)]; FREEMAIL_CC(0.00)[optusnet.com.au]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 10:13:58 -0000 On Fri, 5 Apr 2019, Mark Millard wrote: > On 2019-Apr-4, at 21:38, Bruce Evans wrote: > >> On Thu, 4 Apr 2019, Mark Millard wrote: >>> ... >>> Unfortunately, all the multi-socket contexts that I sometimes have >>> access to are old PowerMacs. And, currently, the only such context >>> is the G5 with 2 sockets, 2 cores per socket (powerpc64). So I've >>> not been able to set up other types of examples to see if problems >>> repeat. >>> >>> I do not have access to a single-socket powerpc64 for contrast in >>> that direction. >> >> Testing 1 socket is time-consuming enough. Do these old systems >> use the equivalent of an x86 TSC for the timecounter? With multiple >> sockets, it isn't clear how even a hardware timer independent of the >> CPUs can be distributed so as to appear to be monotonic on all cors. > > "The DEC frequency is based on the same implementation-dependent > frequency that drives the time base." The frequency may well be > fixed by the PowerMac G5 model implementation but is not fixed > by the powerpc64 architecture. > > The Time Base Register (TBR) in a powerpc64 core (cpu in FreeBSD terms) > increments at 33,333,333 Hz (nominal) and is 64 bits wide. Its value > can be set (mttb instruction) and the boot sequence in FreeBSD does > attempt to adjust as the FreeBSD CPU is brought-up/started. mftb is > used to read the 64-bit value. FreeBSD masks it down to 32-bits to > contribute to the time-counter. > > (Is that description sufficient for what you were after? I've never > seen documentation of how the 33,333,333 MHz is produced.) This seems to be equivalent to an x86 TSC. x86 P-state-invariant TSCs apparently use a 100 MHz clock which corresponds even more closely to this, except for historical reasons this clock is scaled and interpolated to a clock resembling the CPU cycle count at a nominal frequency. > As FreeBSD supports multi-socket, what are its criteria for a sufficient > context for it to work with for supporting sbinuptime and the like? Is > FreeBSD supposed to then make it appear that sbinputime and the like are > weakly increasing, even as threads migrate between CPUs (cores, > hw-threads)? CLOCK_MONOTONIC has to be monotonic, but it is only clear what this means within a single thread: sequential clock_gettime() calls must occur in program order and the results must be consistent with that order. Across threads, I think it should mean that the results must be consistent with any order established using any supported ordering methods. >... >>> One oddity is that the eventtimer's decrementer and timecounter >>> may change (nearly) together: both change at 33,333,333 Hz, as if >>> they are tied to the same clock (at least on one socket). >> >> I think this is from a normal hardware implementation. On all of >> my x86 systems with a TSC, the TSC frequency is an exact fractional >> multiple of the i8254, the ACPI timer (if present) and the HPET (if >> present). Only the RTC has an independent frequency. The fraction is >> changed by changing the nominal TSC frequency in the BIOS, but is not >> changed by temperature variations. This must be because most clocks are >> derived from a common clock using a PLL. I use this to calibrate all >> clocks (except the RTC) by calibrating only 1. > > I'm not aware of the OpenFirmware having any control over the > TBR-change frequency behavior. I've no evidence about any variability > based on temperature. Temperature changes usually affect the actual frequency but not the nominal frequency. Bruce