From owner-svn-src-all@freebsd.org Sun Mar 27 17:23:10 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23E3FADFE02; Sun, 27 Mar 2016 17:23:10 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id C33561E5B; Sun, 27 Mar 2016 17:23:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c110-21-41-193.carlnfd1.nsw.optusnet.com.au (c110-21-41-193.carlnfd1.nsw.optusnet.com.au [110.21.41.193]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 7D1DE783AC2; Mon, 28 Mar 2016 04:23:00 +1100 (AEDT) Date: Mon, 28 Mar 2016 04:22:59 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov cc: Bruce Evans , John Baldwin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, "'rstone@freebsd.org'" Subject: Re: svn commit: r297039 - head/sys/x86/x86 In-Reply-To: <20160327130706.GA1741@kib.kiev.ua> Message-ID: <20160328033624.N850@besplex.bde.org> References: <20160323075842.GX1741@kib.kiev.ua> <2922763.uITxoCVqGR@ralph.baldwin.cx> <20160324090917.GC1741@kib.kiev.ua> <20160325010649.H898@besplex.bde.org> <20160324162447.GD1741@kib.kiev.ua> <20160325060901.N2059@besplex.bde.org> <20160325084902.GH1741@kib.kiev.ua> <20160326021219.X911@besplex.bde.org> <20160326174210.GU1741@kib.kiev.ua> <20160327051601.S2318@besplex.bde.org> <20160327130706.GA1741@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=c+ZWOkJl c=1 sm=1 tr=0 a=73JWPhLeruqQCjN69UNZtQ==:117 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=kj9zAlcOel0A:10 a=QyXUC8HyAAAA:8 a=EvKqFsTwAAAA:8 a=NWmcdJ4kUMLG-LNHZ9wA:9 a=pLC86x-TDcFJlN1_:21 a=W5IfI8Gavtc64Nsx:21 a=CjuIK1q_8ugA:10 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Mar 2016 17:23:10 -0000 On Sun, 27 Mar 2016, Konstantin Belousov wrote: > On Sun, Mar 27, 2016 at 06:49:40AM +1100, Bruce Evans wrote: >> On Sat, 26 Mar 2016, Konstantin Belousov wrote: >> >>> On Sat, Mar 26, 2016 at 03:17:43AM +1100, Bruce Evans wrote: >>>> This uses the i8254. >>> Well, this is the part which I do not like most. It is ridiculous >>> to calibrate relatively high-quality CPU oscillator with 8245 timer, >>> which often fed from separate, low-quality crystal which currently >>> selected by cost and exists only for legacy purposes. At least >> >> I haven't seen a single (hardware) system since I started measuring >> this 15-20 yeas ago on which the i8524 is not fed from the same clock >> as the TSC. Normally there is a PLL that can derives hundreds of >> different frequencies from the same clock. One is fed to the i8254, >> one to the ACPI timer and one to the CPU clock/TSC. > Hm, I was able to track this down in the public documents. I was wrong, > but your description is not completely correct either. Please see > http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/x99-chipset-pch-datasheet.pdf > page 64 for the diagram of the clock signals interconnect on Romley and > later. Also the CK420BQ (clock synthesizer) and DB1200Z (clock buffer > for PCIe and CPU) datasheets are available on the idt.com site, but > there I can only understand the claims about jitter. Of course the details are machine-dependent. My oldest system on which I discovered PLL-like behaviour was from late 1997. The PLL might not have been in standard bus chips then. It had a K5 (?) running at 233 MHz. The ratio of the TSC to the i8254 was precisely 196. This was very easy to measure or to guess without measurement and then verify by measurements since it is an integer. This integer is just the one that gives the best approximation to the nominal TSC frequency of 233 MHz starting with the nominal i8254 frequency of 1193182. Newer systems have much higher resolution so the ratios are rarely integers. >> From the diagram at the page 64, main clock is fed into CPU PLL (directly > driving TSC and LAPIC timer) and into PCH (south bridge, where all other > counters reside). In PCH, misc. PLLs are used to drive all counters except > RTC. RTC needs its own dedicated crystal. > > I was not able to find any specifications for allowed jitter in PCH PLLs, > but I would expect that above IDT chips have much better stability than > something in overheating PCH. > >>> some hypervisors start offering modes where old ISA peripherals >>> are not emulated, and I suspect that hardware would start do the >>> same. Quite possible, some variants of SoCs already do this. >> >> Bug in these hypervisors. A slow clock like the i8254 is much easier >> to emulate than a fast one. It is impossible to emulate real time on >> a non-real-time system anyway, so almost all calibration code is likely >> to break in inverse proportion to its accuracy on real hardware. > RTC does not require any emulation at all, only some adjustments which > are already facilitated by the virtualization hardware. If the RTC is in direct hardware and the timecounters are partly in software, then it would be hard to preserve order and minimize (virtual) jitter in measurements comparing the timers. The timecounters would robably need additinal complications to keep them in step with the RTC. This care would usually be wasted since the RTC is rarely used like that. It would probably be easier to virtualise the RTC to keep it in step with the timecounters. >>>> xdel() is a specialized version of i8254 DELAY() >>>> with getit() inline. It returns the initial and final values of the >> >> It can use any readable timer, but only the i8254 is easy to use and >> available on all (non-broken) x86 systems. Just the >> ifdefs/configuratation to use another timer would be very complicated. >> Using the cputicker or timecounter read functions doesn't quite work >> because these functions have unknown internals which might involve >> unusable timing or lockimg. > I think we have no choice but do something for ISA/LPC-less configurations. Indeed. > We could even trust CPU report about its frequency as the last resort. > From what I saw, the calibrated RTC frequency is very close to the > reported frequency, and if runtime recalibration based on ntp client > feedback is implemented, all would be good. phk axed my RTC calibration code (actually i8254 and TSC calibration code starting from the RTC). It worked perfectly except it didn't adjust for the time to read the hardware. The problem with it is that the RTC is no more or less accurate than the other timers. It should be, but isn't in practice. Normally it it is controlled by different hardware (crystal?) with different temperature characteristics and different exposure to temperature changes by being physically separate. The i8254 is supposed to run at precisely 1193182 Hz but usually has an inaccuracy of 10-50 Hz. The RTC seconds update is supposed to occur at precisely 1 Hz but usually has an unrelated inaccuracy of similar magnitude. Calibrating these clocks relative to each other just gives the relative difference in their inaccuracy. The difference can be measured very accurately at any time. It varies later with temperature. I never finished code to track the termperature changes and calibrate adjustments based on the temperature. It is simpler to start with a hard-coded i8254 frequency of 1193182 and adjust everything later. ntpd handles adjustment of a single timer well enough. This works because the nominal frequency of 1193182 is so standard that inaccuracies in it are closer to 10 ppm than the 1000 ppm that would break ntpd. Bruce