From owner-freebsd-current@FreeBSD.ORG Tue Jun 21 20:53:50 2011 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id 19CDD106567E; Tue, 21 Jun 2011 20:53:50 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-current@FreeBSD.org Date: Tue, 21 Jun 2011 16:53:32 -0400 User-Agent: KMail/1.6.2 References: <201106171323.43864.jkim@FreeBSD.org> <201106171455.00088.jkim@FreeBSD.org> In-Reply-To: <201106171455.00088.jkim@FreeBSD.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: Multipart/Mixed; boundary="Boundary-00=_OTQAOvu1ci8u4Zp" Message-Id: <201106211653.34894.jkim@FreeBSD.org> Cc: Ian FREISLICH Subject: Re: Time keeping Issues with the low-resolution TSC timecounter X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2011 20:53:51 -0000 --Boundary-00=_OTQAOvu1ci8u4Zp Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline On Friday 17 June 2011 02:54 pm, Jung-uk Kim wrote: > On Friday 17 June 2011 01:45 pm, Ian FREISLICH wrote: > > Jung-uk Kim wrote: > > > On Thursday 16 June 2011 03:10 am, Ian FREISLICH wrote: > > > > Jung-uk Kim wrote: > > > > > 1481522037 14459060 1.0098392393 > > > > > 1495969404 14447367 1.0090225853 > > > > > > > > > > As you can see, HPET increases normally (within errors from > > > > > sleep(3) accuracy, syscall overhead, etc.) but TSC-low is > > > > > totally erratic (and too low). I don't know how this can > > > > > happen, though. > > > > > > > > > > :-( > > > > > > > > > > I need some time to figure it out. > > > > > > > > Even though sleep states have been disabled in the past when > > > > on AC power, they seem to have mysteriously been enabled. > > > > Perhaps this accounts for the strangeness: > > > > > > > > /etc/rc.conf > > > > performance_cx_lowest="HIGH" > > > > performance_cpu_freq="HIGH" > > > > economy_cx_lowest="LOW" > > > > economy_cpu_freq="HIGH" > > > > > > > > > > > > [mini] /usr/home/ianf $ sysctl dev.cpu > > > > dev.cpu.0.%desc: ACPI CPU > > > > dev.cpu.0.%driver: cpu > > > > dev.cpu.0.%location: handle=\_PR_.CPU0 > > > > dev.cpu.0.%pnpinfo: _HID=none _UID=0 > > > > dev.cpu.0.%parent: acpi0 > > > > dev.cpu.0.freq: 1600 > > > > dev.cpu.0.freq_levels: 1600/2000 1400/1750 1333/1533 > > > > 1166/1341 1066/1066 932/932 800/600 700/525 600/450 500/375 > > > > 400/300 300/225 200/150 100/75 dev.cpu.0.cx_supported: C1/1 > > > > C2/1 C3/57 dev.cpu.0.cx_lowest: C3 > > > > dev.cpu.0.cx_usage: 0.00% 8.69% 91.30% last 693us > > > > dev.cpu.1.%desc: ACPI CPU > > > > dev.cpu.1.%driver: cpu > > > > dev.cpu.1.%location: handle=\_PR_.CPU1 > > > > dev.cpu.1.%pnpinfo: _HID=none _UID=0 > > > > dev.cpu.1.%parent: acpi0 > > > > dev.cpu.1.cx_supported: C1/1 C2/1 C3/57 > > > > dev.cpu.1.cx_lowest: C3 > > > > dev.cpu.1.cx_usage: 0.00% 14.96% 85.03% last 2897us > > > > > > > > Pulling the power cord and re-inserting it has the cx_lowest > > > > correctly trantsition to C1 and then TSC-low behaves properly > > > > as the system timecounter. But, time will be wierd when on > > > > battery. > > > > > > > > In light of this, I doubt the patch in your other email will > > > > have any effect. Perhaps the thing to do is to have the > > > > timecounter code aware of the lowest Cx sleep state and to > > > > pick best time counter for that state and to re-evaluate the > > > > choice on cx_lowest transitions. > > > > > > > > ie: TSC-low, HPET or ACPI-fast for C1 and HPET or ACPI-fast > > > > for C2 and lower. > > > > > > Hmm... So, you are saying this CPU model is P-state invariant > > > but not C-state invariant (i.e., it stops incrementing in C2 > > > state and above). If that's the case, it is really useless for > > > timecounter. :-( > > > > > > What happens if you set it to C2, i.e., > > > > > > economy_cx_lowest="C2" > > > > > > In other words, does it really stop in C2-state? > > > > The folowing is with timecounter=HPET, just to see what the > > effect on TSC-low is. It looks like it does stop in C3. > > > > hw.acpi.cpu.cx_lowest=C3 > > [mini] /usr/home/ianf $ sh -c 'count=10; while [ $count -gt 0 ]; > > do count=$((count - 1)); sysctl > > kern.timecounter.tc.TSC-low.counter; sleep 1; done' > > kern.timecounter.tc.TSC-low.counter: 722687906 > > kern.timecounter.tc.TSC-low.counter: 724328394 > > kern.timecounter.tc.TSC-low.counter: 726038743 > > kern.timecounter.tc.TSC-low.counter: 727690855 > > kern.timecounter.tc.TSC-low.counter: 729245616 > > kern.timecounter.tc.TSC-low.counter: 730786569 > > kern.timecounter.tc.TSC-low.counter: 732398571 > > kern.timecounter.tc.TSC-low.counter: 733910987 > > kern.timecounter.tc.TSC-low.counter: 735711469 > > kern.timecounter.tc.TSC-low.counter: 737368279 > > > > hw.acpi.cpu.cx_lowest=C2 > > kern.timecounter.tc.TSC-low.counter: 897318486 > > kern.timecounter.tc.TSC-low.counter: 909873821 > > kern.timecounter.tc.TSC-low.counter: 922416894 > > kern.timecounter.tc.TSC-low.counter: 934960462 > > kern.timecounter.tc.TSC-low.counter: 947504154 > > kern.timecounter.tc.TSC-low.counter: 960050573 > > kern.timecounter.tc.TSC-low.counter: 972590754 > > kern.timecounter.tc.TSC-low.counter: 985133990 > > kern.timecounter.tc.TSC-low.counter: 997677052 > > kern.timecounter.tc.TSC-low.counter: 1010220299 > > > > CPU: Intel(R) Atom(TM) CPU N270 @ 1.60GHz (1596.04-MHz > > 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Family = 6 > > Model = 1c Stepping = 2 > > Features=0xbfe9fbff >TR > > R,PGE,MCA,CMOV,PAT,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,P > >BE> > > Features2=0x40c39d >,M OVBE> AMD Features2=0x1 > > TSC: P-state invariant, performance statistics > > Thanks for the info, it confirmed my speculation. > > Somewhere from an Intel manual, I think I read TSC stops when > DPSLP# pin is asserted for Core/Core2/Atom processors and I guess > that means entering C3 stops TSC. :-( Can you please try the attached patch? It should disable TSC/TSC-low timecounter for your CPU models, I think. Sorry for the delay, Jung-uk Kim --Boundary-00=_OTQAOvu1ci8u4Zp Content-Type: text/plain; charset="iso-8859-1"; name="tsc_c3.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tsc_c3.diff" Index: sys/dev/acpica/acpi_cpu.c =================================================================== --- sys/dev/acpica/acpi_cpu.c (revision 223372) +++ sys/dev/acpica/acpi_cpu.c (working copy) @@ -856,6 +856,8 @@ acpi_cpu_cx_list(struct acpi_cpu_softc *sc) sbuf_printf(&sb, "C%d/%d ", i + 1, sc->cpu_cx_states[i].trans_lat); if (sc->cpu_cx_states[i].type < ACPI_STATE_C3) sc->cpu_non_c3 = i; + else + cpu_can_deep_sleep = 1; } sbuf_trim(&sb); sbuf_finish(&sb); Index: sys/kern/kern_clocksource.c =================================================================== --- sys/kern/kern_clocksource.c (revision 223372) +++ sys/kern/kern_clocksource.c (working copy) @@ -59,6 +59,7 @@ __FBSDID("$FreeBSD$"); cyclic_clock_func_t cyclic_clock_func = NULL; #endif +int cpu_can_deep_sleep = 0; /* C3 state is available. */ int cpu_disable_deep_sleep = 0; /* Timer dies in C3. */ static void setuptimer(void); Index: sys/sys/systm.h =================================================================== --- sys/sys/systm.h (revision 223372) +++ sys/sys/systm.h (working copy) @@ -253,6 +253,7 @@ void cpu_startprofclock(void); void cpu_stopprofclock(void); void cpu_idleclock(void); void cpu_activeclock(void); +extern int cpu_can_deep_sleep; extern int cpu_disable_deep_sleep; int cr_cansee(struct ucred *u1, struct ucred *u2); Index: sys/x86/x86/tsc.c =================================================================== --- sys/x86/x86/tsc.c (revision 223372) +++ sys/x86/x86/tsc.c (working copy) @@ -444,6 +444,19 @@ init_TSC_tc(void) goto init; } + /* + * We cannot use the TSC if it stops incrementing in deep sleep. + * Currently only Intel CPUs are known for this problem unless + * the invariant TSC bit is set. + */ + if (cpu_vendor_id == CPU_VENDOR_INTEL && cpu_exthigh >= 0x80000007 && + (amd_pminfo & AMDPM_TSC_INVARIANT) == 0 && cpu_can_deep_sleep) { + tsc_timecounter.tc_quality = -1000; + if (bootverbose) + printf("TSC timecounter disabled: C3 enabled.\n"); + goto init; + } + #ifdef SMP /* * We can not use the TSC in SMP mode unless the TSCs on all CPUs are --Boundary-00=_OTQAOvu1ci8u4Zp--