From owner-freebsd-hackers@FreeBSD.ORG Tue May 24 17:56:53 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id 140D1106564A; Tue, 24 May 2011 17:56:53 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-hackers@FreeBSD.org Date: Tue, 24 May 2011 13:56:43 -0400 User-Agent: KMail/1.6.2 MIME-Version: 1.0 Content-Disposition: inline Content-Type: Multipart/Mixed; boundary="Boundary-00=_dF/2NU1LNz9TGHi" Message-Id: <201105241356.45543.jkim@FreeBSD.org> Cc: Andriy Gapon Subject: [RFC] Enabling invariant TSC timecounter on SMP X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 May 2011 17:56:53 -0000 --Boundary-00=_dF/2NU1LNz9TGHi Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline I think it's about time to enable invariant TSC timecounter on SMP by default. Please see the attached patch. It is also available from here: http://people.freebsd.org/~jkim/tsc_smp_test4.diff avg convinced me enough that it should be an opt-out feature going forward. :-) Comments? Cheers, Jung-uk Kim --Boundary-00=_dF/2NU1LNz9TGHi Content-Type: text/plain; charset="iso-8859-1"; name="tsc_smp_test4.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="tsc_smp_test4.diff" Index: sys/x86/x86/tsc.c =================================================================== --- sys/x86/x86/tsc.c (revision 222262) +++ sys/x86/x86/tsc.c (working copy) @@ -79,7 +79,8 @@ static void tsc_freq_changed(void *arg, const stru int status); static void tsc_freq_changing(void *arg, const struct cf_level *level, int *status); -static unsigned tsc_get_timecount(struct timecounter *tc); +static unsigned tsc_get_timecount(struct timecounter *tc); +static unsigned tsc_get_timecount_lowres(struct timecounter *tc); static void tsc_levels_changed(void *arg, int unit); static struct timecounter tsc_timecounter = { @@ -385,7 +386,7 @@ test_smp_tsc(void) if (bootverbose) printf("SMP: %sed TSC synchronization test\n", smp_tsc ? "pass" : "fail"); - return (smp_tsc ? 800 : -100); + return (smp_tsc ? (tsc_is_invariant ? 1000 : 800) : -100); } #undef N @@ -395,11 +396,19 @@ test_smp_tsc(void) static void init_TSC_tc(void) { + uint64_t max_freq; + int shift; if ((cpu_feature & CPUID_TSC) == 0 || tsc_disabled) return; /* + * Limit timecounter frequency to fit in an int and prevent it from + * overflowing too fast. + */ + max_freq = UINT_MAX; + + /* * We can not use the TSC if we support APM. Precise timekeeping * on an APM'ed machine is at best a fools pursuit, since * any and all of the time spent in various SMM code can't @@ -421,13 +430,30 @@ init_TSC_tc(void) * We can not use the TSC in SMP mode unless the TSCs on all CPUs are * synchronized. If the user is sure that the system has synchronized * TSCs, set kern.timecounter.smp_tsc tunable to a non-zero value. + * We also limit the frequency even lower to avoid "temporal anomalies" + * as much as possible. */ - if (smp_cpus > 1) + if (smp_cpus > 1) { tsc_timecounter.tc_quality = test_smp_tsc(); + max_freq >>= 8; + } else #endif + if (tsc_is_invariant) + tsc_timecounter.tc_quality = 1000; + init: + for (shift = 0; shift < 32 && (tsc_freq >> shift) > max_freq; shift++) + ; + if (shift > 0) { + tsc_timecounter.tc_get_timecount = tsc_get_timecount_lowres; + tsc_timecounter.tc_name = "TSC-low"; + if (bootverbose) + printf("TSC timecounter discards lower %d bit(s).\n", + shift); + } if (tsc_freq != 0) { - tsc_timecounter.tc_frequency = tsc_freq; + tsc_timecounter.tc_frequency = tsc_freq >> shift; + tsc_timecounter.tc_priv = (void *)(intptr_t)shift; tc_init(&tsc_timecounter); } } @@ -499,7 +525,8 @@ tsc_freq_changed(void *arg, const struct cf_level /* Total setting for this level gives the new frequency in MHz. */ freq = (uint64_t)level->total_set.freq * 1000000; atomic_store_rel_64(&tsc_freq, freq); - atomic_store_rel_64(&tsc_timecounter.tc_frequency, freq); + tsc_timecounter.tc_frequency = + freq >> (int)(intptr_t)tsc_timecounter.tc_priv; } static int @@ -514,7 +541,8 @@ sysctl_machdep_tsc_freq(SYSCTL_HANDLER_ARGS) error = sysctl_handle_64(oidp, &freq, 0, req); if (error == 0 && req->newptr != NULL) { atomic_store_rel_64(&tsc_freq, freq); - atomic_store_rel_64(&tsc_timecounter.tc_frequency, freq); + tsc_timecounter.tc_frequency = + freq >> (int)(intptr_t)tsc_timecounter.tc_priv; } return (error); } @@ -523,8 +551,15 @@ SYSCTL_PROC(_machdep, OID_AUTO, tsc_freq, CTLTYPE_ 0, 0, sysctl_machdep_tsc_freq, "QU", "Time Stamp Counter frequency"); static u_int -tsc_get_timecount(struct timecounter *tc) +tsc_get_timecount(struct timecounter *tc __unused) { return (rdtsc32()); } + +static u_int +tsc_get_timecount_lowres(struct timecounter *tc) +{ + + return (rdtsc() >> (int)(intptr_t)tc->tc_priv); +} --Boundary-00=_dF/2NU1LNz9TGHi--