From owner-svn-src-all@FreeBSD.ORG Thu Apr 18 17:07:05 2013 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 548ECE02; Thu, 18 Apr 2013 17:07:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 36D7FD32; Thu, 18 Apr 2013 17:07:05 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.6/8.14.6) with ESMTP id r3IH75si032785; Thu, 18 Apr 2013 17:07:05 GMT (envelope-from mav@svn.freebsd.org) Received: (from mav@localhost) by svn.freebsd.org (8.14.6/8.14.5/Submit) id r3IH756J032784; Thu, 18 Apr 2013 17:07:05 GMT (envelope-from mav@svn.freebsd.org) Message-Id: <201304181707.r3IH756J032784@svn.freebsd.org> From: Alexander Motin Date: Thu, 18 Apr 2013 17:07:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r249625 - head/sys/x86/x86 X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 17:07:05 -0000 Author: mav Date: Thu Apr 18 17:07:04 2013 New Revision: 249625 URL: http://svnweb.freebsd.org/changeset/base/249625 Log: Introduce kern.timecounter.smp_tsc_adjust tunable (disabled by default) and respective functionality, allowing to synchronize TSC on APs to match BSP's during boot. It may be unsafe in general case due to theoretical chance of later drift if CPUs are using different clock rate or source, but it allows to use TSC in some cases when difference caused by some initialization bug, while TSCs are known to increment synchronously. Reviewed by: jimharris, kib MFC after: 1 month Modified: head/sys/x86/x86/tsc.c Modified: head/sys/x86/x86/tsc.c ============================================================================== --- head/sys/x86/x86/tsc.c Thu Apr 18 13:19:41 2013 (r249624) +++ head/sys/x86/x86/tsc.c Thu Apr 18 17:07:04 2013 (r249625) @@ -65,6 +65,11 @@ int smp_tsc; SYSCTL_INT(_kern_timecounter, OID_AUTO, smp_tsc, CTLFLAG_RDTUN, &smp_tsc, 0, "Indicates whether the TSC is safe to use in SMP mode"); TUNABLE_INT("kern.timecounter.smp_tsc", &smp_tsc); + +int smp_tsc_adjust = 0; +SYSCTL_INT(_kern_timecounter, OID_AUTO, smp_tsc_adjust, CTLFLAG_RDTUN, + &smp_tsc_adjust, 0, "Try to adjust TSC on APs to match BSP"); +TUNABLE_INT("kern.timecounter.smp_tsc_adjust", &smp_tsc_adjust); #endif static int tsc_shift = 1; @@ -403,25 +408,77 @@ comp_smp_tsc(void *arg) } } +static void +adj_smp_tsc(void *arg) +{ + uint64_t *tsc; + int64_t d, min, max; + u_int cpu = PCPU_GET(cpuid); + u_int first, i, size; + + first = CPU_FIRST(); + if (cpu == first) + return; + min = INT64_MIN; + max = INT64_MAX; + size = (mp_maxid + 1) * 3; + for (i = 0, tsc = arg; i < N; i++, tsc += size) { + d = tsc[first * 3] - tsc[cpu * 3 + 1]; + if (d > min) + min = d; + d = tsc[first * 3 + 1] - tsc[cpu * 3 + 2]; + if (d > min) + min = d; + d = tsc[first * 3 + 1] - tsc[cpu * 3]; + if (d < max) + max = d; + d = tsc[first * 3 + 2] - tsc[cpu * 3 + 1]; + if (d < max) + max = d; + } + if (min > max) + return; + d = min / 2 + max / 2; + __asm __volatile ( + "movl $0x10, %%ecx\n\t" + "rdmsr\n\t" + "addl %%edi, %%eax\n\t" + "adcl %%esi, %%edx\n\t" + "wrmsr\n" + : /* No output */ + : "D" ((uint32_t)d), "S" ((uint32_t)(d >> 32)) + : "ax", "cx", "dx", "cc" + ); +} + static int test_tsc(void) { uint64_t *data, *tsc; - u_int i, size; + u_int i, size, adj; if ((!smp_tsc && !tsc_is_invariant) || vm_guest) return (-100); size = (mp_maxid + 1) * 3; data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK); + adj = 0; +retry: for (i = 0, tsc = data; i < N; i++, tsc += size) smp_rendezvous(tsc_read_0, tsc_read_1, tsc_read_2, tsc); smp_tsc = 1; /* XXX */ smp_rendezvous(smp_no_rendevous_barrier, comp_smp_tsc, smp_no_rendevous_barrier, data); + if (!smp_tsc && adj < smp_tsc_adjust) { + adj++; + smp_rendezvous(smp_no_rendevous_barrier, adj_smp_tsc, + smp_no_rendevous_barrier, data); + goto retry; + } free(data, M_TEMP); if (bootverbose) - printf("SMP: %sed TSC synchronization test\n", - smp_tsc ? "pass" : "fail"); + printf("SMP: %sed TSC synchronization test%s\n", + smp_tsc ? "pass" : "fail", + adj > 0 ? " after adjustment" : ""); if (smp_tsc && tsc_is_invariant) { switch (cpu_vendor_id) { case CPU_VENDOR_AMD: