From owner-svn-src-all@FreeBSD.ORG  Tue Jul 24 22:10:12 2012
Return-Path: <owner-svn-src-all@FreeBSD.ORG>
Delivered-To: svn-src-all@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 7BF49106566B;
	Tue, 24 Jul 2012 22:10:12 +0000 (UTC)
	(envelope-from jimharris@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c])
	by mx1.freebsd.org (Postfix) with ESMTP id 66F608FC16;
	Tue, 24 Jul 2012 22:10:12 +0000 (UTC)
Received: from svn.freebsd.org (localhost [127.0.0.1])
	by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q6OMACQR079605;
	Tue, 24 Jul 2012 22:10:12 GMT
	(envelope-from jimharris@svn.freebsd.org)
Received: (from jimharris@localhost)
	by svn.freebsd.org (8.14.4/8.14.4/Submit) id q6OMACqV079603;
	Tue, 24 Jul 2012 22:10:12 GMT
	(envelope-from jimharris@svn.freebsd.org)
Message-Id: <201207242210.q6OMACqV079603@svn.freebsd.org>
From: Jim Harris <jimharris@FreeBSD.org>
Date: Tue, 24 Jul 2012 22:10:12 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
	svn-src-head@freebsd.org
X-SVN-Group: head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Cc: 
Subject: svn commit: r238755 - head/sys/x86/x86
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
	user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Jul 2012 22:10:12 -0000

Author: jimharris
Date: Tue Jul 24 22:10:11 2012
New Revision: 238755
URL: http://svn.freebsd.org/changeset/base/238755

Log:
  Add rmb() to tsc_read_##x to enforce serialization of rdtsc captures.
  
  Intel Architecture Manual specifies that rdtsc instruction is not serialized,
  so without this change, TSC synchronization test would periodically fail,
  resulting in use of HPET timecounter instead of TSC-low.  This caused
  severe performance degradation (40-50%) when running high IO/s workloads due to
  HPET MMIO reads and GEOM stat collection.
  
  Tests on Xeon E5-2600 (Sandy Bridge) 8C systems were seeing TSC synchronization
  fail approximately 20% of the time.
  
  Sponsored by: Intel
  Reviewed by: kib
  MFC after: 3 days

Modified:
  head/sys/x86/x86/tsc.c

Modified: head/sys/x86/x86/tsc.c
==============================================================================
--- head/sys/x86/x86/tsc.c	Tue Jul 24 20:15:41 2012	(r238754)
+++ head/sys/x86/x86/tsc.c	Tue Jul 24 22:10:11 2012	(r238755)
@@ -328,6 +328,7 @@ init_TSC(void)
 
 #ifdef SMP
 
+/* rmb is required here because rdtsc is not a serializing instruction. */
 #define	TSC_READ(x)			\
 static void				\
 tsc_read_##x(void *arg)			\
@@ -335,6 +336,7 @@ tsc_read_##x(void *arg)			\
 	uint32_t *tsc = arg;		\
 	u_int cpu = PCPU_GET(cpuid);	\
 					\
+	rmb();				\
 	tsc[cpu * 3 + x] = rdtsc32();	\
 }
 TSC_READ(0)