From owner-svn-src-projects@FreeBSD.ORG  Wed Jan 23 16:28:35 2013
Return-Path: <owner-svn-src-projects@FreeBSD.ORG>
Delivered-To: svn-src-projects@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id 1A9C4946;
 Wed, 23 Jan 2013 16:28:35 +0000 (UTC) (envelope-from kib@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org
 [IPv6:2001:1900:2254:2068::e6a:0])
 by mx1.freebsd.org (Postfix) with ESMTP id 0DC6BED9;
 Wed, 23 Jan 2013 16:28:35 +0000 (UTC)
Received: from svn.freebsd.org ([127.0.1.70])
 by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id r0NGSY4v067777;
 Wed, 23 Jan 2013 16:28:34 GMT (envelope-from kib@svn.freebsd.org)
Received: (from kib@localhost)
 by svn.freebsd.org (8.14.5/8.14.5/Submit) id r0NGSYsD067776;
 Wed, 23 Jan 2013 16:28:34 GMT (envelope-from kib@svn.freebsd.org)
Message-Id: <201301231628.r0NGSYsD067776@svn.freebsd.org>
From: Konstantin Belousov <kib@FreeBSD.org>
Date: Wed, 23 Jan 2013 16:28:34 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r245846 - projects/counters/sys/i386/include
X-SVN-Group: projects
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "SVN commit messages for the src &quot; projects&quot;
 tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-src-projects>, 
 <mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-projects>, 
 <mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 23 Jan 2013 16:28:35 -0000

Author: kib
Date: Wed Jan 23 16:28:34 2013
New Revision: 245846
URL: http://svnweb.freebsd.org/changeset/base/245846

Log:
  Implement the counters for i386 without use of a critical section,
  when the cmpxchg8b instruction is avaliable. It relies on the same
  offset-from-curpcpu trick as the amd64 implementation. Due to
  the lack of the single instruction to increment 64bit value, the
  cas loop is performed.

Modified:
  projects/counters/sys/i386/include/counter.h

Modified: projects/counters/sys/i386/include/counter.h
==============================================================================
--- projects/counters/sys/i386/include/counter.h	Wed Jan 23 14:37:05 2013	(r245845)
+++ projects/counters/sys/i386/include/counter.h	Wed Jan 23 16:28:34 2013	(r245846)
@@ -30,23 +30,46 @@
 #define __MACHINE_COUNTER_H__
 
 #include <sys/pcpu.h>
+#include <machine/md_var.h>
+#include <machine/specialreg.h>
+
+static inline void
+counter_64_inc_8b(uint64_t *p, uint64_t inc)
+{
+
+	__asm __volatile(
+	"movl	%%fs:(%%esi),%%eax\n\t"
+	"movl	%%fs:4(%%esi),%%edx\n"
+"1:\n\t"
+	"movl	%%eax,%%ebx\n\t"
+	"movl	%%edx,%%ecx\n\t"
+	"addl	(%%edi),%%ebx\n\t"
+	"adcl	4(%%edi),%%ecx\n\t"
+	"cmpxchg8b %%fs:(%%esi)\n\t"
+	"jnz	1b"
+	:
+	: "S" (p), "D" (&inc)
+	: "memory", "cc", "eax", "edx", "ebx", "ecx");
+}
 
 static __inline void
 counter_u64_inc(counter_u64_t c, uint64_t inc)
 {
 
-	critical_enter();
-	*(uint64_t *)((char *)c + sizeof(struct pcpu) * curcpu) += inc;
-	critical_exit();
+	if ((cpu_feature & CPUID_CX8) == 0) {
+		critical_enter();
+		*(uint64_t *)((char *)c + sizeof(struct pcpu) * curcpu) += inc;
+		critical_exit();
+	} else {
+		counter_64_inc_8b(c, inc);
+	}
 }
 
 static __inline void
 counter_u64_dec(counter_u64_t c, uint64_t dec)
 {
 
-	critical_enter();
-	*(uint64_t *)((char *)c + sizeof(struct pcpu) * curcpu) -= dec;
-	critical_exit();
+	counter_u64_inc(c, -(int64_t)dec);
 }
 
 #endif	/* ! __MACHINE_COUNTER_H__ */