From owner-freebsd-amd64@FreeBSD.ORG Mon Feb 7 22:25:50 2005 Return-Path: Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6906016A4DB for ; Mon, 7 Feb 2005 22:25:50 +0000 (GMT) Received: from mail26.sea5.speakeasy.net (mail26.sea5.speakeasy.net [69.17.117.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C53843D66 for ; Mon, 7 Feb 2005 22:25:49 +0000 (GMT) (envelope-from jhb@FreeBSD.org) Received: (qmail 9554 invoked from network); 7 Feb 2005 22:25:49 -0000 Received: from server.baldwin.cx ([216.27.160.63]) (envelope-sender )AES256-SHA encrypted SMTP for ; 7 Feb 2005 22:25:48 -0000 Received: from [10.50.40.202] (gw1.twc.weather.com [216.133.140.1]) (authenticated bits=0) by server.baldwin.cx (8.13.1/8.13.1) with ESMTP id j17MPRrh088725; Mon, 7 Feb 2005 17:25:42 -0500 (EST) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-amd64@FreeBSD.org Date: Mon, 7 Feb 2005 14:27:42 -0500 User-Agent: KMail/1.6.2 References: <200502041558.28521.jhb@FreeBSD.org> In-Reply-To: <200502041558.28521.jhb@FreeBSD.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200502071427.42202.jhb@FreeBSD.org> X-Spam-Status: No, score=-102.8 required=4.2 tests=ALL_TRUSTED, USER_IN_WHITELIST autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on server.baldwin.cx cc: amd64@FreeBSD.org cc: peter@FreeBSD.org Subject: Re: [PATCH] Change atomic operations to use fences for memory barriers X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2005 22:25:50 -0000 On Friday 04 February 2005 03:58 pm, John Baldwin wrote: > The patch below changes the atomic operations on amd64 to use the cheaper > fence instructions for memory barriers. I'd like people to test it to see > if 1) it breaks things or not, and 2) if it impacts performance either in a > good way or a bad way. For this last I'm curious about both UP and SMP as > my initial guess is that it will help on UP but might hurt on SMP. Would help if I included it: --- //depot/vendor/freebsd/src/sys/amd64/include/atomic.h 2003/11/21 03:05:42 +++ //depot/projects/smpng/sys/amd64/include/atomic.h 2004/11/19 20:16:10 @@ -70,7 +70,7 @@ int atomic_cmpset_int(volatile u_int *dst, u_int exp, u_int src); int atomic_cmpset_long(volatile u_long *dst, u_long exp, u_long src); -#define ATOMIC_STORE_LOAD(TYPE, LOP, SOP) \ +#define ATOMIC_STORE_LOAD(TYPE) \ u_##TYPE atomic_load_acq_##TYPE(volatile u_##TYPE *p); \ void atomic_store_rel_##TYPE(volatile u_##TYPE *p, u_##TYPE v) @@ -162,30 +162,22 @@ #if defined(__GNUC__) -#define ATOMIC_STORE_LOAD(TYPE, LOP, SOP) \ +#define ATOMIC_STORE_LOAD(TYPE) \ static __inline u_##TYPE \ atomic_load_acq_##TYPE(volatile u_##TYPE *p) \ { \ - u_##TYPE res; \ + u_##TYPE v; \ \ - __asm __volatile(__XSTRING(MPLOCKED) LOP \ - : "=a" (res), /* 0 (result) */\ - "+m" (*p) /* 1 */ \ - : : "memory"); \ - \ - return (res); \ + v = *p; \ + __asm __volatile("lfence" ::: "memory"); \ + return (v); \ } \ \ -/* \ - * The XCHG instruction asserts LOCK automagically. \ - */ \ static __inline void \ atomic_store_rel_##TYPE(volatile u_##TYPE *p, u_##TYPE v)\ { \ - __asm __volatile(SOP \ - : "+m" (*p), /* 0 */ \ - "+r" (v) /* 1 */ \ - : : "memory"); \ + __asm __volatile("sfence" ::: "memory"); \ + *p = v; \ } \ struct __hack @@ -194,7 +186,7 @@ extern int atomic_cmpset_int(volatile u_int *, u_int, u_int); extern int atomic_cmpset_long(volatile u_long *, u_long, u_long); -#define ATOMIC_STORE_LOAD(TYPE, LOP, SOP) \ +#define ATOMIC_STORE_LOAD(TYPE) \ extern u_##TYPE atomic_load_acq_##TYPE(volatile u_##TYPE *p); \ extern void atomic_store_rel_##TYPE(volatile u_##TYPE *p, u_##TYPE v) @@ -222,10 +214,10 @@ ATOMIC_ASM(add, long, "addq %1,%0", "ir", v); ATOMIC_ASM(subtract, long, "subq %1,%0", "ir", v); -ATOMIC_STORE_LOAD(char, "cmpxchgb %b0,%1", "xchgb %b1,%0"); -ATOMIC_STORE_LOAD(short,"cmpxchgw %w0,%1", "xchgw %w1,%0"); -ATOMIC_STORE_LOAD(int, "cmpxchgl %0,%1", "xchgl %1,%0"); -ATOMIC_STORE_LOAD(long, "cmpxchgq %0,%1", "xchgq %1,%0"); +ATOMIC_STORE_LOAD(char); +ATOMIC_STORE_LOAD(short); +ATOMIC_STORE_LOAD(int); +ATOMIC_STORE_LOAD(long); #undef ATOMIC_ASM #undef ATOMIC_STORE_LOAD --- //depot/vendor/freebsd/src/sys/amd64/include/bus_amd64.h 2005/01/05 20:20:40 +++ //depot/projects/smpng/sys/amd64/include/bus_amd64.h 2005/01/05 22:38:38 @@ -1215,9 +1215,9 @@ { #ifdef __GNUC__ if (flags & BUS_SPACE_BARRIER_READ) - __asm __volatile("lock; addl $0,0(%%rsp)" : : : "memory"); + __asm __volatile("lfence" : : : "memory"); else - __asm __volatile("" : : : "memory"); + __asm __volatile("sfence" : : : "memory"); #endif } -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org