From owner-svn-src-all@freebsd.org Tue Nov 24 09:13:23 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BAE8A36DD7; Tue, 24 Nov 2015 09:13:23 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C3FB21553; Tue, 24 Nov 2015 09:13:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id tAO9DL4s032186; Tue, 24 Nov 2015 09:13:21 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id tAO9DLUC032185; Tue, 24 Nov 2015 09:13:21 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201511240913.tAO9DLUC032185@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 24 Nov 2015 09:13:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r291242 - head/sys/powerpc/include X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Nov 2015 09:13:23 -0000 Author: kib Date: Tue Nov 24 09:13:21 2015 New Revision: 291242 URL: https://svnweb.freebsd.org/changeset/base/291242 Log: On PowerPC 64bit, the linux-compat mb() definition is implemented with lwsync instruction, which does not provide Store/Load barrier. Fix this by using "full" sync barrier for mb(). atomic_store_rel() does not need full barrier, change mb() call there to the lwsync instruction if not hitting the known CPU erratas (i.e. on 32bit). Provide powerpc_lwsync() helper to isolate the lwsync/sync compile time selection, and use it in atomic_store_rel() and several other places which duplicate the code. Noted by: alc Reviewed and tested by: nwhitehorn Sponsored by: The FreeBSD Foundation Modified: head/sys/powerpc/include/atomic.h Modified: head/sys/powerpc/include/atomic.h ============================================================================== --- head/sys/powerpc/include/atomic.h Tue Nov 24 09:08:31 2015 (r291241) +++ head/sys/powerpc/include/atomic.h Tue Nov 24 09:13:21 2015 (r291242) @@ -48,7 +48,7 @@ */ #ifdef __powerpc64__ -#define mb() __asm __volatile("lwsync" : : : "memory") +#define mb() __asm __volatile("sync" : : : "memory") #define rmb() __asm __volatile("lwsync" : : : "memory") #define wmb() __asm __volatile("lwsync" : : : "memory") #define __ATOMIC_REL() __asm __volatile("lwsync" : : : "memory") @@ -61,6 +61,17 @@ #define __ATOMIC_ACQ() __asm __volatile("isync" : : : "memory") #endif +static __inline void +powerpc_lwsync(void) +{ + +#ifdef __powerpc64__ + __asm __volatile("lwsync" : : : "memory"); +#else + __asm __volatile("sync" : : : "memory"); +#endif +} + /* * atomic_add(p, v) * { *p += v; } @@ -506,7 +517,8 @@ atomic_load_acq_##TYPE(volatile u_##TYPE static __inline void \ atomic_store_rel_##TYPE(volatile u_##TYPE *p, u_##TYPE v) \ { \ - mb(); \ + \ + powerpc_lwsync(); \ *p = v; \ } @@ -734,34 +746,21 @@ static __inline void atomic_thread_fence_acq(void) { - /* See above comment about lwsync being broken on Book-E. */ -#ifdef __powerpc64__ - __asm __volatile("lwsync" : : : "memory"); -#else - __asm __volatile("sync" : : : "memory"); -#endif + powerpc_lwsync(); } static __inline void atomic_thread_fence_rel(void) { -#ifdef __powerpc64__ - __asm __volatile("lwsync" : : : "memory"); -#else - __asm __volatile("sync" : : : "memory"); -#endif + powerpc_lwsync(); } static __inline void atomic_thread_fence_acq_rel(void) { -#ifdef __powerpc64__ - __asm __volatile("lwsync" : : : "memory"); -#else - __asm __volatile("sync" : : : "memory"); -#endif + powerpc_lwsync(); } static __inline void