From owner-svn-src-head@FreeBSD.ORG  Thu Dec 27 12:36:58 2012
Return-Path: <owner-svn-src-head@FreeBSD.ORG>
Delivered-To: svn-src-head@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
 by hub.freebsd.org (Postfix) with ESMTP id D6224192;
 Thu, 27 Dec 2012 12:36:58 +0000 (UTC)
 (envelope-from attilio@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org
 [IPv6:2001:1900:2254:2068::e6a:0])
 by mx1.freebsd.org (Postfix) with ESMTP id BA2B08FC0A;
 Thu, 27 Dec 2012 12:36:58 +0000 (UTC)
Received: from svn.freebsd.org (localhost [127.0.0.1])
 by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qBRCawUZ078204;
 Thu, 27 Dec 2012 12:36:58 GMT (envelope-from attilio@svn.freebsd.org)
Received: (from attilio@localhost)
 by svn.freebsd.org (8.14.5/8.14.5/Submit) id qBRCawuU078203;
 Thu, 27 Dec 2012 12:36:58 GMT (envelope-from attilio@svn.freebsd.org)
Message-Id: <201212271236.qBRCawuU078203@svn.freebsd.org>
From: Attilio Rao <attilio@FreeBSD.org>
Date: Thu, 27 Dec 2012 12:36:58 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
 svn-src-head@freebsd.org
Subject: svn commit: r244732 - head/sys/sys
X-SVN-Group: head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-head@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: SVN commit messages for the src tree for head/-current
 <svn-src-head.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-src-head>,
 <mailto:svn-src-head-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-head>
List-Post: <mailto:svn-src-head@freebsd.org>
List-Help: <mailto:svn-src-head-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-head>,
 <mailto:svn-src-head-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Dec 2012 12:36:58 -0000

Author: attilio
Date: Thu Dec 27 12:36:58 2012
New Revision: 244732
URL: http://svnweb.freebsd.org/changeset/base/244732

Log:
  br_prod_tail and br_cons_tail members are used as barrier to
  signal bug_ring ownership. However, instructions can be reordered
  around members write leading to stale values for ie. br_prod_bufs.
  
  Use correct memory barriers to ensure proper ordering of the
  ownership tokens updates.
  
  Sponsored by:	EMC / Isilon storage division
  MFC after:	2 weeks

Modified:
  head/sys/sys/buf_ring.h

Modified: head/sys/sys/buf_ring.h
==============================================================================
--- head/sys/sys/buf_ring.h	Thu Dec 27 09:15:21 2012	(r244731)
+++ head/sys/sys/buf_ring.h	Thu Dec 27 12:36:58 2012	(r244732)
@@ -114,10 +114,10 @@ buf_ring_enqueue(struct buf_ring *br, vo
 	 * that preceeded us, we need to wait for them
 	 * to complete 
 	 */   
-	while (br->br_prod_tail != prod_head)
+	while (atomic_load_acq_32(&br->br_prod_tail) != prod_head)
 		cpu_spinwait();
 	br->br_prod_bufs++;
-	br->br_prod_tail = prod_next;
+	atomic_store_rel_32(&br->br_prod_tail, prod_next);
 	critical_exit();
 	return (0);
 }
@@ -161,10 +161,10 @@ buf_ring_dequeue_mc(struct buf_ring *br)
 	 * that preceeded us, we need to wait for them
 	 * to complete 
 	 */   
-	while (br->br_cons_tail != cons_head)
+	while (atomic_load_acq_32(&br->br_cons_tail) != cons_head)
 		cpu_spinwait();
 
-	br->br_cons_tail = cons_next;
+	atomic_store_rel_32(&br->br_cons_tail, cons_next);
 	critical_exit();
 
 	return (buf);
@@ -209,7 +209,7 @@ buf_ring_dequeue_sc(struct buf_ring *br)
 		panic("inconsistent list cons_tail=%d cons_head=%d",
 		    br->br_cons_tail, cons_head);
 #endif
-	br->br_cons_tail = cons_next;
+	atomic_store_rel_32(&br->br_cons_tail, cons_next);
 	return (buf);
 }