From owner-svn-src-user@FreeBSD.ORG Mon Jun 8 07:44:29 2009
Return-Path: <owner-svn-src-user@FreeBSD.ORG>
Delivered-To: svn-src-user@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
by hub.freebsd.org (Postfix) with ESMTP id 280AD106566B;
Mon, 8 Jun 2009 07:44:29 +0000 (UTC)
(envelope-from kmacy@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c])
by mx1.freebsd.org (Postfix) with ESMTP id 16BF58FC17;
Mon, 8 Jun 2009 07:44:29 +0000 (UTC)
(envelope-from kmacy@FreeBSD.org)
Received: from svn.freebsd.org (localhost [127.0.0.1])
by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n587iSx0019977;
Mon, 8 Jun 2009 07:44:28 GMT (envelope-from kmacy@svn.freebsd.org)
Received: (from kmacy@localhost)
by svn.freebsd.org (8.14.3/8.14.3/Submit) id n587iSVQ019974;
Mon, 8 Jun 2009 07:44:28 GMT (envelope-from kmacy@svn.freebsd.org)
Message-Id: <200906080744.n587iSVQ019974@svn.freebsd.org>
From: Kip Macy <kmacy@FreeBSD.org>
Date: Mon, 8 Jun 2009 07:44:28 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-user@freebsd.org
X-SVN-Group: user
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Cc:
Subject: svn commit: r193693 - in user/kmacy/releng_7_2_fcs/sys: net sys
X-BeenThere: svn-src-user@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "SVN commit messages for the experimental " user"
src tree" <svn-src-user.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-user>,
<mailto:svn-src-user-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-user>
List-Post: <mailto:svn-src-user@freebsd.org>
List-Help: <mailto:svn-src-user-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-user>,
<mailto:svn-src-user-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 08 Jun 2009 07:44:29 -0000
Author: kmacy
Date: Mon Jun 8 07:44:28 2009
New Revision: 193693
URL: http://svn.freebsd.org/changeset/base/193693
Log:
- add prefetch to single consumer dequeue
- remove critical section - protected by client lock
- remove gratuitous memory barrier from buf_ring_peek
Modified:
user/kmacy/releng_7_2_fcs/sys/net/flowtable.c
user/kmacy/releng_7_2_fcs/sys/net/if_var.h
user/kmacy/releng_7_2_fcs/sys/sys/buf_ring.h
Modified: user/kmacy/releng_7_2_fcs/sys/net/flowtable.c
==============================================================================
--- user/kmacy/releng_7_2_fcs/sys/net/flowtable.c Mon Jun 8 07:42:19 2009 (r193692)
+++ user/kmacy/releng_7_2_fcs/sys/net/flowtable.c Mon Jun 8 07:44:28 2009 (r193693)
@@ -185,20 +185,6 @@ static struct cv flowclean_cv;
static struct mtx flowclean_lock;
static uint64_t flowclean_cycles;
-#if defined(__i386__) || defined(__amd64__)
-static __inline
-void prefetch(void *x)
-{
- __asm volatile("prefetcht0 %0" :: "m" (*(unsigned long *)x));
-}
-#else
-static __inline
-void prefetch(void *x)
-{
- ;
-}
-#endif
-
/*
* TODO:
* - Make flowtable stats per-cpu, aggregated at sysctl call time,
Modified: user/kmacy/releng_7_2_fcs/sys/net/if_var.h
==============================================================================
--- user/kmacy/releng_7_2_fcs/sys/net/if_var.h Mon Jun 8 07:42:19 2009 (r193692)
+++ user/kmacy/releng_7_2_fcs/sys/net/if_var.h Mon Jun 8 07:44:28 2009 (r193693)
@@ -561,9 +561,9 @@ drbr_stats_update(struct ifnet *ifp, int
{
#ifndef NO_SLOW_STATS
ifp->if_obytes += len;
-#endif
if (mflags & M_MCAST)
ifp->if_omcasts++;
+#endif
}
static __inline int
Modified: user/kmacy/releng_7_2_fcs/sys/sys/buf_ring.h
==============================================================================
--- user/kmacy/releng_7_2_fcs/sys/sys/buf_ring.h Mon Jun 8 07:42:19 2009 (r193692)
+++ user/kmacy/releng_7_2_fcs/sys/sys/buf_ring.h Mon Jun 8 07:44:28 2009 (r193693)
@@ -181,24 +181,27 @@ buf_ring_dequeue_mc(struct buf_ring *br)
static __inline void *
buf_ring_dequeue_sc(struct buf_ring *br)
{
- uint32_t cons_head, cons_next;
+ uint32_t cons_head, cons_next, cons_next_next;
uint32_t prod_tail;
void *buf;
- critical_enter();
cons_head = br->br_cons_head;
prod_tail = br->br_prod_tail;
cons_next = (cons_head + 1) & br->br_cons_mask;
-
- if (cons_head == prod_tail) {
- critical_exit();
+ cons_next_next = (cons_head + 2) & br->br_cons_mask;
+
+ if (cons_head == prod_tail)
return (NULL);
+
+ if (cons_next != prod_tail) {
+ prefetch(br->br_ring[cons_next]);
+ if (cons_next_next != prod_tail)
+ prefetch(br->br_ring[cons_next_next]);
}
-
br->br_cons_head = cons_next;
buf = br->br_ring[cons_head];
-
+
#ifdef DEBUG_BUFRING
br->br_ring[cons_head] = NULL;
if (!mtx_owned(br->br_lock))
@@ -208,7 +211,6 @@ buf_ring_dequeue_sc(struct buf_ring *br)
br->br_cons_tail, cons_head);
#endif
br->br_cons_tail = cons_next;
- critical_exit();
return (buf);
}
@@ -225,7 +227,12 @@ buf_ring_peek(struct buf_ring *br)
if ((br->br_lock != NULL) && !mtx_owned(br->br_lock))
panic("lock not held on single consumer dequeue");
#endif
- wmb();
+ /*
+ * I believe it is safe to not have a memory barrier
+ * here because we control cons and tail is worst case
+ * a lagging indicator so we worst case we might
+ * return NULL immediately after a buffer has been enqueued
+ */
if (br->br_cons_head == br->br_prod_tail)
return (NULL);