From owner-freebsd-net@FreeBSD.ORG  Wed Apr  8 13:16:54 2009
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 60A25106566C;
	Wed,  8 Apr 2009 13:16:54 +0000 (UTC)
	(envelope-from rwatson@FreeBSD.org)
Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42])
	by mx1.freebsd.org (Postfix) with ESMTP id 3D3A98FC17;
	Wed,  8 Apr 2009 13:16:54 +0000 (UTC)
	(envelope-from rwatson@FreeBSD.org)
Received: from fledge.watson.org (fledge.watson.org [65.122.17.41])
	by cyrus.watson.org (Postfix) with ESMTPS id D4DB646B86;
	Wed,  8 Apr 2009 09:16:53 -0400 (EDT)
Date: Wed, 8 Apr 2009 14:16:53 +0100 (BST)
From: Robert Watson <rwatson@FreeBSD.org>
X-X-Sender: robert@fledge.watson.org
To: Barney Cordoba <barney_cordoba@yahoo.com>
In-Reply-To: <871699.35154.qm@web63906.mail.re1.yahoo.com>
Message-ID: <alpine.BSF.2.00.0904081412540.61921@fledge.watson.org>
References: <871699.35154.qm@web63906.mail.re1.yahoo.com>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Cc: freebsd-net@freebsd.org, Ivan Voras <ivoras@freebsd.org>
Subject: Re: Advice on a multithreaded netisr  patch?
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 13:16:54 -0000


On Wed, 8 Apr 2009, Barney Cordoba wrote:

> Is there any work being done on lighter weight locks for queues? It seems 
> ridiculous to avoid using queues because of lock contention when the locks 
> are only protecting a couple lines of code.

My reading is that there are two, closely related, things going on: the first 
is lock contention, and the second is cache line contention.  We have a 
primitive in 8.x (don't think it's been MFC'd yet) for a lockless atomic 
buffer primitive for use in drivers and other parts of the stack.  However, 
that addresses only lock contention, not line contention, which at a high PPS 
will be an issue as well.  Only by moving to independent data structures 
(i.e., on independent cache lines) can we reduce line contention.

Robert N M Watson
Computer Laboratory
University of Cambridge