From owner-freebsd-net@FreeBSD.ORG  Tue Jan 22 17:14:58 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id DFBA83CE;
 Tue, 22 Jan 2013 17:14:58 +0000 (UTC) (envelope-from jhb@freebsd.org)
Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net
 [IPv6:2001:470:1f10:75::2])
 by mx1.freebsd.org (Postfix) with ESMTP id BD6016A5;
 Tue, 22 Jan 2013 17:14:58 +0000 (UTC)
Received: from pakbsde14.localnet (unknown [38.105.238.108])
 by bigwig.baldwin.cx (Postfix) with ESMTPSA id 349B1B96B;
 Tue, 22 Jan 2013 12:14:58 -0500 (EST)
From: John Baldwin <jhb@freebsd.org>
To: Adrian Chadd <adrian@freebsd.org>
Subject: Re: two problems in dev/e1000/if_lem.c::lem_handle_rxtx()
Date: Tue, 22 Jan 2013 12:12:16 -0500
User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p22; KDE/4.5.5; amd64; ; )
References: <1358610450.75691.YahooMailClassic@web121604.mail.ne1.yahoo.com>
 <201301191114.29959.jhb@freebsd.org>
 <CAJ-Vmomd1ivZjWdiC8_O1qwim2dctq1o+y5=UH2eivU4NdCOAQ@mail.gmail.com>
In-Reply-To: <CAJ-Vmomd1ivZjWdiC8_O1qwim2dctq1o+y5=UH2eivU4NdCOAQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Message-Id: <201301221212.16628.jhb@freebsd.org>
X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7
 (bigwig.baldwin.cx); Tue, 22 Jan 2013 12:14:58 -0500 (EST)
Cc: Barney Cordoba <barney_cordoba@yahoo.com>, Luigi Rizzo <rizzo@iet.unipi.it>,
 freebsd-net@freebsd.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 22 Jan 2013 17:14:58 -0000

On Saturday, January 19, 2013 8:19:19 pm Adrian Chadd wrote:
> On 19 January 2013 08:14, John Baldwin <jhb@freebsd.org> wrote:
> 
> > However, I did describe an alternate setup where you can fix this.  Part of
> > the key is to get various NICs to share a single logical queue of tasks.  You
> > could simulate this now by having all the deferred tasks share a single
> > taskqueue with a pool of tasks, but that will still not fully cooperate with
> > ithreads.  To do that you have to get the interrupt handlers themselves into
> > the shared taskqueue.  Some changes I have in a p4 branch allow you to do that
> > by letting interrupt handlers reschedule themselves (avoiding the need for a
> > separate task and preventing the task from running concurrently with the
> > interrupt handler) and providing some (but not yet all) of the framework to
> > allow multiple devices to share a single work queue backed by a shared pool of
> > threads.
> 
> How would that work when I want to pin devices to specific cores?

Note that the setup allows you to bind things however you want.  By default it
uses the current model (each IRQ uses a dedicated queue with a single thread).
The idea is to provide the flexbility so that you can glue things together in
whatever way makes the most sense.  In a router that tends to get into livelock
using a shared queue may make more sense.  However, you are not forced to use that
for other workloads where it does not.

-- 
John Baldwin