From owner-freebsd-net@FreeBSD.ORG  Thu May  3 22:33:54 2012
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 0CAD5106566C;
	Thu,  3 May 2012 22:33:54 +0000 (UTC)
	(envelope-from seanbru@yahoo-inc.com)
Received: from mrout1-b.corp.bf1.yahoo.com (mrout1-b.corp.bf1.yahoo.com
	[98.139.253.104])
	by mx1.freebsd.org (Postfix) with ESMTP id BE3858FC08;
	Thu,  3 May 2012 22:33:53 +0000 (UTC)
Received: from [IPv6:::1] (rideseveral.corp.yahoo.com [10.73.160.231])
	by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id
	q43MXcwu065325; Thu, 3 May 2012 15:33:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com;
	s=cobra; t=1336084419;
	bh=aJivQP/l8ZF5ns2Pt5ngFfycEU1p9CKLvMdIaSwq3hE=;
	h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date:
	Message-ID:Mime-Version:Content-Transfer-Encoding;
	b=bjWbTu5m8gRtH/PlRSy4KBbmqNQym/oe1fhlGvWUUkAMKGqPl1y76u6sk+pRDU6jo
	3SZWvY4lD/q1c9HQWBp8CwUZe7OQida/X1Fe5sgHDwj3ei4Q4CIRFSHzYDWuZdpkRa
	9u4Fv3jTtz7c4M6e8TZ5kJjRWPRqSdRRk6xQc4RA=
From: Sean Bruno <seanbru@yahoo-inc.com>
To: John Baldwin <jhb@freebsd.org>
In-Reply-To: <1335382225.2722.6.camel@powernoodle-l7.corp.yahoo.com>
References: <1335312667.11564.13.camel@powernoodle-l7.corp.yahoo.com>
	<201204250932.21378.jhb@freebsd.org>
	<1335382225.2722.6.camel@powernoodle-l7.corp.yahoo.com>
Content-Type: text/plain; charset="UTF-8"
Date: Thu, 03 May 2012 15:33:38 -0700
Message-ID: <1336084418.3077.21.camel@powernoodle-l7.corp.yahoo.com>
Mime-Version: 1.0
X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Milter-Version: master.31+4-gbc07cd5+
X-CLX-ID: 084418003
Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>,
	Jack Vogel <jfvogel@gmail.com>
Subject: Re: igb(4) Pondering a bind to cpu patch
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 03 May 2012 22:33:54 -0000



On Wed, 2012-04-25 at 12:30 -0700, Sean Bruno wrote:
> On Wed, 2012-04-25 at 06:32 -0700, John Baldwin wrote:
> > CPU IDs are not guaranteed to be dense.  However, you can use
> > CPU_FIRST() and 
> > CPU_NEXT() with your static global instead.
> > 
> Ah, does CPU_NEXT() reset to 0 when it reaches the end of its list of
> CPUs?
> 

Ah, I see.  So, yeah, here's a v2 of the patch that does "the right"
thing with non-sparse cpus, mulitple queues, and mulitple physical
interfaces.

http://people.freebsd.org/~sbruno/if_igb.c.txt

> 
> > OTOH, if igb were to just leave the interrupts alone instead of
> > binding them 
> > by hand, they would get round-robin assigned among available cores
> > already.  I 
> > think in this case the best approach might be to add a tunable to
> > disable 
> > igb's manual binding and instead let the default system round-robin
> > be 
> > preserved. 
> 
> also, yes.  Why *are* we binding to CPUs in the first place?  Are we
> afraid that the scheduler won't do the right thing and we're trying to
> work around some unknown performance issue ?
> 
> Sean
> 

Still haven't seen a good reason to bind the queues by default in the
first place.  

Sean