From owner-freebsd-net@FreeBSD.ORG  Thu Jun 20 10:34:54 2013
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id 7BC596D
 for <freebsd-net@freebsd.org>; Thu, 20 Jun 2013 10:34:54 +0000 (UTC)
 (envelope-from lars@netapp.com)
Received: from mx12.netapp.com (mx12.netapp.com [216.240.18.77])
 by mx1.freebsd.org (Postfix) with ESMTP id 68C2A1B3B
 for <freebsd-net@freebsd.org>; Thu, 20 Jun 2013 10:34:54 +0000 (UTC)
X-IronPort-AV: E=Sophos;i="4.87,904,1363158000"; d="scan'208";a="66762209"
Received: from vmwexceht03-prd.hq.netapp.com ([10.106.76.241])
 by mx12-out.netapp.com with ESMTP; 20 Jun 2013 03:34:47 -0700
Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.35]) by
 vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.03.0123.003;
 Thu, 20 Jun 2013 03:34:47 -0700
From: "Eggert, Lars" <lars@netapp.com>
To: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject: hw.igb.num_queues default
Thread-Topic: hw.igb.num_queues default
Thread-Index: AQHObaHIf6463S6M5065zbiLDwqGaw==
Date: Thu, 20 Jun 2013 10:34:46 +0000
Message-ID: <843F7891-FD87-4F16-A279-B45D4A674F4E@netapp.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.106.53.51]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <23E91D958892F145A440FDBC87021AA5@hq.netapp.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: Jack Vogel <jfvogel@gmail.com>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 20 Jun 2013 10:34:54 -0000

Hi,

I just popped a new four-port igb card into a -STABLE system and encountere=
d severe issues even when unloaded right after boot, to the point where I c=
ouldn't even ssh into the system anymore. The box has 2x4 cores:

CPU: Intel(R) Xeon(R) CPU           X5450  @ 3.00GHz (2992.60-MHz K8-class =
CPU)
  Origin =3D "GenuineIntel"  Id =3D 0x10676  Family =3D 0x6  Model =3D 0x17=
  Stepping =3D 6
  Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PG=
E,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=3D0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PD=
CM,DCA,SSE4.1>
  AMD Features=3D0x20100800<SYSCALL,NX,LM>
  AMD Features2=3D0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  =3D 8589934592 (8192 MB)
avail memory =3D 8239513600 (7857 MB)
MPTable: <DELL     PE 01B2     >
Event timer "LAPIC" quality 400
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)

By default, the igb driver seems to set up one queue per detected CPU. Goog=
ling around, people seemed to suggest that limiting the number of queues ma=
kes things work better. I can confirm that setting hw.igb.num_queues=3D2 se=
ems to have fixed the issue. (Two was the first value I tried, maybe other =
values other than 0 would work, too.)

In order to uphold POLA, should the igb driver maybe default to a conservat=
ive value for hw.igb.num_queues that may not deliver optimal performance, b=
ut at least works out of the box?

Lars=