From owner-freebsd-stable@freebsd.org  Wed Jul 22 18:34:06 2015
Return-Path: <owner-freebsd-stable@freebsd.org>
Delivered-To: freebsd-stable@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 063F39A8DF6;
 Wed, 22 Jul 2015 18:34:06 +0000 (UTC)
 (envelope-from gpalmer@freebsd.org)
Received: from mail.in-addr.com (mail.in-addr.com
 [IPv6:2a01:4f8:191:61e8::2525:2525])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id C0BF51889;
 Wed, 22 Jul 2015 18:34:05 +0000 (UTC)
 (envelope-from gpalmer@freebsd.org)
Received: from gjp by mail.in-addr.com with local (Exim 4.85 (FreeBSD))
 (envelope-from <gpalmer@freebsd.org>)
 id 1ZHyqC-000Ow0-1b; Wed, 22 Jul 2015 19:34:04 +0100
Date: Wed, 22 Jul 2015 19:34:04 +0100
From: Gary Palmer <gpalmer@freebsd.org>
To: Mark Saad <nonesuch@longcount.org>
Cc: Garrett Cooper <yaneurabeya@gmail.com>,
 "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>,
 FreeBSD-Stable ML <freebsd-stable@freebsd.org>
Subject: Re: Interesting Boot failure on HEAD with a large number of IGB nics
Message-ID: <20150722183404.GF41419@in-addr.com>
References: <CAMXt9NbJ5quUXwLhLzU9KzpJ0cXGCbHsnmc1D-L3ycKktn6j7A@mail.gmail.com>
 <20150722171419.GE41419@in-addr.com>
 <95653312-0F92-40E4-89EE-E8235403D3B9@gmail.com>
 <CAMXt9Naxhk7qqkU8-tjBR_=Y87HYMUHrE8QmK2yc-yabXnMQfQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAMXt9Naxhk7qqkU8-tjBR_=Y87HYMUHrE8QmK2yc-yabXnMQfQ@mail.gmail.com>
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: gpalmer@freebsd.org
X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Jul 2015 18:34:06 -0000

On Wed, Jul 22, 2015 at 02:20:06PM -0400, Mark Saad wrote:
> On Wed, Jul 22, 2015 at 1:54 PM, Garrett Cooper <yaneurabeya@gmail.com>
> wrote:
> 
> >
> > > On Jul 22, 2015, at 10:14, Gary Palmer <gpalmer@freebsd.org> wrote:
> >
> > ...
> >
> > > I'm no expert, but you may want to try setting
> > >
> > > hw.igb.num_queues=1
> > >
> > > and maybe
> > >
> > > hw.ixgbe.num_queues=1
> > >
> > > in the boot loader and trying that.
> >
> > There was another discussion that took place around June on current that
> > might be helpful. I think the solution boiled down to what Gary described
> > above, because the driver auto tuning was broken...
> > Thanks!
> > -NGie
> 
> 
> NGie
>  I'll try the queues option but; why was does that kill my ehci hub ? Note
> 
> ehci0: <Intel Patsburg USB 2.0 controller> mem 0xdd923000-0xdd9233ff irq 16
> at device 26.0 on pci0
> panic: Couldn't find an APIC vector for IRQ 16

I believe the error message could be better written as "Couldn't find a
free APIC vector".  The IGB cards allocted multiple MSI-X interrupts for
multi-queue use and exhausted the available pool, and the USB hub (ehci0)
just happened to be the next thing probed by the PCI bus and tripped the
panic.

At least that is my understanding.

Regards,

Gary