From owner-freebsd-stable@FreeBSD.ORG Fri Oct 15 16:29:04 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B4AD1065695 for ; Fri, 15 Oct 2010 16:29:04 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2E8068FC1E for ; Fri, 15 Oct 2010 16:29:03 +0000 (UTC) Received: by wwb39 with SMTP id 39so1211979wwb.31 for ; Fri, 15 Oct 2010 09:29:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=vPAgKoQ91hkTu5kgImiU1OWV6XgJv7ktnvSaQvNFf9c=; b=bICoz5VJq7vKvMbS9GMAmHy2dmf+yYVeJAFXXwx8IBu0bczn/DrCqus9QOlXgzEnJj D6Qojan0Ms5QjKCYnDmSFsVWE7bzVN053HUy5Wjzlk8Tw7EguhxcfvBn2sTNN+yK2Jpm 9J1JXzEecaSDeILVZ6iq0J0UVGcFtwCkyI2Qw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=QtzgfGKNiEQ44p+Yrl89rwlhp/WELI/ZXNybwt5gvNba1NqgxoJJ+n3kuobo+6cxt9 iwZLsgXdi3AJ4tvIVRs29g9WwCC6gRoOhB7OfCCSCDfNMLs2D7DW/+tqi/mqGXZCY2x7 m4y5hCZiBcoxXHU4pIIkH7uPK6pF76rAb9XTE= MIME-Version: 1.0 Received: by 10.216.10.69 with SMTP id 47mr1045975weu.86.1287160142319; Fri, 15 Oct 2010 09:29:02 -0700 (PDT) Received: by 10.216.48.21 with HTTP; Fri, 15 Oct 2010 09:29:02 -0700 (PDT) In-Reply-To: <01NT25TZHDDY008CFX@tmk.com> References: <01NT1YE1I98Q008KN2@tmk.com> <01NT25TZHDDY008CFX@tmk.com> Date: Fri, 15 Oct 2010 09:29:02 -0700 Message-ID: From: Jack Vogel To: Terry Kennedy Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: Bogus "igb1: Could not setup receive structures" in 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Oct 2010 16:29:04 -0000 The number of MSIX vectors it uses is the number of queues PLUS one vector for link. I would use two or four rather than 3, but it should be ok with that if that's what you wish. Jack On Thu, Oct 14, 2010 at 6:58 PM, Terry Kennedy wrote: > > The problem is mbuf resources, the driver is autoconfiguring the number > of > > queues based on the number of cores, on newer systems with lots of them > > this is outstripping the mbuf resource pool. > > That would make sense, as these systems have 16 cores (dual E5520's). > > > I have decided to hard limit the queues to 8, you can fix the number > > manually > > by searching for num_queues in if_igb.c and setting it to something other > > than > > 0 for now. > > I changed it to 8, and saw the same problem. I noted that the igb boot > messages changed from: > > Oct 14 18:28:02 rz1m kernel: igb0: Using MSIX interrupts with 10 vectors > Oct 14 18:28:02 rz1m kernel: igb1: Using MSIX interrupts with 10 vectors > > to: > > Oct 14 21:53:44 rz1m kernel: igb0: Using MSIX interrupts with 9 vectors > Oct 14 21:53:44 rz1m kernel: igb1: Using MSIX interrupts with 9 vectors > > So I dropped the value to 3 (on the assumption that the system uses one > more than the specified value per interface), and got: > > igb0: Using MSIX interrupts with 4 vectors > igb1: Using MSIX interrupts with 4 vectors > > and both igb interfaces came up. I didn't try to find the maximum > number of queues that would work. > > > I am at work on a number of issues with igb and em right now which is why > > there has not been an MFC yet. > > Understood. Thanks for the quick response and workaround. > > Terry Kennedy http://www.tmk.com > terry@tmk.com New York, NY USA >