From owner-freebsd-net@FreeBSD.ORG Fri Nov 21 20:58:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA185520 for ; Fri, 21 Nov 2014 20:58:48 +0000 (UTC) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D9FAAE1 for ; Fri, 21 Nov 2014 20:58:48 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id l18so7447856wgh.26 for ; Fri, 21 Nov 2014 12:58:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ju2FayDWYjdLXEo/zg6VIcvppgvKNwnKSwrLYtmzA4I=; b=riUYAI0QbnoClSVLeaFxqQK0x6SJmWQnYeGykvphtTlIUvd7mPwAgJFqtzhCnRr+Aa RPhS8RtuSDWYQ0du03m3bsMmcGMW40a+h99AWEy4jcS0tLkSuaHh3tfXprODmX50opyQ hI1KTSAmZfbxvM9lJaxjSSthKTx6/n89P0m+7nvsSPBF/uGnZV9hXdd0xmlt9zZgqP4d g6BOeYdAGsyhWN8RXde4WMXuQ7j/Zv6BYmncDZYSQX7PWPyXWR6xR3JnkNaGk+1GWgMd H69M0c6l1KIFg0J7Ot3ZqznUvQ/Vx4x5wCHCYb5N77K16xxwzyr8RrUnh/J46Wrp8aw+ adHg== MIME-Version: 1.0 X-Received: by 10.180.99.1 with SMTP id em1mr286487wib.29.1416603526831; Fri, 21 Nov 2014 12:58:46 -0800 (PST) Received: by 10.194.76.34 with HTTP; Fri, 21 Nov 2014 12:58:46 -0800 (PST) In-Reply-To: <4a92a47b86474b92b5b4fd7601df9cc2@BLUPR0801MB674.namprd08.prod.outlook.com> References: <20141121114852.8b7d9a6c071c3e0799d5f30c@aei.mpg.de> <4a92a47b86474b92b5b4fd7601df9cc2@BLUPR0801MB674.namprd08.prod.outlook.com> Date: Fri, 21 Nov 2014 12:58:46 -0800 Message-ID: Subject: Re: igb Could not setup receive structures (again) From: Jack Vogel To: David DeSimone Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Nov 2014 20:58:49 -0000 The driver does not calculate what it needs, it just keeps taking :) Hmmm, well it would be difficult to calculate a real total, as each instance of the driver (one per port) has no idea about other instances. I suppose there could be a display of the total needed per port? Jack On Fri, Nov 21, 2014 at 12:34 PM, David DeSimone wrote: > Would it be possible for the driver to report how many clusters it > calculated that it needs, whenever it runs into this memory shortage duri= ng > attach? That way an administrator might have some idea how much to > increase their tunables in order to meet the driver's requirements. > > As it is, it's merely a guessing game or requires digging into driver > source to come up with the right numbers. > > > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org= ] > On Behalf Of Jack Vogel > Sent: Friday, November 21, 2014 10:23 AM > To: Gerrit K=FChn > Cc: FreeBSD Net > Subject: Re: igb Could not setup receive structures (again) > > The message is pretty straightforward, its only cause is the inability > of the driver to get the necessary clusters to populate the RX rings. > > After you get the error do a `netstat -m` and see what the state of the > 9K jumbo pool is, for that is the size you would be using. > > Depending on the specific device type you may have up to 8 rings > per interface, and with a ring size of 1K... > > It does seem like you should have enough, but maybe something else > in your system is using the pool? As I said, look at netstat, it should > give > you the truth, and then adjust the allocated size to fit your needs. > > Jack > > > On Fri, Nov 21, 2014 at 2:48 AM, Gerrit K=FChn > wrote: > > > Hi all, > > > > I get the error message above when trying to go for jumbo frames (mtu > > 9000) on a system here. I found a lot of comments on this, but they all > > state that this is due to a too low setting for mbufs in FreeBSD8/9 > > settings. > > However, I have 10-stable here, and the settings look rather high to me= : > > > > kern.ipc.nmbclusters: 1014856 > > kern.ipc.maxmbufmem: 8313700352 > > kern.ipc.nmbufs: 6495090 > > kern.ipc.nmbjumbop: 507428 > > kern.ipc.nmbjumbo9: 451047 > > kern.ipc.nmbjumbo16: 338284 > > > > FreeBSD mclane 10.0-STABLE FreeBSD 10.0-STABLE #5 r261710: Mon Feb 10 > > 16:55:29 CET 2014 root@mclane.rt.aei.uni-hannover.de: > /usr/obj/usr/src/sys/GENERIC > > amd64 > > > > > > Is this still too low? I have a 6core machine with 6 igb interfaces, an= d > I > > cannot even get one of these set to mtu 9000 without getting "Could not > > setup receive structures". > > > > > > cu > > Gerrit > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > This email message is intended for the use of the person to whom it has > been sent, and may contain information that is confidential or legally > protected. If you are not the intended recipient or have received this > message in error, you are not authorized to copy, distribute, or otherwis= e > use this message or its attachments. Please notify the sender immediately > by return e-mail and permanently delete this message and any attachments. > Verio Inc. makes no warranty that this email is error or virus free. Than= k > you. >