From owner-freebsd-net@FreeBSD.ORG Fri Nov 21 21:05:48 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D8338F2 for ; Fri, 21 Nov 2014 21:05:48 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0056.outbound.protection.outlook.com [65.55.169.56]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47343BDE for ; Fri, 21 Nov 2014 21:05:47 +0000 (UTC) Received: from BLUPR0801MB674.namprd08.prod.outlook.com (10.141.255.11) by BLUPR0801MB673.namprd08.prod.outlook.com (10.141.254.27) with Microsoft SMTP Server (TLS) id 15.1.26.15; Fri, 21 Nov 2014 20:34:03 +0000 Received: from BLUPR0801MB674.namprd08.prod.outlook.com ([10.141.255.11]) by BLUPR0801MB674.namprd08.prod.outlook.com ([10.141.255.11]) with mapi id 15.01.0026.003; Fri, 21 Nov 2014 20:34:03 +0000 From: David DeSimone To: Jack Vogel Subject: RE: igb Could not setup receive structures (again) Thread-Topic: igb Could not setup receive structures (again) Thread-Index: AQHQBXofby0q0CluWU+mfdV7RVj30JxrQ76AgABFyGA= Date: Fri, 21 Nov 2014 20:34:03 +0000 Message-ID: <4a92a47b86474b92b5b4fd7601df9cc2@BLUPR0801MB674.namprd08.prod.outlook.com> References: <20141121114852.8b7d9a6c071c3e0799d5f30c@aei.mpg.de> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [173.74.209.33] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;SRVR:BLUPR0801MB673; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:;SRVR:BLUPR0801MB673; x-forefront-prvs: 0402872DA1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(53754006)(189002)(377454003)(24454002)(51704005)(13464003)(199003)(51884002)(97736003)(108616004)(40100003)(95666004)(105586002)(106116001)(106356001)(31966008)(122556002)(99396003)(15202345003)(77156002)(62966003)(99286002)(4396001)(21056001)(1411001)(107046002)(92566001)(66066001)(33646002)(110136001)(120916001)(46102003)(101416001)(76576001)(19580405001)(19580395003)(87936001)(2656002)(50986999)(86362001)(20776003)(64706001)(74316001)(76176999)(15975445006)(54356999)(24736002); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0801MB673; H:BLUPR0801MB674.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: verio.net Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Nov 2014 21:05:48 -0000 Would it be possible for the driver to report how many clusters it calculat= ed that it needs, whenever it runs into this memory shortage during attach?= That way an administrator might have some idea how much to increase their= tunables in order to meet the driver's requirements. As it is, it's merely a guessing game or requires digging into driver sourc= e to come up with the right numbers. -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Jack Vogel Sent: Friday, November 21, 2014 10:23 AM To: Gerrit K=FChn Cc: FreeBSD Net Subject: Re: igb Could not setup receive structures (again) The message is pretty straightforward, its only cause is the inability of the driver to get the necessary clusters to populate the RX rings. After you get the error do a `netstat -m` and see what the state of the 9K jumbo pool is, for that is the size you would be using. Depending on the specific device type you may have up to 8 rings per interface, and with a ring size of 1K... It does seem like you should have enough, but maybe something else in your system is using the pool? As I said, look at netstat, it should giv= e you the truth, and then adjust the allocated size to fit your needs. Jack On Fri, Nov 21, 2014 at 2:48 AM, Gerrit K=FChn wrote: > Hi all, > > I get the error message above when trying to go for jumbo frames (mtu > 9000) on a system here. I found a lot of comments on this, but they all > state that this is due to a too low setting for mbufs in FreeBSD8/9 > settings. > However, I have 10-stable here, and the settings look rather high to me: > > kern.ipc.nmbclusters: 1014856 > kern.ipc.maxmbufmem: 8313700352 > kern.ipc.nmbufs: 6495090 > kern.ipc.nmbjumbop: 507428 > kern.ipc.nmbjumbo9: 451047 > kern.ipc.nmbjumbo16: 338284 > > FreeBSD mclane 10.0-STABLE FreeBSD 10.0-STABLE #5 r261710: Mon Feb 10 > 16:55:29 CET 2014 root@mclane.rt.aei.uni-hannover.de:/usr/obj/usr/src/sys= /GENERIC > amd64 > > > Is this still too low? I have a 6core machine with 6 igb interfaces, and = I > cannot even get one of these set to mtu 9000 without getting "Could not > setup receive structures". > > > cu > Gerrit > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" This email message is intended for the use of the person to whom it has bee= n sent, and may contain information that is confidential or legally protect= ed. If you are not the intended recipient or have received this message in = error, you are not authorized to copy, distribute, or otherwise use this me= ssage or its attachments. Please notify the sender immediately by return e-= mail and permanently delete this message and any attachments. Verio Inc. ma= kes no warranty that this email is error or virus free. Thank you.