From owner-freebsd-arch@FreeBSD.ORG Tue Nov 25 00:09:13 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66E8B1065740 for ; Tue, 25 Nov 2008 00:09:13 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from proxy.meer.net (proxy.meer.net [64.13.141.13]) by mx1.freebsd.org (Postfix) with ESMTP id 3367F8FC0C for ; Tue, 25 Nov 2008 00:09:13 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [64.13.141.3]) by proxy.meer.net (8.14.2/8.14.2) with ESMTP id mAONm0m3004755; Mon, 24 Nov 2008 15:48:01 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id mAONk4Wd006211; Mon, 24 Nov 2008 15:46:04 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from minion.local.neville-neil.com (209.249.190.254.available.above.net [209.249.190.254] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id mAONk3PU009335; Mon, 24 Nov 2008 15:46:04 -0800 (PST) (envelope-from gnn@neville-neil.com) Date: Mon, 24 Nov 2008 18:46:02 -0500 Message-ID: From: gnn@freebsd.org To: Jeff Roberson In-Reply-To: <20081123213232.A971@desktop> References: <20081123213232.A971@desktop> User-Agent: Wanderlust/2.15.5 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.7 Emacs/22.3 (i386-apple-darwin9.5.0) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Canit-CHI2: 0.50 X-Bayes-Prob: 0.5 (Score 0, tokens from: ) X-Spam-Score: 0.10 () [Tag at 5.00] COMBINED_FROM X-CanItPRO-Stream: default X-Canit-Stats-ID: 2387736 - 476b1cc662f4 X-Scanned-By: CanIt (www . roaringpenguin . com) on 64.13.141.13 Cc: arch@freebsd.org Subject: Re: Limiting mbuf memory. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 00:09:13 -0000 At Sun, 23 Nov 2008 21:46:08 -1000 (HST), Jeff Roberson wrote: > > I'm developing a patch for an alternate memory layout for mbuf clusters > that relies on contigmalloc. Since this can fail, we'll still have to > retain the capability of allocating traditional clusters. I'll report > details on that later. I'm writing this email to address the issue of > resource accounting in mbufs. > > Presently we use a set of limits on individual zones or sizes of mbufs. > Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos. > Each is administered sperately. I think this is getting a bit unwieldy. > In the future, we may have even more sizes. This also introduces problems > because I will have two cluster zones do they each get their own limit? > > I would like to consolidate this into a single limit on the number of > pages in total allocated to networking. With perhaps some fractional > reservation for standard mbufs and clusters to make sure they aren't > overwhelmed by the larger buffers. > > This would be implemented by overriding the uma zone page allocator for > each of the mbuf zones with one that counts pages for all. Should we > reach the limit we'll block depending on the wait settings of the > requestor. The limit and sleep will probably be protected by a global > lock which won't be an issue because trips to the back end allocator are > infrequent and protected by their own global lock anyhow. > > How do people feel about this? To be clear this would eliminate: > > nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings > and sysctls. They would be replaced by something like 'maxmbufbytes'. > Presently we place no limit on small mbufs. I could go either way on > this. It could be added to the limit or not. > I think this is a good idea with the caveat that I prefer the idea in paragraph 3 about reserving a bit of head room so we don't deadlock. A very common bug in the past was to run out of mbufs when using a lot of small UDP packets. Best, George