From owner-freebsd-stable Mon Jul 1 8:17:13 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C40CC37B400 for ; Mon, 1 Jul 2002 08:17:08 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 205D143E26 for ; Mon, 1 Jul 2002 08:17:08 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id LAA22828; Mon, 1 Jul 2002 11:17:07 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g61FGbH21968; Mon, 1 Jul 2002 11:16:37 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15648.29269.138339.602900@grasshopper.cs.duke.edu> Date: Mon, 1 Jul 2002 11:16:37 -0400 (EDT) To: Don Bowman Cc: "'freebsd-stable@freebsd.org'" Subject: RE: number mbufs / cluster In-Reply-To: References: X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Don Bowman writes: > > Doug White wrote: > > > On Sat, 29 Jun 2002, Don Bowman wrote: > > > > > > > > I have an application where there is lots of connections, > > > but packet sizes aren't that large. I'm running into the > > > problem where I need an enormous amount of clusters, > > > but the number of mbufs needed isn't as big: > > > > > > $ netstat -m > > > 141164/141200/512000 mbufs in use (current/peak/max): > > > 141163 mbufs allocated to data > > > 1 mbufs allocated to fragment reassembly queue headers > > > 127997/128000/128000 mbuf clusters in use (current/peak/max) > > > 291300 Kbytes allocated to network (10% of mb_map in use) > > > 121192 requests for memory denied > > > > holy cow. How much memory does this thing have and what did you do to > > abuse it soo much? :-) I didn't think you could jack those values that > > high and not get panics. > > > > I tried changing MCLSHIFT to 10. Although this would seem like > the right thing to do, some trouble ensues... Some things like nfs > don't seem to work right (just UDP traffic as far as I could see). > (10 would yield a 1K cluster). Are there > any assumptions somewhere that a cluster is >= MTU size? Possibly. What nic driver are you using? > Also, how does one change the number of mbufs down? The sysctl > value is read-only. # Mbufs seems to be a constant 4x # clusters. You can set the boot-time tunables kern.ipc.nmbufs and kern.ipc.nmbclusters in /boot/loader.conf. There's a sanity check that enfornces that nmbufs be at least 2x nmbclusters in sys/kern/uipc_mbuf.c. You might want to start with: kern.ipc.nmbclusters=256000 kern.ipc.nmbufs=512000 This will use a max of 625MB of kernel memory for mbufs... You may also need to increase the limit of the mbuf malloc type Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message