From owner-freebsd-questions Wed Mar 5 17:56:29 2003 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD59F37B405 for ; Wed, 5 Mar 2003 17:56:27 -0800 (PST) Received: from smtp.acd.net (smtp.acd.net [207.179.102.146]) by mx1.FreeBSD.org (Postfix) with ESMTP id A2CEC43FA3 for ; Wed, 5 Mar 2003 17:56:26 -0800 (PST) (envelope-from taxman@ACD.NET) Received: from lannas5modem15.acd.net ([207.179.65.15]) by smtp.acd.net with Microsoft SMTPSVC(5.0.2195.5329); Wed, 5 Mar 2003 20:56:23 -0500 From: taxman To: Kirk Strauser , freebsd-questions@freebsd.org Subject: Re: How many mbufs do I need? Date: Wed, 5 Mar 2003 20:59:14 -0500 User-Agent: KMail/1.5 References: <87bs0ppk4d.fsf@strauser.com> In-Reply-To: <87bs0ppk4d.fsf@strauser.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200303052059.14368.taxman@acd.net> X-OriginalArrivalTime: 06 Mar 2003 01:56:24.0377 (UTC) FILETIME=[979E9A90:01C2E383] Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 05 March 2003 04:40 pm, Kirk Strauser wrote: > I killed my FreeBSD 4.8-PRE (2003-02-18) server today by exhausting the > available mbufs. I'd seen warnings like "All mbuf clusters exhausted, > please see tuning(7)." in /var/log/messages, so I added > `kern.ipc.nmbclusters="16384"' to /boot/loader.conf.local a while back. > > This server has 768MB of memory, and I run at least two copies of > PostgreSQL (it has multiple jails) at any given time and use it to server > NFS to a couple of Linux clients. Today I wanted to test Bochs on one of > the Linux machines, ran the bochsconf program, watched it run "createdisk" > to make a 512MB disk image in my home directory, and saw the server panic > within a few seconds. When it came back up, I bumped nmbclusters to 32768, > rebooted, and tried createdisk it. This time I watched `netstat -m' as the > dreaded "All mbuf clusters exhausted..." message began to scroll up the > screen. This time, the server stayed alive, but it clearly maxed out all > available buffers. After createdisk was finished, the mbuf usage went back > to normal: > > $ netstat -m > 389/36736/131072 mbufs in use (current/peak/max): > 389 mbufs allocated to data > 388/32768/32768 mbuf clusters in use (current/peak/max) > 74720 Kbytes allocated to network (33% of mb_map in use) > 345 requests for memory denied > 1 requests for memory delayed > 0 calls to protocol drain routines > > My questions are these: > > 1) Is there a guideline for how many nmbclusters to allocate? Short of > running out of physical memory, is there such a thing as too many? > > 2) I've already allocated 64MB to network buffering. This seems like a > huge amount to me. Is it? > > 3) Should running out of mbufs be expected to crash the server, or should I > try to reproduce it and file a PR? Reproducible errors are the most helpful thing a developer can have to improve FreeBSD's stability. So try to reproduce it, but make sure you're doing it on the most recent -stable code, as your problem may already be fixed. If you can update to -stable and still get that error then file a pr and ask on the -stable mailing list as that is the appropriate place. No idea on the rest, sorry, Tim > > 4) What could createdisk have been doing to spike usage that heavily? > > 5) Could Linux's NFS client implementation have aggravated the situation? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message