From owner-freebsd-bugs Tue May 23 07:20:27 1995 Return-Path: bugs-owner Received: (from majordom@localhost) by freefall.cdrom.com (8.6.10/8.6.6) id HAA04012 for bugs-outgoing; Tue, 23 May 1995 07:20:27 -0700 Received: from Root.COM (implode.Root.COM [198.145.90.1]) by freefall.cdrom.com (8.6.10/8.6.6) with ESMTP id HAA04004 for ; Tue, 23 May 1995 07:20:21 -0700 Received: from corbin.Root.COM (corbin.Root.COM [198.145.90.18]) by Root.COM (8.6.8/8.6.5) with ESMTP id HAA25513; Tue, 23 May 1995 07:23:24 -0700 Received: from localhost (localhost [127.0.0.1]) by corbin.Root.COM (8.6.11/8.6.5) with SMTP id HAA00246; Tue, 23 May 1995 07:20:27 -0700 Message-Id: <199505231420.HAA00246@corbin.Root.COM> To: Heikki Suonsivu cc: freebsd-bugs@freefall.cdrom.com Subject: Re: kern/435: mb_map full, network starts jumping off and on, finally stays down In-reply-to: Your message of "Tue, 23 May 95 17:04:18 +0300." <199505231404.RAA03308@katiska.clinet.fi> From: David Greenman Reply-To: davidg@Root.COM Date: Tue, 23 May 1995 07:20:26 -0700 Sender: bugs-owner@FreeBSD.org Precedence: bulk > >David Greenman writes: > > > P90 as news server plus various other jobs. > > ... > > >May 23 00:00:32 katiska /kernel: mb_map full > > > > > >And network interface starts dropping packets. Occasionally a packet > > >or two gets through, as sometimes nfs says is alive again, and even a > > >login gets though. A couple of hours later the connections die > > >completely. > > > > How many simultaneous TCP connections does the machine typically have, and > >Haven't counted, now it is about 40 but we are not at the peak hours yet. >I would guess it stays less 200, probably less than 100 connections. Even >less active connections. > > > how many mbuf clusters do you have configured in the kernel? > >Haven't changed the number of MBUFs. maxusers is 256. That's the problem. >It still is a problem it the system locks up all IO when mbufs run out, it >probably should panic if a critical resource isn't available, though I can >see that it isn't easy to know when it did run out all of the available >space. But it is the difference between short break and driving 10 miles >down there to reboot the damn thing. Linux is particularly bad in these >things, I have hardly ever seen it panic, it always locks up either IO or >itself, although less often than FreeBSD panics :-(. The system should be more dynamic about things like this. It definately shouldn't gratuitously panic when transient resource problems occur, however. In your case, the limit is so low that TCP connections waiting to run-down will consume all of them and result in a deadlock. Please add: options "NMBCLUSTERS=2048" to your kernel config file and monitor the peak number of in-use mbuf clusters with netstat -m, as in the following: [corbin:davidg] netstat -m 24 mbufs in use: 16 mbufs allocated to data 5 mbufs allocated to packet headers 2 mbufs allocated to protocol control blocks 1 mbufs allocated to socket names and addresses 17/26 mbuf clusters in use 55 Kbytes allocated to network (67% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines The ratio of "x/y mbuf clusters in use" in the above is the current/peak mbuf cluster usage. If the peak hits your kernel configured maximum, then the maximum isn't high enough and you should increase it. Someday this will be based on maxusers...and will be better documented. -DG