From owner-freebsd-stable Thu Apr 5 17:38: 1 2001 Delivered-To: freebsd-stable@freebsd.org Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 79C1C37B42C; Thu, 5 Apr 2001 17:37:52 -0700 (PDT) (envelope-from cdillon@wolves.k12.mo.us) Received: from mail.wolves.k12.mo.us (cdillon@mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (8.9.3/8.9.3) with ESMTP id TAA55840; Thu, 5 Apr 2001 19:37:43 -0500 (CDT) (envelope-from cdillon@wolves.k12.mo.us) Date: Thu, 5 Apr 2001 19:37:43 -0500 (CDT) From: Chris Dillon To: Archie Cobbs Cc: Archie Cobbs , , Subject: Re: mbuf leak? fxp? In-Reply-To: <200104051718.f35HIcF73652@arch20m.dellroad.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 5 Apr 2001, Archie Cobbs wrote: > Archie Cobbs writes: > > I have this machine that starts running out of mbufs every few days > > ("looutput: mbuf allocation failed") and then crashes, and was wondering > > if anyone else has seen similar behavior... > > > > For example... > > > > Yesterday... > > $ netstat -m > > 461/624/4096 mbufs in use (current/peak/max): > > 459 mbufs allocated to data > > 2 mbufs allocated to packet headers > > 434/490/1024 mbuf clusters in use (current/peak/max) > > 1136 Kbytes allocated to network (36% of mb_map in use) > > 0 requests for memory denied > > 0 requests for memory delayed > > 0 calls to protocol drain routines > > > > Today... > > $ netstat -m > > 947/1072/4096 mbufs in use (current/peak/max): > > 945 mbufs allocated to data > > 2 mbufs allocated to packet headers > > 920/946/1024 mbuf clusters in use (current/peak/max) > > 2160 Kbytes allocated to network (70% of mb_map in use) > > 0 requests for memory denied > > 0 requests for memory delayed > > 0 calls to protocol drain routines > > > > It appears that something is slowly eating up mbuf clusters. > > The machine is on a network with continuous but very low volume > > traffic, including some random multicast, NTP, etc. The machine > > itself is doing hardly anything at all. > > Well, my current guess is that this is simply an NMBCLUSTERS problem. > I increased NMBCLUSTERS to 8192 and it hasn't happened again yet. > > This machine has 5 ethernet interfaces, which must be probably more > than the default NMBCLUSTERS can handle. Just a datapoint... I'm running a 4.3-BETA box with 8 fxp interfaces all on 100Mbit networks (several heavily trafficed, others spurious) and MAXUSERS set to 128, which gives me 2560 mbuf clusters: 565/2784/10240 mbufs in use (current/peak/max): 537 mbufs allocated to data 28 mbufs allocated to packet headers 524/2038/2560 mbuf clusters in use (current/peak/max) 4772 Kbytes allocated to network (62% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines Could probably use a few more mbuf clusters, since its getting close, but read on... This box has been up for 22 days (been up for many moons before, but I wanted to test 4.3-BETA on it... yeah, its an "old" BETA already), and does LOTS of stuff in addition to routing across the 8 fxp interfaces, including ipfw with over 60 static rules and many hundreds of dynamic rules, just a little bit of NAT using natd, arpwatch and snort on about five of the interfaces, and Squid as a HTTP proxy with about 30GB of cache doing about 30000 requests/hour on average (handles about 60000 requests during the peak hour -- lunchtime). It still has plenty of power left over to run a distrubuted.net personal proxy and chew on lots of RC5 keys as well (I love FreeBSD). :-) Its doing pretty much the gamut of network related abuse you could do to a box -- routing on lots of interfaces, bpfilter (two per interface in most cases), ipfw, NAT, a fair amount of incoming and outgoing connections -- except I'm not doing anything Netgraph related (assuming you might be, being one who wrote it). Maybe its related to that? -- Chris Dillon - cdillon@wolves.k12.mo.us - cdillon@inter-linc.net FreeBSD: The fastest and most stable server OS on the planet. For IA32 and Alpha architectures. IA64, PPC, and ARM under development. http://www.freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message