From owner-freebsd-net@FreeBSD.ORG Sat May 20 15:20:46 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 06EAE16A41F for ; Sat, 20 May 2006 15:20:46 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id A970243D48 for ; Sat, 20 May 2006 15:20:44 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 99F7746CF0; Sat, 20 May 2006 11:20:43 -0400 (EDT) Date: Sat, 20 May 2006 16:20:43 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Peter Blok In-Reply-To: <001501c67c03$9c3457e0$8a01a8c0@ntpc> Message-ID: <20060520162001.Y246@fledge.watson.org> References: <001501c67c03$9c3457e0$8a01a8c0@ntpc> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org, "'Jin Guojun \[VFFS\]'" , 'OxY' Subject: RE: mbuf denied problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 May 2006 15:20:46 -0000 On Sat, 20 May 2006, Peter Blok wrote: > I have posted a reply to similar questions a couple of times. Although I'm > subscribed to freebsd-net I never saw the answer appear in the list. > > The ... denied counts is coming from calling uma_reclaim, which will in the > end call uma_zfree_internal which is increasing the fail counter. > > I am not sure, but I believe the mbuf and cluster zones are static. So if I > set the UMA_ZONE_NOFREE flag in kern_mbuf.c, no attempt will be made to do a > free at all. > > Here's my patch. It works, but I'm not sure it is ok. Allowing mbuf memory to be reclaimed for the purposes of other subsystems was an intentional design choice. If it's not working right, it should be fixed, but not disabled. The below change means that after a network load spike, the memory can't be returned for use by other subsystems. Robert N M Watson > > --- sys/kern/kern_mbuf.c.orig Sun Apr 9 13:32:51 2006 > +++ sys/kern/kern_mbuf.c Sun Apr 9 13:33:19 2006 > @@ -168,7 +168,7 @@ > #else > NULL, NULL, > #endif > - MSIZE - 1, UMA_ZONE_MAXBUCKET); > + MSIZE - 1, UMA_ZONE_MAXBUCKET|UMA_ZONE_NOFREE); > > zone_clust = uma_zcreate(MBUF_CLUSTER_MEM_NAME, MCLBYTES, > mb_ctor_clust, mb_dtor_clust, > @@ -177,7 +177,7 @@ > #else > NULL, NULL, > #endif > - UMA_ALIGN_PTR, UMA_ZONE_REFCNT); > + UMA_ALIGN_PTR, UMA_ZONE_REFCNT|UMA_ZONE_NOFREE); > > if (nmbclusters > 0) > uma_zone_set_max(zone_clust, nmbclusters); > > -----Original Message----- > From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] > On Behalf Of OxY > Sent: Friday, May 19, 2006 12:06 AM > To: freebsd-net@freebsd.org > Cc: Jin Guojun [VFFS] > Subject: mbuf denied problem > > hi! > > i have a problem with mbuf... > when all my free memory is gone ( i have 2gb ram) and > memory allocation looks like this: > > Mem: 30M Active, 1607M Inact, 245M Wired, 84M Cache, 214M Buf, 3028K Free > Swap: 695M Total, 695M Free > > mbuf starts to deny... > netstat -m show 0 deny till has memory, after that i see this: > > Field root# netstat -m > 629/466/1095 mbufs in use (current/cache/total) > 512/176/688/65536 mbuf clusters in use (current/cache/total/max) > 512/133 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) > 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) > 1181K/468K/1649K bytes allocated to network (current/cache/total) > 7410219/101093/101499 requests for mbufs denied > (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 25288 requests for I/O initiated by sendfile > 65 calls to protocol drain routines > > this is my sysctl.conf: > net.inet.tcp.delayed_ack=0 > net.inet.ip.intr_queue_maxlen=400 > net.inet.tcp.inflight.enable=0 > net.route.netisr_maxqlen=512 > kern.ipc.somaxconn=1024 > > should i attach any other conf/log,etc? > thanks! > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >