From owner-freebsd-current@FreeBSD.ORG Fri Nov 6 08:40:27 2009 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 944AF1065702 for ; Fri, 6 Nov 2009 08:40:27 +0000 (UTC) (envelope-from ianf@clue.co.za) Received: from inbound01.jnb1.gp-online.net (inbound01.jnb1.gp-online.net [41.161.16.135]) by mx1.freebsd.org (Postfix) with ESMTP id C4E078FC1B for ; Fri, 6 Nov 2009 08:40:26 +0000 (UTC) Received: from [41.154.0.9] (helo=clue.co.za) by inbound01.jnb1.gp-online.net with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1N6KMp-0000VD-8B; Fri, 06 Nov 2009 10:40:23 +0200 Received: from localhost ([127.0.0.1] helo=clue.co.za) by clue.co.za with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1N6KMm-000AGE-QC; Fri, 06 Nov 2009 10:40:20 +0200 To: Weldon S Godfrey 3 From: Ian FREISLICH In-Reply-To: References: <1257185816.44755.29.camel@buffy.york.ac.uk> <1257261214.98619.92.camel@buffy.york.ac.uk> X-Attribution: BOFH Date: Fri, 06 Nov 2009 10:40:20 +0200 Message-Id: Cc: freebsd-current@FreeBSD.org, Gavin Atkinson Subject: Re: FreeBSD 8.0 - network stack crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2009 08:40:27 -0000 Weldon S Godfrey 3 wrote: > >> OK, at least we've figured out what is going wrong then. As a > >> workaround to get the machine to stay up longer, you should be able to > >> set kern.ipc.nmbclusters=256000 in /boot/loader.conf -but hopefully we > >> can resolve this soon. > >> > > I upped it to 256K. What I am trying to wrap my head around is how it was > working somewhat for so long at 24K, but it got to near 65K before I > rebooted it with the higher setting. Or did I reboot too early? Is > there any cleanup that isn't triggered intil it reaches max nmbclusters? > I am trying to see if anything on our network has changed to cause this to > become cronic. We have a ngaios server which handles up to 5000 concurrent nsca daemons and connections which manifested a similar problem on a Dell R905 (4x4core AMD, 16GB RAM, bce). Setting the following in /boot/loader.conf sorted out the problem for us: kern.ipc.nmbclusters="131072" kern.maxusers="1024" mbuf usage is pretty static at: $ netstat -m 40165/16220/56385 mbufs in use (current/cache/total) 40154/10500/50654/131072 mbuf clusters in use (current/cache/total/max) 40154/3359 mbuf+clusters out of packet secondary zone in use (current/cache) 0/1493/1493/65536 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/32768 9k jumbo clusters in use (current/cache/total/max) 0/0/0/16384 16k jumbo clusters in use (current/cache/total/max) 90349K/31027K/121376K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 246 requests for I/O initiated by sendfile 0 calls to protocol drain routines Ian -- Ian Freislich