From owner-freebsd-net@FreeBSD.ORG  Mon Apr  4 07:22:01 2011
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id DB1151065675
	for <net@freebsd.org>; Mon,  4 Apr 2011 07:22:01 +0000 (UTC)
	(envelope-from egrosbein@rdtc.ru)
Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [62.231.161.221])
	by mx1.freebsd.org (Postfix) with ESMTP id 0FA228FC08
	for <net@freebsd.org>; Mon,  4 Apr 2011 07:22:00 +0000 (UTC)
Received: from eg.sd.rdtc.ru (localhost [127.0.0.1])
	by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p347LtnX058788;
	Mon, 4 Apr 2011 14:21:55 +0700 (NOVST)
	(envelope-from egrosbein@rdtc.ru)
Message-ID: <4D99718E.2050100@rdtc.ru>
Date: Mon, 04 Apr 2011 14:21:50 +0700
From: Eugene Grosbein <egrosbein@rdtc.ru>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU;
	rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7
MIME-Version: 1.0
To: Vlad Galu <dudu@dudu.ro>
References: <4D9969A8.1060701@rdtc.ru>
	<BANLkTikQc_N6RKE-Ov_nNqivDemefuCpdQ@mail.gmail.com>
In-Reply-To: <BANLkTikQc_N6RKE-Ov_nNqivDemefuCpdQ@mail.gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Cc: "net@freebsd.org" <net@freebsd.org>
Subject: Re: mbuf clusters exhaustion & keglimit
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Apr 2011 07:22:01 -0000

On 04.04.2011 14:15, Vlad Galu wrote:

>     "vmstat -z -M vmcore" says that system was out of mbuf clusters:
> 
>     ITEM                     SIZE     LIMIT      USED      FREE  REQUESTS  FAILURES
>     mbuf_cluster:            2048,   100000,   100000,        0, 18897242,   317691

> 
> I've been having the same kind of issues with another 8.2/amd64 box with bge(4) NICs. Unfortunately I don't have console access to that machine and haven't yet graphed anything, but it just so happened for the symptom to occur while I was logged in a couple of days ago and the machine wasn't busy handling anything else than my SSH session. The ISP has checked their switch graphs and told me there was no spike that would correlate to this event either. My machine is UP and I tried both direct and queued (with various queue lenghts) ISR dispatch modes. I never had more than 250k mbuf clusters allocated but for this machine's workload even that is quite generous...

Forgot to note, my problem occured late night with nearly zero load.

Eugene Grosbein