From owner-freebsd-stable@FreeBSD.ORG Tue Nov 3 19:23:44 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31C34106568B for ; Tue, 3 Nov 2009 19:23:44 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from tomjudge.vm.bytemark.co.uk (tomjudge.vm.bytemark.co.uk [80.68.91.100]) by mx1.freebsd.org (Postfix) with ESMTP id E52458FC12 for ; Tue, 3 Nov 2009 19:23:43 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id BB62948A99; Tue, 3 Nov 2009 19:23:42 +0000 (GMT) X-Virus-Scanned: Debian amavisd-new at tomjudge.vm.bytemark.co.uk Received: from tomjudge.vm.bytemark.co.uk ([127.0.0.1]) by localhost (tomjudge.vm.bytemark.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id byA6loXF1Ofq; Tue, 3 Nov 2009 19:23:38 +0000 (GMT) Received: from rita.nodomain (unknown [192.168.205.6]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id F388248A7C; Tue, 3 Nov 2009 19:23:37 +0000 (GMT) Message-ID: <4AF082FF.2070601@tomjudge.com> Date: Tue, 03 Nov 2009 19:22:39 +0000 From: Tom Judge User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: Tim Chen References: <1f51039c0911030029k7e25e3bcxea941ff542311d59@mail.gmail.com> In-Reply-To: <1f51039c0911030029k7e25e3bcxea941ff542311d59@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: Increasing number of "requests for jumbo clusters denied" in netstat -m X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2009 19:23:44 -0000 Tim Chen wrote: > My machine is an IBM HS21 blade server and the NIC is bce. > > bce1: mem > 0xd8000000-0xd9ffffff irq 19 at device 0.0 on pci6 > miibus1: on bce1 > bce1: Ethernet address: 00:1a:64:34:f2:fa > bce1: [ITHREAD] > bce1: ASIC (0x57081021); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C > (0x03040405) ; Flags( MSI ) > > That machine is serving as a mail and web server. Recently I found that the > number of "requests for jumbo clusters denied" in netstat -m increases all > the time. > > 1031/3469/4500 mbufs in use (current/cache/total) > 510/3326/3836/32768 mbuf clusters in use (current/cache/total/max) > 510/2278 mbuf+clusters out of packet secondary zone in use (current/cache) > 1/1453/1454/8704 4k (page size) jumbo clusters in use > (current/cache/total/max) > 510/1086/1596/4352 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/2176 16k jumbo clusters in use (current/cache/total/max) > 5871K/23105K/28977K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/4337166/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > ifconfig shows: > bce1: flags=8843 metric 0 mtu 9000 > > options=1bb TSO4> > ether 00:1a:64:34:f2:fa > inet 192.168.152.152 netmask 0xffffff00 broadcast 192.168.152.255 > media: Ethernet autoselect (1000baseSX ) > status: active > > And in rc.conf: > ifconfig_bce1="inet 192.168.152.152 netmask 255.255.255.0 mtu 9000" > > Is the problem of increasing "requests for jumbo clusters denied" resulted > from my "mtu 9000" jumbo frame setting? > Will it be the problem of "bce" nic's driver or hardware? > Most of all, how will my system be efftected by that ""requests for jumbo > clusters denied" problem? Will it degrade > the performance or it is harmless? > > Second thread on this bug today :) There is a bug in bce(4) that causes memory fragmentation, which results in denied mbuf requests as you are seeing. You can correct this issue by applying the patch created by this command: svn diff -r 198319:198320 http://svn.freebsd.org/base/head Once you have applied the patch you need to add: options BCE_JUMBO_HDRSPLIT To you kernel config and recompile and install the new kernel. Hope this works for you. Tom