Date: Mon, 7 Jun 2010 16:06:11 +0200 From: Olaf Seibert <O.Seibert@cs.ru.nl> To: freebsd-stable@freebsd.org Cc: Pyun YongHyeon <pyunyh@gmail.com>, freebsd-stable@freebsd.org, Olaf Seibert <O.Seibert@cs.ru.nl> Subject: Re: nfe0 loses network connectivity (8.0-RELEASE-p2) Message-ID: <20100607140611.GX883@twoquid.cs.ru.nl> In-Reply-To: <20100527174211.GC1211@michelle.cdnetworks.com> References: <20100527131310.GS883@twoquid.cs.ru.nl> <20100527174211.GC1211@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu 27 May 2010 at 10:42:11 -0700, Pyun YongHyeon wrote:
> On Thu, May 27, 2010 at 03:13:10PM +0200, Olaf Seibert wrote:
> > Here is the output of netstat -m while the problem was going on:
> >
> > 25751/1774/27525 mbufs in use (current/cache/total)
> > 24985/615/25600/25600 mbuf clusters in use (current/cache/total/max)
> ^^^^^^^^^^^^^^^^^^^^^
> As Jeremy said, it seems you're hitting mbuf shortage situation. I
> think nfe(4) is dropping received frames in that case. See how many
> packets were dropped due to mbuf shortage from the output of
> "netstat -ndI nfe0". You can also use "sysctl dev.nfe.0.stats" to
> see MAC statistics maintained in nfe(4) if your MCP controller
> supports hardware MAC counters.
The sysctl command gives me (among other figures):
dev.nfe.0.stats.rx.drops: 338180
so indeed frames seem to be dropped.
Jeremy Chadwick mentioned that one can tune kern.ipc.nmbclusters in
boot.conf, but apparently it is also changeable at runtime with sysctl.
Since the problem recurred today, I increased the value from 25600 to
32768, the maximum recommended value in the Handbook. (I can probably go
higher if needed; the box has 8 GB of RAM, although up to half of it is
eaten by ZFS)
I do get the impression there is a mbuf leak somehow. On a much older
file server (FreeBSD 6.1, serves a bit of NFS but has no ZFS) the mbuf
cluster useage is much lower, despite a longer uptime:
256/634/890/25600 mbuf clusters in use (current/cache/total/max)
Also, it shows signs that measures are taken in case of mbuf shortage:
2259806/466391/598621 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
1016 calls to protocol drain routines
whereas the FreeBSD 8.0 machine has zero or very low numbers:
0/3956/1959 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0 calls to protocol drain routines
and useage keeps growing:
26122/1782/27904/32768 mbuf clusters in use (current/cache/total/max)
-Olaf.
--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100607140611.GX883>