Date: Mon, 7 Jun 2010 16:09:55 -0700 From: Pyun YongHyeon <pyunyh@gmail.com> To: Olaf Seibert <O.Seibert@cs.ru.nl> Cc: freebsd-stable@freebsd.org Subject: Re: nfe0 loses network connectivity (8.0-RELEASE-p2) Message-ID: <20100607230955.GD1369@michelle.cdnetworks.com> In-Reply-To: <20100607140611.GX883@twoquid.cs.ru.nl> References: <20100527131310.GS883@twoquid.cs.ru.nl> <20100527174211.GC1211@michelle.cdnetworks.com> <20100607140611.GX883@twoquid.cs.ru.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jun 07, 2010 at 04:06:11PM +0200, Olaf Seibert wrote: > On Thu 27 May 2010 at 10:42:11 -0700, Pyun YongHyeon wrote: > > On Thu, May 27, 2010 at 03:13:10PM +0200, Olaf Seibert wrote: > > > Here is the output of netstat -m while the problem was going on: > > > > > > 25751/1774/27525 mbufs in use (current/cache/total) > > > 24985/615/25600/25600 mbuf clusters in use (current/cache/total/max) > > ^^^^^^^^^^^^^^^^^^^^^ > > As Jeremy said, it seems you're hitting mbuf shortage situation. I > > think nfe(4) is dropping received frames in that case. See how many > > packets were dropped due to mbuf shortage from the output of > > "netstat -ndI nfe0". You can also use "sysctl dev.nfe.0.stats" to > > see MAC statistics maintained in nfe(4) if your MCP controller > > supports hardware MAC counters. > > The sysctl command gives me (among other figures): > > dev.nfe.0.stats.rx.drops: 338180 > > so indeed frames seem to be dropped. > > Jeremy Chadwick mentioned that one can tune kern.ipc.nmbclusters in > boot.conf, but apparently it is also changeable at runtime with sysctl. > > Since the problem recurred today, I increased the value from 25600 to > 32768, the maximum recommended value in the Handbook. (I can probably go > higher if needed; the box has 8 GB of RAM, although up to half of it is > eaten by ZFS) > > I do get the impression there is a mbuf leak somehow. On a much older > file server (FreeBSD 6.1, serves a bit of NFS but has no ZFS) the mbuf > cluster useage is much lower, despite a longer uptime: > Yeah, it surely indicates mbuf leakage in kernel. There was no mbuf leakage report in nfe(4) so I think the leakage is in other part of kernel. As Mikolaj said, it would be great if you give the latest stable/8 try and let us know whether the issue was fixed or not. > 256/634/890/25600 mbuf clusters in use (current/cache/total/max) > > Also, it shows signs that measures are taken in case of mbuf shortage: > > 2259806/466391/598621 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 1016 calls to protocol drain routines > > whereas the FreeBSD 8.0 machine has zero or very low numbers: > > 0/3956/1959 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0 calls to protocol drain routines > > and useage keeps growing: > > 26122/1782/27904/32768 mbuf clusters in use (current/cache/total/max) > > -Olaf. > --
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100607230955.GD1369>