Date: Mon, 26 Jan 2009 12:52:46 -0500 From: gnn@freebsd.org To: Rui Paulo <rpaulo@freebsd.org> Cc: Liran Liss <liranl@mellanox.co.il>, freebsd-net@freebsd.org, Yony Yossef <yonyossef.lists@gmail.com>, johan@nocrew.org, Amit Krig <amitk@mellanox.co.il>, Eitan Shefi <eitans@mellanox.co.il> Subject: Re: freebsd 7.0-RELEASE BUG ping: sendto: No buffer space available Message-ID: <7iocxu7y4x.wl%gnn@neville-neil.com> In-Reply-To: <EDD4152E-8BE7-40C3-A153-21A110152BE8@freebsd.org> References: <000001c97e23$0d81df20$39ed1aac@mtl.com> <EDD4152E-8BE7-40C3-A153-21A110152BE8@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
At Sat, 24 Jan 2009 16:20:06 +0000, Rui Paulo wrote: > > > On 24 Jan 2009, at 12:54, Yony Yossef wrote: > > > Hi All, > > > > I'm facing a temporary network hang on my interfaces following a flood > > ping/stress udp test. > > > > I'm running a netperf UDP test which is giving results but does not > > return > > to the shell. > > client output: > > > > UDP UNIDIRECTIONAL SEND TEST from fe80::202:c9ff:fe02:e1fe%mtnic0 > > (fe80::202:c9ff:fe02:e1fe) port 0 AF_INET6 to > > fe80::202:c9ff:fe02:e1f4%mt > > nic0 (fe80::202:c9ff:fe02:e1f4) port 0 AF_INET6 > > Socket Message Elapsed Messages > > Size Size Time Okay Errors Throughput > > bytes bytes secs # # 10^6bits/sec > > > > 32768 1472 10.02 547428 1694280 643.60 > > 32768 10.02 25089 29.50 > > > > > > (HANG) > > > > After a minute or two it returns to the shell with the following > > message: > > shutdown_control: no response received errno 55 > > > > 20 minutes later (!!) the interface is working again. > > > > netstat -m and vmstat -z outputs during the hang time: > > > > # netstat -m > > 25687/6578/32265 mbufs in use (current/cache/total) > > 17404/2438/19842/65536 mbuf clusters in use (current/cache/total/max) > > 0/1024 mbuf+clusters out of packet secondary zone in use (current/ > > cache) > > 2071/1369/3440/65536 4k (page size) jumbo clusters in use > > (current/cache/total/max) > > 0/0/0/65536 9k jumbo clusters in use (current/cache/total/max) > > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > > 49513K/11996K/61510K bytes allocated to network (current/cache/total) > > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > > 0/0/0 sfbufs in use (current/peak/max) > > 0 requests for sfbufs denied > > 0 requests for sfbufs delayed > > 0 requests for I/O initiated by sendfile > > 0 calls to protocol drain routines > > I think there are too many mbufs in use. You're probably facing an > mbuf leakage and that causes an interface hang. > If this is a large memory machine try upping the number of clusters and mbufs. On 64 bit systems with large memories 1,000,000 mbufs is not unheard of. kern.ipc.nmbclusters: 1000000 Also, with UDP you can easily overrun different buffers within the system. You might also look at: netstat -id and see if the driver is dropping packets, and if so you might up its send queue. Best, George
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7iocxu7y4x.wl%gnn>