Date: Fri, 24 Feb 2006 20:02:15 +0300 From: "Andrew Pantyukhin" <infofarmer@gmail.com> To: "Chuck Swiger" <cswiger@mac.com> Cc: FreeBSD Questions <questions@freebsd.org> Subject: Re: Trouble with resources under network load Message-ID: <cb5206420602240902n40d7c966wbd0f45bf345642a2@mail.gmail.com> In-Reply-To: <43E62B9D.5040302@mac.com> References: <cb5206420602041148x76816902j23abb0dc21809029@mail.gmail.com> <43E62B9D.5040302@mac.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2/5/06, Chuck Swiger <cswiger@mac.com> wrote:
> Andrew Pantyukhin wrote:
> > I'm constantly stumbling upon some out-of-resources
> > problems. Just to name a couple:
> >
> > named[400]: client 10.32.23.92#1714:
> > error sending response: not enough free resources
> >
> > snmpd[806]: sysctl get: Cannot allocate memory
> >
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D
> >
> > I have these in loader.conf and sysctl.conf:
> >
> > kern.maxfiles=3D65536
> > kern.maxfilesperproc=3D65536
> > net.graph.maxdgram=3D65536
> > net.graph.recvspace=3D65536
> > kern.maxusers=3D512
> > kern.ipc.maxpipekva=3D268435456
> > net.graph.maxalloc=3D65536
> [ ... ]
> > What's wrong?
>
> For one thing, if you've got a machine with 256MB of RAM, you cannot poss=
ibly be
> able to dedicate 256MB just to kern.ipc.maxpipekva. Likewise, a machine =
with
> 256MB of users would auto-tune kern.maxusers to ~100 or so, and kern.maxf=
iles
> ought to be under 10000, if not half that.
>
> You should revert to the defaults and make gradual tuning changes, if nee=
ded,
> from there.
A couple of days ago I've added 512MB RAM, bringing
the total up to 768MB. I followed your advice and eased
the sysctl settings:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
kern.maxfiles=3D7500
kern.maxfilesperproc=3D7000
net.graph.maxdgram=3D65536
net.graph.recvspace=3D65536
net.inet.tcp.blackhole=3D2
net.inet.udp.blackhole=3D1
net.link.ether.inet.maxtries=3D2
net.link.ether.inet.max_age=3D7200
net.link.ether.inet.prune_intvl=3D1200
net.inet.tcp.minmssoverload=3D50
net.inet.tcp.msl=3D7500
net.inet.icmp.icmplim=3D50
kern.ipc.somaxconn=3D16384
kern.ipc.maxpipekva=3D50331648
net.graph.maxalloc=3D65536
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
gw# netstat -m
872/4123/4995 mbufs in use (current/cache/total)
870/3872/4742/24768 mbuf clusters in use (current/cache/total/max)
0/5/6448 sfbufs in use (current/peak/max)
1982K/8774K/10757K bytes allocated to network (current/cache/total)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
gw# netstat -s | grep drop
2776 connections closed (including 435 drops)
11 embryonic connections dropped
198 connections dropped by rexmit timeout
0 connections dropped by persist timeout
9 connections dropped by keepalive
1 dropped
53405 dropped due to no socket
460821 broadcast/multicast datagrams dropped due to no socket
0 dropped due to full socket buffers
0 fragments dropped (dup or out of space)
685 fragments dropped after timeout
193 output packets dropped due to no bufs, etc.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Feb 24 19:47:21 gw named[482]: client 10.32.7.32#1027: error sending
response: not enough free resources
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Feb 24 19:46:57 gw snmpd[714]: sysctl get: Cannot allocate memory
Feb 24 19:56:00 gw last message repeated 50 times
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
last pid: 86717; load averages: 0.00, 0.00, 0.00 up
3+02:57:27 19:59:39
29 processes: 1 running, 28 sleeping
CPU states: 0.4% user, 0.0% nice, 0.0% system, 1.5% interrupt, 98.1% id=
le
Mem: 39M Active, 311M Inact, 132M Wired, 20K Cache, 84M Buf, 246M Free
Swap: 453M Total, 453M Free
PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
714 root 1 96 0 3036K 2744K select 14:22 2.78% bsnmpd
17259 root 1 96 0 7268K 5664K select 7:54 0.00% mpd
287 root 1 96 0 3044K 2584K select 4:31 0.00% natd
482 bind 1 96 0 20224K 19280K select 2:36 0.00% named
55185 root 1 96 0 3772K 2456K select 0:36 0.00% nmbd
473 root 1 96 0 1352K 880K select 0:24 0.00% syslogd
574 root 1 96 0 3052K 1764K select 0:08 0.00% ntpd
636 dhcpd 1 96 0 3028K 2656K select 0:06 0.00% dhcpd
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Is there a change to trace the problem to its source?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?cb5206420602240902n40d7c966wbd0f45bf345642a2>