Date: Fri, 11 Apr 1997 15:15:42 +0400 (MSD) From: bag@sinbin.demos.su (Alex G. Bulushev) To: hackers@freebsd.org Cc: mishania@demos.su Subject: mbuf clusters problem in 2.2.1R Message-ID: <199704111115.PAA11170@sinbin.demos.su>
next in thread | raw e-mail | index | archive | help
we use p6-200 with two 3940UW and fxp (100baseT fullduplex) 256M RAM
under fbsd 2.2.1R
system reboots once per 2-6 days with out of mbuf clusters diagnostic
on console ... we use maxusers=256 without any effect, then we
increase NMBCLUSTERS to 8192 and then to 12288 with the same effect ...
while working, number of allocated mbuf clusters permanently increase
(some mbufs not freeing?) and then system crashed ...
this effect exists only on system with big network traffic ...
now we run:
squid - up to 200 connection simultaneously
ircd - up to 50 connection
cvsupd ...
our config parameters for now:
machine "i386"
cpu "I686_CPU"
ident SKRALDESPAND
maxusers 256
options "MAXMEM=(1024*256)"
options "CHILD_MAX=256"
options "OPEN_MAX=256"
options "NMBCLUSTERS=12288"
...
for exemple:
we reboot system 9 Apr 8:10 MSD and system
crash 11 Apr 13:11 MSD (moscow time)
now output of "netstat -m" 11 Apr 10:08 (3:03 h before crash)
7407 mbufs in use:
7005 mbufs allocated to data
385 mbufs allocated to packet headers
14 mbufs allocated to protocol control blocks
3 mbufs allocated to socket names and addresses
4662/5808 mbuf clusters in use
12541 Kbytes allocated to network (81% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
now output of "netstat -m" 11 Apr 13:07 (4 min before crash)
14911 mbufs in use:
13719 mbufs allocated to data
1176 mbufs allocated to packet headers
14 mbufs allocated to protocol control blocks
2 mbufs allocated to socket names and addresses
11062/11070 mbuf clusters in use
24003 Kbytes allocated to network (-74% in use)
^^^^^^^ ??????????????????
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
now output of "netstat -m" 11 Apr 13:10 (1 min before crash)
18053 mbufs in use:
16818 mbufs allocated to data
1219 mbufs allocated to packet headers
14 mbufs allocated to protocol control blocks
2 mbufs allocated to socket names and addresses
14042/14060 mbuf clusters in use
^^^^^^^^^^^????????????????????????????????????????????
30376 Kbytes allocated to network (-38% in use)
^^^^?????????????????????
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
and finaly output of "vmstat -m" 11 Apr 13:10 (1 min before crash)
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 64 1216 19595 1280 0
32 803 221 1530461 640 0
64 26699 181 11521706 320 4260
128 42695 441 17518361 160 209
256 24294 218 13739144 80 7
512 287 33 470915 40 4067
1K 403 193 28390362 20 6966374
2K 12 40 6629 10 5033
4K 18 1 282 5 0
8K 5 0 15 5 0
16K 13 0 13 5 0
32K 1 0 2 5 0
64K 5 0 34 5 0
Memory usage type by bucket size
Size Type(s)
16 devbuf, pcb, routetbl, vnodes, proc, temp
32 devbuf, pcb, routetbl, pgrp, session, subproc, ether_multi, temp,
sysctl
64 routetbl, ifaddr, namecache, VM mapent, VM pgdata, file, lockf,
in_multi, temp
128 devbuf, pcb, routetbl, zombie, ifaddr, cred, vnodes, VM map,
VM object, VM pgdata, file desc, temp, ttys, isa_devlist
256 devbuf, socket, pcb, routetbl, vnodes, VM map, VM pgdata, file desc,
subproc, FFS node, temp, isa_devlist
512 devbuf, pcb, ioctlops, mount, UFS mount, VM pgdata, file desc, proc,
temp, BIO buffer
1K devbuf, namei, UFS mount, VM pgdata, file desc, proc, BIO buffer,
select
2K devbuf, ioctlops, UFS mount, VM pgdata, file desc, BIO buffer
4K devbuf, ioctlops, VM pgdata, proc, temp
8K UFS mount, VM pgdata
16K mbuf, devbuf, VM pgdata
32K devbuf, VM pgdata
64K namecache, UFS quota, UFS mount, VM pgdata
Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
mbuf 1 16K 16K 19661K 1 0 0 16K
devbuf 79 269K 269K 19661K 120 0 0 16,32,128,256,512,1K,2K,4K,16K,32K
socket 1054 264K 292K 19661K 3327511 0 0 256
pcb 2058 386K 426K 19661K 4729672 0 0 16,32,128,256,512
routetbl 539 74K 222K 19661K 411222 0 0 16,32,64,128,256
zombie 1 1K 11K 19661K 470538 0 0 128
ifaddr 13 2K 2K 19661K 13 0 0 64,128
namei 0 0K 44K 19661K 25243015 0 0 1K
ioctlops 0 0K 4K 19661K 7 0 0 512,2K,4K
cred 174 22K 26K 19661K 445711 0 0 128
pgrp 139 5K 6K 19661K 423158 0 0 32
session 127 4K 6K 19661K 422463 0 0 32
mount 4 2K 2K 19661K 4 0 0 512
vnodes 21508 2685K 2703K 19661K 25175 0 0 16,128,256
namecache 21458 1406K 1406K 19661K 21458 0 0 64,64K
UFS quota 1 64K 64K 19661K 1 0 0 64K
UFS mount 10 89K 89K 19661K 10 0 0 512,1K,2K,8K,64K
VM map 263 65K 74K 19661K 470800 0 0 128,256
VM mapent 3757 235K 235K 19661K 3757 0 0 64
VM object 18658 2333K 2373K 19661K 13208180 0 0 128
VM pgdata 968 299K 299K 19661K 3521 0 0 64,128,256,512,1K,2K,4K,8K,16K,32K,64K
file 918 58K 66K 19661K 11150140 0 0 64
file desc 270 38K 42K 19661K 471019 0 0 128,256,512,1K,2K
lockf 13 1K 3K 19661K 344384 0 0 64
proc 268 138K 154K 19661K 486666 0 0 16,512,1K,4K
subproc 284 15K 17K 19661K 941359 0 0 32,256
FFS node 21453 5364K 5364K 19661K 7386682 0 0 256
in_multi 2 1K 1K 19661K 2 0 0 64
ether_multi 1 1K 1K 19661K 1 0 0 32
temp 49 10K 24K 19661K 48682 0 0 16,32,64,128,256,512,4K
ttys 830 104K 119K 19661K 8279 0 0 128
isa_devlist 1 1K 1K 19661K 2 0 0 128,256
sysctl 0 0K 1K 19661K 2 0 0 32
BIO buffer 203 203K 419K 19661K 2714243 0 0 512,1K,2K
select 195 195K 226K 19661K 439721 0 0 1K
Memory Totals: In Use Free Requests
14336K 441K 73197519
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199704111115.PAA11170>