Date: Sun, 1 Feb 2004 00:30:43 +0100 From: Jorn Argelo <jorn@wcborstel.nl> To: dpk <dpk@dpk.net> Cc: questions@freebsd.org Subject: Re: Memory leak on 5.1-RELEASE? Message-ID: <200402010030.43426.jorn@wcborstel.nl> In-Reply-To: <Pine.BSF.4.58.0401311354210.98773@shared10.hosting.flyingcroc.net> References: <Pine.BSF.4.58.0401301550310.98773@shared10.hosting.flyingcroc.net> <012701c3e8ea$d6436c40$050110ac@computerjorn> <Pine.BSF.4.58.0401311354210.98773@shared10.hosting.flyingcroc.net>
next in thread | previous in thread | raw e-mail | index | archive | help
How large are you MySQL databases? I suppose a large and intensly-used database eats away quite some RAM. What services is the inetd giving? You can disable some of your shells by editing the /etc/ttys file, saves some RAM as well, but it won't really make a difference when you have such a large amount of RAM in your machine. You should really try to disable some unnessesairy services. If it isn't a production box, then I'd say you kill the services one at a time and bring them back up again, so you can see what is eating away all your RAM. It could be that top is just giving wrong information though, but I can't be certain of that. If your machine is just running the required processes and top still says you're using much RAM, you can try to compile the kernel / world by yourself. This can improve performance and reducing the required RAM needed by the kernel (support for several RAID and SCSI controllers which you don't need). Of course, this isn't possible if it's a production box. Cheers, Jorn. On Saturday 31 January 2004 23:04, you wrote: > Here's the top output from this morning: > > Mem: 1300M Active, 292M Inact, 295M Wired, 104M Cache, 112M Buf, 5116K Free > Swap: 1024M Total, 8036K Used, 1016M Free > > At this time there are ~130 processes, whereas in the previous one > there were ~76, with more memory marked as "Active". vmstat says this: > > procs memory page disks faults cpu > r b w avm fre flt re pi po fr sr da0 ac0 in sy cs us sy > id 0 0 1 848048 100880 937 113 23 0 1503 248 0 0 2678 0 8211 2 > 4 95 0 0 1 830192 110484 340 4 1 0 2719 0 1 0 2574 0 5917 > 1 5 93 > > avm is supposed to be the number of active virtual pages. The page size > being 4096, according to hw.pagesize, would actually come up to around > 3.4GB. I'm not sure what this number refers to - maybe it's memory + swap? > > The whole reason I'm bringing this up with the list is that, ultimately, > this is causing the machine to crash, after running out of swap. If I > could identify where this Active memory has gone, I may be able to recover > the "lost" memory and prevent the server from swapping itself to death. > > Here's the ps -ax output - note that nfsd is running, but it's not being > used. It does have an NFS client mounted, but from "vmstat -m", it isn't > using any memory at all (and it would be in Buf I think?) > > PID TT STAT TIME COMMAND > 0 ?? DLs 0:00.82 (swapper) > 0 ?? ZW 0:00.00 (httpd) > 1 ?? ILs 0:00.08 /sbin/init -- > 2 ?? DL 0:13.59 (g_event) > 3 ?? DL 3:41.95 (g_up) > 4 ?? DL 4:56.15 (g_down) > 5 ?? DL 1:41.88 (pagedaemon) > 6 ?? DL 0:00.00 (vmdaemon) > 7 ?? DL 16:25.54 (pagezero) > 8 ?? DL 2:07.11 (bufdaemon) > 9 ?? DL 0:01.51 (vnlru) > 10 ?? DL 0:00.00 (ktrace) > 11 ?? RL 0:00.00 (idle: cpu3) > 12 ?? RL 5037:43.41 (idle: cpu2) > 13 ?? RL 0:00.00 (idle: cpu1) > 14 ?? RL 4281:37.99 (idle: cpu0) > 15 ?? WL 127:48.20 (swi1: net) > 16 ?? WL 6:02.49 (swi7: tty:sio clock) > 18 ?? DL 1:26.21 (random) > 20 ?? WL 1:39.12 (swi3: cambio) > 23 ?? WL 0:00.00 (irq14: ata0) > 26 ?? DL 0:00.02 (usb0) > 27 ?? DL 0:00.00 (usbtask) > 30 ?? WL 2:13.96 (irq16: ahc0) > 31 ?? WL 0:00.00 (irq17: ahc1) > 32 ?? WL 58:17.09 (irq18: em0) > 33 ?? WL 0:00.00 (irq1: atkbd0) > 34 ?? WL 0:00.00 (irq6: fdc0) > 35 ?? WL 0:00.00 (swi0: tty:sio) > 40 ?? DL 13:22.17 (syncer) > 41 ?? IL 3:06.26 (nfsiod 0) > 42 ?? IL 2:47.50 (nfsiod 1) > 43 ?? IL 2:28.19 (nfsiod 2) > 44 ?? IL 3:37.52 (nfsiod 3) > 148 ?? Is 0:00.00 adjkerntz -i > 257 ?? Ss 0:02.08 /usr/sbin/syslogd -s > 352 ?? Ss 0:00.25 /usr/sbin/rpcbind > 375 ?? Is 0:00.01 /usr/sbin/mountd -r > 377 ?? Is 0:00.02 nfsd: master (nfsd) > 380 ?? I 0:00.00 nfsd: server (nfsd) > 381 ?? I 0:00.00 nfsd: server (nfsd) > 382 ?? I 0:00.00 nfsd: server (nfsd) > 383 ?? I 0:00.00 nfsd: server (nfsd) > 394 ?? Ss 0:00.35 /usr/sbin/usbd > 443 ?? Is 0:02.00 /usr/sbin/sshd -u0 > 449 ?? Ss 0:06.17 sendmail: accepting connections (sendmail) > 452 ?? Is 0:00.13 sendmail: Queue runner@00:30:00 for > /var/spool/client 466 ?? Is 0:01.25 /usr/sbin/cron > 829 ?? Ss 0:02.75 proftpd: (accepting connections) (proftpd) > 855 ?? Is 0:00.00 /usr/sbin/inetd -wW > 21727 ?? Ss 2:11.11 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 31784 ?? S 0:00.23 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 52298 ?? S > 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 52989 ?? S > 0:00.18 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 53224 ?? > S 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 54180 > ?? S 0:00.18 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 54282 ?? S 0:00.13 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 54310 ?? S 0:00.13 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 55563 ?? S > 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 55970 ?? S > 0:00.16 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56261 ?? > S 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56369 > ?? S 0:00.10 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56422 ?? S 0:00.04 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56424 ?? S 0:00.18 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56465 ?? S > 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56491 ?? S > 0:00.17 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56523 ?? > S 0:00.12 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56575 > ?? Rs 0:00.07 sshd: toor@ttyp0 (sshd) > 56606 ?? S 0:00.33 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56617 ?? S 0:00.15 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56729 ?? I > 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56763 ?? S > 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56768 ?? > S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56779 > ?? S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56786 ?? S 0:00.05 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56788 ?? S 0:00.00 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56789 ?? S > 0:00.12 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56790 ?? S > 0:00.08 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56800 ?? > S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56809 > ?? I 0:00.01 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56811 ?? S 0:00.13 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56840 ?? S 0:00.15 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56843 ?? S > 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56845 ?? S > 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56846 ?? > S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56852 > ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56853 ?? S 0:00.05 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56864 ?? S 0:00.11 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56865 ?? S > 0:00.17 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56866 ?? S > 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56869 ?? > S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56870 > ?? S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56871 ?? S 0:00.05 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56872 ?? S 0:00.09 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56873 ?? S > 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56874 ?? S > 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56875 ?? > S 0:00.07 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56876 > ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56877 ?? S 0:00.03 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56878 ?? S 0:00.04 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56879 ?? S > 0:00.07 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56880 ?? S > 0:00.07 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56881 ?? > S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56882 > ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56883 ?? S 0:00.05 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56884 ?? S 0:00.03 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56885 ?? S > 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56886 ?? S > 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56887 ?? > S 0:00.12 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56888 > ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56889 ?? S 0:00.03 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56890 ?? S 0:00.03 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56891 ?? S > 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56892 ?? S > 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56893 ?? > S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56894 > ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht > 56896 ?? S 0:00.00 /usr/local/bin/httpd -f > /usr/local/www/server/conf/ht 56897 ?? S 0:00.02 > /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56898 ?? S > 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56900 ?? S > 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56577 p0 > Ss 0:00.02 -sh (sh) > 56904 p0 R+ 0:00.00 ps -ax > 3574 v0 Is+ 0:00.00 /usr/libexec/getty Pc ttyv0 > 868 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1 > 869 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2 > 870 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3 > 871 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4 > 872 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5 > 873 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6 > 874 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7 > 520 con- I 0:02.81 /usr/local/sbin/snmpd > 534 con- I 0:00.01 /bin/sh /usr/local/bin/mysqld_safe > --datadir=/usr/loc 566 con- S 138:26.09 /usr/local/libexec/mysqld > --basedir=/usr/local --data > > According to top and ps, mysql is only using around 59M VSZ. > > On Sun, 1 Feb 2004, Jorn Argelo wrote: > > Are you sure that there isn't anything else running? Why don't you give > > us an ps -ax output? I don't think there's a memory leak in 5.1, since > > I've seen running 5.1 just fine on a PE2650 with 2 GB RAM. You shouldn't > > rely on top too much acually. Vmstat is a better program when looking at > > memory. > > > > Cheers, > > > > Jorn > > ----- Original Message ----- > > From: "dpk" <dpk@dpk.net> > > To: <questions@freebsd.org> > > Sent: Saturday, January 31, 2004 1:10 AM > > Subject: Memory leak on 5.1-RELEASE? > > > > > (I'm not a member of the list; please Cc me on any replies.) > > > > > > We're running Apache 1.3.28 on a 5.1-RELEASE machine. It's a Dell PE > > > 2650 w/ 2GB RAM. The site contains a lot of large files > > > (multi-megabyte) - otherwise there's nothing unusual running. > > > > > > The Active memory use, according to top, seems rather high: > > > > > > last pid: 21487; load averages: 0.19, 0.33, 0.32 up 2+16:45:20 > > > > 15:52:21 > > > > > 76 processes: 1 running, 75 sleeping > > > CPU states: 0.5% user, 0.0% nice, 4.0% system, 1.4% interrupt, > > > 94.2% > > > > idle > > > > > Mem: 1413M Active, 187M Inact, 299M Wired, 93M Cache, 112M Buf, 2632K > > > Free Swap: 1024M Total, 21M Used, 1003M Free, 2% Inuse > > > > > > We can't seem to get the Active number down much, even after stopping > > > Apache it still stays around 1100M. There's no shared memory in use, > > > and nothing in vmstat -m seems to indicate where the "missing" memory > > > is. top, sorting by "size", does not indicate anything unusual either. > > > > > > sysctl vm.vmtotal says: > > > > > > vm.vmtotal: > > > System wide totals computed every five seconds: (values in kilobytes) > > > =============================================== > > > Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 76) > > > Virtual Memory: (Total: 8172K, Active 636472K) > > > Real Memory: (Total: 2051312K Active 389176K) > > > Shared Virtual Memory: (Total: 16436K Active: 11760K) > > > Shared Real Memory: (Total: 6004K Active: 4436K) > > > Free Memory Pages: 79228K > > > > > > whereas on other servers, the Real Memory "Active" number seems to > > > match the one found in top, on this one it is about 1GB lower. > > > > > > A similar machine running Apache on 5.1-R, generally serving smaller > > > files, has the same problem in a smaller scale (about 640M even when > > > Apache is stopped). > > > > > > Are there any other data that I should send to help diagnose this > > > problem, or any programs I can run to try and track this stray memory > > > use down? > > > > > > - dpk > > > _______________________________________________ > > > freebsd-questions@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > > > To unsubscribe, send any mail to > > > > "freebsd-questions-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200402010030.43426.jorn>