Date: Tue, 28 Oct 2008 11:27:55 -0700 From: Chuck Swiger <cswiger@mac.com> To: =?ISO-8859-1?Q?Francis_Dub=E9?= <freebsd@optiksecurite.com> Cc: Jeremy Chadwick <koitsu@FreeBSD.org>, freebsd-questions@freebsd.org Subject: Re: collecting pv entries -- suggest increasing PMAP_SHPGPERPROC Message-ID: <25FDD059-6BE7-4FAB-8FEB-C3D88789EFF2@mac.com> In-Reply-To: <4907428A.4030305@optiksecurite.com> References: <49060AE0.3000301@optiksecurite.com> <E0956D53-6EA3-4E70-91B2-6606EA48869A@mac.com> <49061898.60903@optiksecurite.com> <6DE465B2-C9D2-4A97-994F-D08CD7FE6211@mac.com> <20081027200253.GA29814@icarus.home.lan> <49072561.90904@optiksecurite.com> <20081028162511.GA53758@icarus.home.lan> <4907428A.4030305@optiksecurite.com>
index | next in thread | previous in thread | raw e-mail
On Oct 28, 2008, at 9:49 AM, Francis Dubé wrote: >>> Here's an example of top's output regarding our httpd process : >>> 54326 apache 1 96 0 156M 13108K select 1 0:00 >>> 0.15% httpd >>> 54952 apache 1 96 0 156M 12684K select 1 0:00 >>> 0.10% httpd >>> 52343 apache 1 4 0 155M 12280K select 0 0:01 >>> 0.10% httpd >>> >>> Most of our page are in HTML with a LOT of images. Few PHP pages, >>> very light PHP processing. >>> >>> 156M x 450 process = way more RAM than what we have (same for >>> RES). Concretely, how must I interpret these results? First, your Apache children are huge, at least for FreeBSD. :-) Also, they are mostly paged out, which suggests your system is under significant VM pressure, but the vmstat output would be helpful to confirm. >> It's as I expected -- you don't understand the difference between >> SIZE (SZ) and RES (RSS). The simple version: >> >> SIZE == amount of memory that's shared across all processes on the >> machine, e.g. shared libraries. It doesn't mean "156MB is being >> taken >> up per process". SIZE == the amount of VM address space allocated by the process. It includes things shared (copy-on-write) between many processes like the shared libraries; it also includes memory-mapped files (including .so's like apache modules being loaded into the process), VM allocated but not yet used by malloc()/brk(), the stack, and so forth. >> RES == amount of memory that's specifically allocated to that >> individual >> process. The three httpd processes above are taking up a total of >> ~38MBytes of memory (13108K + 12684K + 12280K). RES == the amount of process VM that is resident in actual physical RAM; the rest of the process is paged out to the swapfile or filesystem for memory-mapped files. > As I said, even with RES the numbers dont seems to have any sense. > > Let's say 12500K x 450 = ~5500MBytes. Considering there's a lot of > process other than Apache running on the server...there's something > wrong. Is there something shared in RES too ? Yep. Quite probably a lot, but the amount of memory which is specific to just that process is not easily found from FreeBSD's top, regrettably. For the sake of example, and because the same explanation applies pretty closly to FreeBSD, consider an httpd running on a MacOSX system. Here's top output, which includes columns "RPRVT" for "resident memory used by just this process", "RSHRD" which is "resident, shared with other processes", "RSIZE" which is FreeBSD's "RES", and "VSIZE", which is FreeBSD's "SIZE": Processes: 136 total, 4 running, 132 sleeping... 215 threads 11:06:40 Load Avg: 1.71, 1.66, 1.62 CPU usage: 12.5% user, 59.7% sys, 27.8% idle SharedLibs: num = 141, resident = 18.3M code, 2.92M data, 6.40M LinkEdit MemRegions: num = 10360, resident = 101M + 5.91M private, 159M shared PhysMem: 159M wired, 252M active, 99.0M inactive, 510M used, 1.50G free VM: 7.16G + 88.8M 1378510(0) pageins, 88743(0) pageouts PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE 2868 httpd 0.0% 43:21.28 1 12 92 1.82M 144M 72.9M 169M 2869 httpd 0.0% 46:29.45 1 12 92 1.95M 144M 73.2M 169M 2870 httpd 0.0% 46:55.84 1 12 92 1.89M 144M 73.0M 169M ...and the vmmap command, documented here: http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/vmmap.1.html ...provides detailed info about a single process' VM usage: # vmmap 2870 Virtual Memory Map of process 2870 (httpd) Output report format: 2.0 ==== Non-writable regions for process 2870 __PAGEZERO 00000000-00001000 [ 4K] ---/--- SM=NUL /usr/ sbin/httpd __TEXT 00001000-00050000 [ 316K] r-x/rwx SM=COW /usr/ sbin/httpd __LINKEDIT 0005a000-00065000 [ 44K] r--/rwx SM=COW /usr/ sbin/httpd __TEXT 00065000-00068000 [ 12K] r-x/rwx SM=COW /usr/ libexec/httpd/mod_log_config.so __LINKEDIT 00069000-0006a000 [ 4K] r--/rwx SM=COW /usr/ libexec/httpd/mod_log_config.so __TEXT 0006a000-0006c000 [ 8K] r-x/rwx SM=COW /usr/ libexec/httpd/mod_mime.so __LINKEDIT 0006d000-0006e000 [ 4K] r--/rwx SM=COW /usr/ libexec/httpd/mod_mime.so [ ... ] __DATA a1a0e000-a1a20000 [ 72K] r--/r-- SM=COW /usr/ lib/libcrypto.0.9.7.dylib __DATA a1a20000-a1a23000 [ 12K] r--/r-- SM=COW /usr/ lib/libcrypto.0.9.7.dylib __DATA a4f2c000-a4f2f000 [ 12K] r--/r-- SM=COW /usr/ lib/libssl.0.9.7.dylib __DATA a7233000-a7235000 [ 8K] r--/r-- SM=NUL / System/Library/Perl/lib/5.8/libperl.dylib system fffec000-fffef000 [ 12K] ---/rwx SM=NUL commpage [libobjc.A.dylib] system fffef000-ffff0000 [ 4K] r-x/rwx SM=COW commpage [libobjc.A.dylib] system ffff8000-ffffa000 [ 8K] r--/r-- SM=SHM commpage [libSystem.B.dylib] ==== Writable regions for process 2870 __DATA 00050000-00059000 [ 36K] rw-/rwx SM=COW /usr/ sbin/httpd __DATA 00059000-0005a000 [ 4K] rw-/rwx SM=COW /usr/ sbin/httpd __DATA 00068000-00069000 [ 4K] rw-/rwx SM=COW /usr/ libexec/httpd/mod_log_config.so __DATA 0006c000-0006d000 [ 4K] rw-/rwx SM=COW /usr/ libexec/httpd/mod_mime.so [ ... ] __DATA a0a3a000-a0a4f000 [ 84K] rw-/rw- SM=COW /usr/ lib/libobjc.A.dylib __OBJC a0a4f000-a0a50000 [ 4K] rw-/rw- SM=COW /usr/ lib/libobjc.A.dylib __DATA a0b70000-a0b71000 [ 4K] rw-/rw- SM=COW /usr/ lib/libauto.dylib __DATA a1425000-a1426000 [ 4K] rw-/rw- SM=COW /usr/ lib/libgcc_s.1.dylib __DATA a7229000-a7233000 [ 40K] rw-/rw- SM=COW / System/Library/Perl/lib/5.8/libperl.dylib Stack bf800000-c0000000 [ 8192K] rw-/rwx SM=COW thread 0 ==== Legend SM=sharing mode: COW=copy_on_write PRV=private NUL=empty ALI=aliased SHM=shared ZER=zero_filled S/A=shared_alias ==== Summary for process 2870 ReadOnly portion of Libraries: Total=18840KB resident=10976KB(58%) swapped_out_or_unallocated=7864KB(42%) Writable regions: Total=51100KB written=548KB(1%) resident=27276KB(53%) swapped_out=0KB(0%) unallocated=23824KB(47%) REGION TYPE [ VIRTUAL] =========== [ =======] MALLOC [ 42400K] Stack [ 8192K] VM_ALLOCATE ? [ 364K] __DATA [ 1048K] __LINKEDIT [ 4352K] __OBJC [ 4K] __PAGEZERO [ 4K] __TEXT [ 14488K] mapped file [ 120368K] shared memory [ 4K] system [ 24K] Note that you can obtain somewhat similar information under FreeBSD using the sysutils/pmap port: PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 40037 www 1 4 0 28820K 11532K accept 1:10 0.00% httpd # pmap 40037 40037: /usr/local/sbin/httpd Address Kbytes RSS Shared Priv Mode Mapped File 08048000 280 208 280 - r-x /usr/local/sbin/httpd 0808E000 8 8 - 8 rw- /usr/local/sbin/httpd 08090000 16 - - - rw- [swap pager] 08094000 976 - - - rwx [swap pager] 08188000 2420 - - - rwx [swap pager] 083E5000 6968 - - - rwx [swap pager] 2808E000 144 92 144 - r-x /libexec/ld-elf.so.1 280B2000 8 4 8 - rw- /libexec/ld-elf.so.1 280B4000 20 - - - rw- [swap pager] 280B9000 32 - - - rwx [swap pager] 280C1000 64 16 64 - r-x /lib/libz.so.3 [ ... ] 2920F000 4 0 4 - r-x /usr/local/lib/php/ 20060613/dom.so 29210000 16 0 16 - rwx /usr/local/lib/php/ 20060613/dom.so 29214000 60 - - - rwx [swap pager] 29225000 16 - - - rwx [swap pager] BFBE0000 128 - - - rwx [swap pager] -------- ------- ------- ------- ------- Total Kb 28740 3996 17384 456 ...and the pmap manpage talks about how to understand this: " To calculate the amount of memory a group of processes is using: - for each process of the same binary, count the private memory used for n-1 processes. - for each binary, count the total resident size once. For example, using the pmap data above, we can calculate that 20 bash binaries would used 452Kbytes * 19 + 1416Kbytes = 8588 + 1416 Kbytes = 8588Kbytes" Regards, -- -Chuckhelp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?25FDD059-6BE7-4FAB-8FEB-C3D88789EFF2>