Date: Thu, 1 Dec 2011 01:57:22 -0500 From: Jason Hellenthal <jhell@DataIX.net> To: Steven Hartland <killing@multiplay.co.uk> Cc: freebsd-hackers@freebsd.org Subject: Re: Invalid memory stats from vmstat and sysctl vm.vmtotal? Message-ID: <20111201065722.GA97051@DataIX.net> In-Reply-To: <547298A3C38F407887E1AAAAC487DF6D@multiplay.co.uk> References: <547298A3C38F407887E1AAAAC487DF6D@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Nov 30, 2011 at 12:39:10PM -0000, Steven Hartland wrote: > We're seeing some impossible memory usage stats reported on machines > here from vmstat and sysctl vm.vmtotal. > > We have machines reporting to be using 31GB total when they only have > 8GB physical and are not using any swap. > > Here's an output from one of our machines:- > vmstat -c 2 -w 1 -n 0 > procs memory page faults cpu > r b w avm fre flt re pi po fr sr in sy cs us sy id > 0 0 0 31768M 2112M 586 0 0 0 421 0 106 270 569 0 6 94 > 0 0 0 31768M 2112M 2 0 0 0 0 0 370 8139 3996 0 1 99 > > The raw output is:- > vmstat -c 2 -w 1 -n 0 -H > procs memory page faults cpu > r b w avm fre flt re pi po fr sr in sy cs us sy id > 0 0 0 32530228 2162524 586 0 0 0 421 0 106 270 569 0 6 94 > 0 0 0 32530228 2162524 2 0 0 0 0 0 286 8234 4347 0 1 99 > > Top shows:- > last pid: 6665; load averages: 0.00, 0.00, 0.01 up 80+01:24:12 09:35:28 > 1893 processes:1 running, 1892 sleeping > CPU: 0.0% user, 0.0% nice, 0.3% system, 0.0% interrupt, 99.7% idle > Mem: 3754M Active, 84M Inact, 1976M Wired, 4K Cache, 2109M Free > Swap: 4096M Total, 4096M Free > > sysctl vm.vmtotal > vm.vmtotal: > System wide totals computed every five seconds: (values in kilobytes) > =============================================== > Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 1893) > Virtual Memory: (Total: 1106403532K Active: 32540260K) > Real Memory: (Total: 4563648K Active: 3921644K) > Shared Virtual Memory: (Total: 19976K Active: 16396K) > Shared Real Memory: (Total: 9040K Active: 8436K) > Free Memory Pages: 2161740K > > As mentioned this machine has 8GB of ram and according to both top and > swapinfo is using no swap at all > > >From dmesg:- > real memory = 8589934592 (8192 MB) > avail memory = 8255553536 (7873 MB) > > swapinfo > Device 1K-blocks Used Avail Capacity > /dev/gptid/09f211f7-39ce-11e0-8 4194304 0 4194304 0% > > uname -a > FreeBSD test 8.2-RELEASE FreeBSD 8.2-RELEASE #2: Thu Mar 24 17:28:55 UTC 2011 root@test:/usr/obj/usr/src/sys/MULTIPLAY amd64 > > sysctl hw.pagesize > hw.pagesize: 4096 > > It looks like it may be out by a factor of 4, possibly due to the fact > the its a 4k page size not 1k as indicated by the vmstat man page:- > > memory Information about the usage of virtual and real memory. Virtual > pages (reported in units of 1024 bytes) are considered active if > they belong to processes which are running or have run in the > last 20 seconds. > > avm active virtual pages > fre size of the free list > > > Totalling up RSS from ps axo "rss" gives a total in the region of that if > the vm stats are out by a factor of 4, in this case it should be: 8132557 > which is 7.75GB a much more realistic value. > > Am I totally missing something or is there problem here? > > Regards > Steve > This goes along with the thoughts I had about 4 months ago tending to some zfs statistics as well top showing greater than 100% actual CPU usage. This is a big pet peave of mine. Its like saying you ate 134% of a bannanna when in all reallity it is impossible. You can never have more than 100% usage of anything and when seen is a clear notice that some math is considerably incorrect leading to other such miscalculations to be performed. Things like the above already have checks in place that ensure no boundries are being crossed/overflowed or underrun but it surely makes processing results building future products a bitch. One instance is the calculation of threads for example firefox can be seen using upto or more 338% of the CPU. Thats impossible its like saying anyones CPU grew by 400%.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111201065722.GA97051>