From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 4 22:03:21 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAA981065670 for ; Sun, 4 Dec 2011 22:03:21 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 41A158FC0C for ; Sun, 4 Dec 2011 22:03:20 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA00497; Mon, 05 Dec 2011 00:03:18 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RXK9V-000I7S-NH; Mon, 05 Dec 2011 00:03:17 +0200 Message-ID: <4EDBEE25.9050008@FreeBSD.org> Date: Mon, 05 Dec 2011 00:03:17 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111108 Thunderbird/8.0 MIME-Version: 1.0 To: Steven Hartland References: <547298A3C38F407887E1AAAAC487DF6D@multiplay.co.uk> <4ED8E9D0.4070006@FreeBSD.org> <2537A2099932494E9E7019B53EBDAA34@multiplay.co.uk> In-Reply-To: <2537A2099932494E9E7019B53EBDAA34@multiplay.co.uk> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org Subject: Re: Invalid memory stats from vmstat and sysctl vm.vmtotal? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Dec 2011 22:03:22 -0000 on 02/12/2011 19:13 Steven Hartland said the following: [snip] > Given this starting point the following links provided me with addtional > information:- > http://www.freebsd.org/doc/en/books/arch-handbook/vm.html > http://www.freebsd.org/doc/en/books/design-44bsd/overview-memory-management.html > http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/ > http://www.cse.chalmers.se/edu/course/EDA203/unix4.pdf > > I was under the incorrect impression that Virtual Memory (VM) was so named as it > was a unified physical memory and swap (virtual memory), but its not that simple, > as other items such as file-backed objects also count to this total which would > never show in physical or swap allocation of other tools such as top and swapinfo. > > So what I believe is now the big cause of virtual memory uplift vs the memory > totals shown by ps / top is that the vm totals include things like file backed > memory mapped process binaries, shared libs etc many multiple times. > > This would explain why this specific machine shows the applification more than > others here as it runs thousands of very small lightweight processes. > > Thanks for pointer Andy, I now understand a lot more about the BSD VMS :) > > What do people think about expanding that entry in the man page of vmstat to > clarify just what active "virtual pages" really means? Thank you for the excellent additional research. I think that the expanded explanation would be very appreciated, but only if it is completely correct and unambiguous from technical perspective. That might not be trivial to achieve, but is realistic :-) P.S. I think that it would also be nice to further breakdown VM stats by type of object e.g. anonymous (swap-backed) memory vs file-backed backed memory, etc. Proportional RSS reporting is another nice to have feature. -- Andriy Gapon