From owner-freebsd-hackers Thu Jan 30 11:23:33 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B75E437B405 for ; Thu, 30 Jan 2003 11:23:31 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 78BBB43E4A for ; Thu, 30 Jan 2003 11:23:30 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0UJNT0i089038; Thu, 30 Jan 2003 11:23:30 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0UJNT0l089037; Thu, 30 Jan 2003 11:23:29 -0800 (PST) Date: Thu, 30 Jan 2003 11:23:29 -0800 (PST) From: Matthew Dillon Message-Id: <200301301923.h0UJNT0l089037@apollo.backplane.com> To: David Schultz Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> <20030130091419.GA7776@HAL9000.homeunix.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I think the original poster wanted to know the real memory use of :a set of processes, taking sharing into account. I don't see how :your approach could do that. Even if you knew the structure of :the shadow chain, you would have to know specifically which pages :had been COWed, no? : :I thought counting vm_page structures would be the way to go... :I really want to find the time to learn all this stuff better. :I still don't know the difference between COW and NEEDS_COPY, :or why the pageout daemon seems to be biased against shared :pages, or a lot of things about pv_entry structures. It's not possible to get a wholely accurate count no matter what you do. For example, to truely know whether a process is using a page you have to run through the process's page table (PMAP), access the vm_page, then locate where in the shadow chain the VM object the vm_page belongs to resides. But since hardware page tables are throw-away, the system could very well have thrown away whole page tables so this method is no more accurate then any other. Shadow VM objects are optimized on the fly. When a shadow object representing data shared by multiple processes is completely covered (due to COW faults) the system will delete the shadow object. So you can get a pretty good idea in regards to shared pages simply by noting the existance of the shadow object and the number of resident or swap pages residing in that object. MAP_ENTRY_NEEDS_COPY is a vm_map_entry optimization that allows two vm_map_entry's (for two different processes) to share the same vm_object even though they are copy-on-write. This allows us to defer allocating new VM objects (defer creating the shadow structures) for anonymous area of memory when a process fork()s, until an actual copy-on-write occurs. Being able to defer this allocation greatly reduces the time and memory required to fork() and gives us a flatter, more easily optimized VM object structure. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message