From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 31 00:22:31 2007 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A36AA16A421; Mon, 31 Dec 2007 00:22:31 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 67BA113C46B; Mon, 31 Dec 2007 00:22:31 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.1/8.13.7) with ESMTP id lBV0BuWf086564; Sun, 30 Dec 2007 16:11:57 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.14.1/8.13.4/Submit) id lBV0BuvT086563; Sun, 30 Dec 2007 16:11:56 -0800 (PST) Date: Sun, 30 Dec 2007 16:11:56 -0800 (PST) From: Matthew Dillon Message-Id: <200712310011.lBV0BuvT086563@apollo.backplane.com> To: Kris Kennaway References: <47759641.9090604@FreeBSD.org> Cc: alc@freebsd.org, hackers@freebsd.org Subject: Re: prefaulting MAP_ANONYMOUS pages X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 00:22:31 -0000 :1) Lots of page faults, which drop performance by a factor of 10 :compared to the case where everything is faulted in. : :... :Is there a way to achieve this that I am overlooking? If not can :someone give me some advice about what is needed? : :Kris The main problem here is that the backing pages do not exist anywhere. No backing store is assigned and the pages themselves aren't even allocated until the user touches the memory. You might be able to work magic with the default pager. The default pager handles all UNBACKED anonymous mappings. Default pager objects are quietly converted to swap pager objects when the kernel tries to page them out. In otherwords, the 'backing' store for the default pager is always zero-fill. default_pager_haspage() returns FALSE which basically causes the VM layer to fall through to zero-fill fault code. It optimizes the fault by allocating out of the zero'd page pool in order to avoid having to bzero the page. Because default_pager_haspage() returns FALSE, default_pager_getpages() is never called. You could theoretically have default_pager_haspage() return TRUE and also load up the before and after variables, and then manually zero-fill the pages in default_pager_getpages(). This is probably the simplest solution, but not necessarily optimal since you will not necessarily be provided with pre-zero'd pages. Another possibility would be to have the VM code which handles a FALSE return from *_pager_haspage() to check additional adjacent pages and zero-fill a larger block instead of just one page. This would be more work but would also be a more optimal solution. A third possibility would be to add an additional argument to *_pager_haspage() that allows it to tell the caller how many pages do not have backing store, so the kernel doesn't have to poll for each page and can zero-fill several at once. This is more complex but probably the most optimal solution. -- Note that these sorts of optimizations could create non-optimal operation for unrelated programs which rely on sparse allocations. I am not sure if FreeBSD has a heuristic for VM faults... I think it does. That heuristic could be used to determine how much fault-ahead to actually do. It may already exist, I haven't checked, and simply need to be applied to whatever code you cook up for the zero-fill fault. Also note that OBJT_DEFAULT and OBJT_SWAP (default and swap pagers) are special cased all over the code base. There might be unexpected side effects if you mess with the default_pager*() functions. -Matt Matthew Dillon