From owner-freebsd-hackers@FreeBSD.ORG  Sat Dec 29 00:35:15 2007
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: hackers@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 9B5D816A418;
	Sat, 29 Dec 2007 00:35:15 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id F357313C44B;
	Sat, 29 Dec 2007 00:35:14 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <47759641.9090604@FreeBSD.org>
Date: Sat, 29 Dec 2007 01:35:13 +0100
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031)
MIME-Version: 1.0
To: alc@FreeBSD.org, hackers@FreeBSD.org
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: 
Subject: prefaulting MAP_ANONYMOUS pages
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, 
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 29 Dec 2007 00:35:15 -0000

I am trying to optimize a malloc-based benchmark that is mmapping 
anonymous memory (via mmap) and then eventually taking a page fault on 
every page that was allocated.  This is pretty inefficient for two reasons:

1) Lots of page faults, which drop performance by a factor of 10 
compared to the case where everything is faulted in.

2) Lack of concurrency when faulting from multiple threads on the same 
object (it is basically serialized, so no benefits from multiple CPUs)

I think a better way to go might be to do prefetching to reduce the 
number of page faults.  We do this for other types of mappings, but 
apparently not for anonymous mmapped memory.  I am still trying to get 
my head around the code here, but it looks like vm_fault_prefault is 
going to return without doing anything because of

                 while ((m = vm_page_lookup(lobject, pindex)) == NULL &&
...

                 /*
                  * give-up when a page is not in memory
                  */

Also vm_fault_additional_pages() isn't getting called because TRYPAGER 
returns false for default objects.

Is there a way to achieve this that I am overlooking?  If not can 
someone give me some advice about what is needed?

Kris