From owner-freebsd-mips@FreeBSD.ORG Tue Jun 8 06:33:18 2010 Return-Path: Delivered-To: mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F72C106567A; Tue, 8 Jun 2010 06:33:18 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.freebsd.org (Postfix) with ESMTP id 301BC8FC22; Tue, 8 Jun 2010 06:33:17 +0000 (UTC) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 83BAA2C2D0A; Tue, 8 Jun 2010 01:33:17 -0500 (CDT) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id XMHGj+ApzUmh; Tue, 8 Jun 2010 01:33:09 -0500 (CDT) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 269672C2A92; Tue, 8 Jun 2010 01:33:09 -0500 (CDT) Message-ID: <4C0DE424.9080601@cs.rice.edu> Date: Tue, 08 Jun 2010 01:33:08 -0500 From: Alan Cox User-Agent: Thunderbird 2.0.0.24 (X11/20100501) MIME-Version: 1.0 To: "C. Jayachandran" References: <4C07E07B.9060802@cs.rice.edu> <4C09345F.9040300@cs.rice.edu> <4C0D2BEA.6060103@cs.rice.edu> <4C0D3F40.2070101@cs.rice.edu> <20100607202844.GU83316@deviant.kiev.zoral.com.ua> <4C0D64B7.7060604@cs.rice.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , "Jayachandran C." , Alan Cox , mips@freebsd.org Subject: Re: svn commit: r208589 - head/sys/mips/mips X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2010 06:33:18 -0000 C. Jayachandran wrote: > On Tue, Jun 8, 2010 at 2:59 AM, Alan Cox wrote: > >> On 6/7/2010 3:28 PM, Kostik Belousov wrote: >> >>> Selecting a random message in the thread to ask my question. >>> Is the issue that page table pages should be allocated from the specific >>> physical region of the memory ? If yes, doesn't i386 PAE has similar >>> issue with page directory pointer table ? I see a KASSERT in i386 >>> pmap that verifies that the allocated table is below 4G, but I do not >>> understand how uma ensures the constraint (I suspect that it does not). >>> >>> >> For i386 PAE, the UMA backend allocator uses kmem_alloc_contig() to ensure >> that the memory is below 4G. The crucial difference between i386 PAE and >> MIPS is that for i386 PAE only the top-level table needs to be below a >> specific address threshold. Moreover, this level is allocated in a place, >> pmap_pinit(), where we are allowed to sleep. >> > > Yes. I saw the PAE top level page table code and thought I could use > that mechanism for allocating MIPS page table pages in the direct > mapped memory. The other reference I used was > pmap_alloc_zeroed_contig_pages() function in sun4v/sun4v/pmap.c which > uses the vm_phys_alloc_contig() and VM_WAIT. That's unfortunate. :-( Since sun4v is essentially dead code, I've never spent much time thinking about its pmap implementation. I'll mechanically apply changes to it, but that's about it. I wouldn't recommend using it as a reference. > ... I had also thought of > using the VM_FREEPOOL_DIRECT which seemed to be for a similar purpose, > but could find see any usage in the kernel. > > VM_FREEPOOL_DIRECT is used by at least amd64 and ia64 for page table pages and small kernel memory allocations. Unlike mips, these machines don't have MMU support for a direct map. Their direct maps are just a range of mappings in the regular (kernel) page table. So, unlike mips, accesses through their direct map may still miss in the TLB and require a page table walk. VM_FREEPOOL_* is a way to increase the physical locality (or clustering) of page allocations, so that, for example, page table page accesses by the pmap on amd64 are less likely to miss in the TLB. However, it doesn't place a hard restriction on the range of physical addresses that will be used, which you need for mips. The impact of this clustering can be significant. For example, on amd64 we use 2MB page mappings to implement the direct map. However, old Opterons only had 8 data TLB entries for 2MB page mappings. For a uniprocessor kernel running on such an Opteron, I measured an 18% reduction in system time during a buildworld with the introduction of VM_FREEPOOL_DIRECT. (See the commit logs for vm/vm_phys.c and the comment that precedes the VM_NFREEORDER definition on amd64.) Until such time as superpage support is ported to mips from the amd64/i386 pmaps, I don't think there is a point in having more than one VM_FREEPOOL_* on mips. And then, the point would be to reduce fragmentation of the physical memory that could be caused by small allocations, such as page table pages. Alan