From owner-freebsd-ppc@FreeBSD.ORG Mon Feb 25 00:08:13 2013 Return-Path: Delivered-To: ppc@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E050B5B3; Mon, 25 Feb 2013 00:08:13 +0000 (UTC) (envelope-from alc@rice.edu) Received: from mh2.mail.rice.edu (mh2.mail.rice.edu [128.42.201.21]) by mx1.freebsd.org (Postfix) with ESMTP id B9C25F4C; Mon, 25 Feb 2013 00:08:13 +0000 (UTC) Received: from mh2.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh2.mail.rice.edu (Postfix) with ESMTP id 1B484500136; Sun, 24 Feb 2013 18:08:07 -0600 (CST) Received: from mh2.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh2.mail.rice.edu (Postfix) with ESMTP id 1A018500128; Sun, 24 Feb 2013 18:08:07 -0600 (CST) X-Virus-Scanned: by amavis-2.7.0 at mh2.mail.rice.edu, auth channel Received: from mh2.mail.rice.edu ([127.0.0.1]) by mh2.mail.rice.edu (mh2.mail.rice.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id Icj57zG2gxJN; Sun, 24 Feb 2013 18:08:07 -0600 (CST) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) (Authenticated sender: alc) by mh2.mail.rice.edu (Postfix) with ESMTPSA id 326AF500101; Sun, 24 Feb 2013 18:08:06 -0600 (CST) Message-ID: <512AAB65.1000402@rice.edu> Date: Sun, 24 Feb 2013 18:08:05 -0600 From: Alan Cox User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:17.0) Gecko/20130127 Thunderbird/17.0.2 MIME-Version: 1.0 To: ppc@freebsd.org Subject: kernel address space and auto-sizing Content-Type: multipart/mixed; boundary="------------010606070608060509000504" Cc: Kostik Belousov , grehan@freebsd.org, Alan Cox X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2013 00:08:13 -0000 This is a multi-part message in MIME format. --------------010606070608060509000504 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit After the fallout from the recent auto-sizing changes, I've been reviewing the kernel address space configuration on all of our supported architectures. This review turned up a couple of odd things on the AIM platform: 1. Everywhere else, except for 32- and 64-bit AIM, VM_MAX_KERNEL_ADDRESS denotes the end of the kernel map. Instead, 32- and 64-bit AIM define VM_MAX_SAFE_KERNEL_ADDRESS and uses this to initialize the end of the kernel map. On occasion, we use VM_MAX_KERNEL_ADDRESS to implement auto-sizing based on the size of the kernel map, so this difference interferes with that. 2. The size of the kmem submap on 32-bit AIM is hardwired to only 12 MB! ARM had the same issue. There, the consequences were much worse because ARM, unlike 32-bit AIM, doesn't have a direct map, so uma_small_alloc() can't be used on arm. However, even with uma_small_alloc(), this 12 MB cap on the kmem submap still unnecessarily limits multipage allocations by the kernel. Even with a direct map, those come from the kmem submap. The attached patch does the following things to AIM: 1. On 32-bit AIM, it redefines VM_MAX_KERNEL_ADDRESS to be the address that should be the end of the kernel map and eliminates VM_MAX_SAFE_KERNEL_ADDRESS. On 64-bit AIM, VM_MAX_KERNEL_ADDRESS and VM_MAX_SAFE_KERNEL_ADDRESS were the same, so it simply eliminates VM_MAX_SAFE_KERNEL_ADDRESS. 2. It enables auto-sizing of the kmem submap on 32-bit AIM, but places a cap on its maximum size so that it will not consume the entire kernel map. (64-bit AIM already has auto-sizing and a cap for the kmem submap.) I would appreciate review and/or testing of this patch. In particular, I'd like to see the output from two sysctl's before and after applying the patch: vm.max_kernel_address, vm.kmem_size, and vm.kmem_size_max Thanks, Alan P.S. On 64-bit AIM, the following bit of code in the pmap bootstrap function was already a NOP because VM_MAX_KERNEL_ADDRESS and VM_MAX_SAFE_KERNEL_ADDRESS were the same. I didn't touch it, but I thought that I would mention it. #ifndef __powerpc64__ /* KVA is in high memory on PPC64 */ PMAP_LOCK(kernel_pmap); while (virtual_end < VM_MAX_KERNEL_ADDRESS && moea64_pvo_find_va(kernel_pmap, virtual_end+1) == NULL) virtual_end += PAGE_SIZE; PMAP_UNLOCK(kernel_pmap); #endif --------------010606070608060509000504 Content-Type: text/plain; charset=ISO-8859-15; name="aim3.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="aim3.patch" Index: powerpc/aim/mmu_oea.c =================================================================== --- powerpc/aim/mmu_oea.c (revision 247166) +++ powerpc/aim/mmu_oea.c (working copy) @@ -934,7 +934,7 @@ moea_bootstrap(mmu_t mmup, vm_offset_t kernelstart * Set the start and end of kva. */ virtual_avail = VM_MIN_KERNEL_ADDRESS; - virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; + virtual_end = VM_MAX_KERNEL_ADDRESS; /* * Allocate a kernel stack with a guard page for thread0 and map it Index: powerpc/aim/mmu_oea64.c =================================================================== --- powerpc/aim/mmu_oea64.c (revision 247166) +++ powerpc/aim/mmu_oea64.c (working copy) @@ -872,7 +872,7 @@ moea64_late_bootstrap(mmu_t mmup, vm_offset_t kern * Set the start and end of kva. */ virtual_avail = VM_MIN_KERNEL_ADDRESS; - virtual_end = VM_MAX_SAFE_KERNEL_ADDRESS; + virtual_end = VM_MAX_KERNEL_ADDRESS; /* * Map the entire KVA range into the SLB. We must not fault there. Index: powerpc/include/vmparam.h =================================================================== --- powerpc/include/vmparam.h (revision 247166) +++ powerpc/include/vmparam.h (working copy) @@ -93,11 +93,9 @@ #ifdef __powerpc64__ #define VM_MIN_KERNEL_ADDRESS 0xc000000000000000UL #define VM_MAX_KERNEL_ADDRESS 0xc0000001c7ffffffUL -#define VM_MAX_SAFE_KERNEL_ADDRESS VM_MAX_KERNEL_ADDRESS #else #define VM_MIN_KERNEL_ADDRESS ((vm_offset_t)KERNEL_SR << ADDR_SR_SHFT) -#define VM_MAX_SAFE_KERNEL_ADDRESS (VM_MIN_KERNEL_ADDRESS + 2*SEGMENT_LENGTH -1) -#define VM_MAX_KERNEL_ADDRESS (VM_MIN_KERNEL_ADDRESS + 3*SEGMENT_LENGTH - 1) +#define VM_MAX_KERNEL_ADDRESS (VM_MIN_KERNEL_ADDRESS + 2*SEGMENT_LENGTH - 1) #endif /* @@ -188,13 +186,16 @@ struct pmap_physseg { #define VM_KMEM_SIZE (12 * 1024 * 1024) #endif -#ifdef __powerpc64__ #ifndef VM_KMEM_SIZE_SCALE #define VM_KMEM_SIZE_SCALE (3) #endif #ifndef VM_KMEM_SIZE_MAX +#ifdef __powerpc64__ #define VM_KMEM_SIZE_MAX 0x1c0000000 /* 7 GB */ +#else +#define VM_KMEM_SIZE_MAX ((VM_MAX_KERNEL_ADDRESS - \ + VM_MIN_KERNEL_ADDRESS + 1) * 2 / 5) #endif #endif --------------010606070608060509000504--