Date: Sat, 16 Feb 2013 13:05:50 -0600 From: Alan Cox <alc@rice.edu> To: Ian Lepore <ian@FreeBSD.org> Cc: "arm@freebsd.org" <arm@FreeBSD.org>, Kostik Belousov <kostikbel@gmail.com> Subject: Re: kernel address space & auto-tuning Message-ID: <511FD88E.2020403@rice.edu> In-Reply-To: <1361039490.1164.36.camel@revolution.hippie.lan> References: <51192C44.1060204@rice.edu> <1361039490.1164.36.camel@revolution.hippie.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On 02/16/2013 12:31, Ian Lepore wrote: > On Mon, 2013-02-11 at 11:37 -0600, Alan Cox wrote: >> Ian and others here, >> >> Could you please test the attached patch? However, before you apply >> this patch, can you run >> >> sysctl vm.max_kernel_address >> >> and record the output. I'd like to know how this output changes after >> the patch is applied. >> >> Here is an explanation of what the patch does: >> >> 1. Recently, a #define for VM_KMEM_SIZE_SCALE was added to >> arm/include/vmparam.h. This is good. However, it will create a problem >> as soon as someone gets arm hardware with more than ~1.5GB of RAM. This >> patch introduces a cap on the size of the kmem submap so that booting on >> future larger-memory systems won't fail. >> >> 2. It appears that arm is more like sparc64 than x86 in the respect that >> the ending address of the kernel address space can vary from machine to >> machine. To be more precise, I'm talking about the kernel address space >> into which we can dynamically map and unmap code/data and I'm not >> including regions for device access or direct maps. Thus, the current >> #define for VM_MAX_KERNEL_ADDRESS on arm is really incorrect or at least >> inconsistent with how this is defined on other architectures. This >> patch borrows from sparc64 in defining a global variable, >> vm_max_kernel_address, rather than a constant #define to represent the >> end of the kernel address space. >> >> Thanks, >> Alan >> > Took me a while to get to this, here's the results... > > DreamPlug (armv5) > > real memory = 536870912 (512 MB) > avail memory = 516612096 (492 MB) > > before: vm.max_kernel_address: 4294967295 0xFFFFFFFF > after: vm.max_kernel_address: 4026531840 0xF0000000 > > rpi (armv6) > > real memory = 536870912 (512 MB) > avail memory = 386789376 (368 MB) > > before: vm.max_kernel_address: 4294967295 0xFFFFFFFF > after: vm.max_kernel_address: 3741319168 0xDF000000 > > Beaglebone (armv7) > > real memory = 268435456 (256 MB) > avail memory = 254808064 (243 MB) > > before: vm.max_kernel_address: 4294967295 0xFFFFFFFF > after: vm.max_kernel_address: 3741319168 0xDF000000 > > It appears that the values that go into making the max kernel address on > various arm platforms may be choosen somewhat arbitrarily. Most SoCs > map the on-chip register space "somewhere" way up near the end of the > 32-bit virtual address space, and set the max kernel address to that > point. 0xD0000000, 0xE0000000, and 0xF0000000 all seem to be popular > choices. I just changed the value for DreamPlug, which only needs 1MB > of register mappings, from 0xF0000000 to 0xFE000000 and it seems to be > working fine. > > So, given that the variable ending address may be arbitrary and not much > different than the current 0xFFFFFFFF constant, I wonder if your changes > are going to have the intended effect? There are two parts to my change. One is making VM_MAX_KERNEL_ADDRESS accurately reflect the end of the kernel's address space by making it a variable. The other is placing a cap on the size of the kernel's heap, i.e., the kmem submap. Without that cap, machines with larger physical memories won't boot. In fact, I just saw an e-mail a few hours ago from Oleksandr Tymoshenko saying that his 1GB Beaglebone won't boot anymore. > ... It may be that the important > part of the change would come next: pick better arbitrary ending > addresses. Is the location of the register mapping something that the kernel chooses? Or is it something that is passed to the kernel at boot-time? > Or do what sparc seems to be doing, and choose a size then > set the address accordingly. > > Speaking of what sparc is doing, this sparc code looks suspicious: > > virtsz = roundup(5 / 3 * physsz, PAGE_SIZE_4M << > (PAGE_SHIFT - TTE_SHIFT)); > > I wonder if the intent there was "3 * physsz / 5"? > > -- Ian > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?511FD88E.2020403>