Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 16 Feb 2013 13:05:50 -0600
From:      Alan Cox <alc@rice.edu>
To:        Ian Lepore <ian@FreeBSD.org>
Cc:        "arm@freebsd.org" <arm@FreeBSD.org>, Kostik Belousov <kostikbel@gmail.com>
Subject:   Re: kernel address space & auto-tuning
Message-ID:  <511FD88E.2020403@rice.edu>
In-Reply-To: <1361039490.1164.36.camel@revolution.hippie.lan>
References:  <51192C44.1060204@rice.edu> <1361039490.1164.36.camel@revolution.hippie.lan>

next in thread | previous in thread | raw e-mail | index | archive | help
On 02/16/2013 12:31, Ian Lepore wrote:
> On Mon, 2013-02-11 at 11:37 -0600, Alan Cox wrote:
>> Ian and others here,
>>
>> Could you please test the attached patch?  However, before you apply
>> this patch, can you run
>>
>>     sysctl vm.max_kernel_address
>>
>> and record the output.  I'd like to know how this output changes after
>> the patch is applied.
>>
>> Here is an explanation of what the patch does:
>>
>> 1. Recently, a #define for VM_KMEM_SIZE_SCALE was added to
>> arm/include/vmparam.h.  This is good.  However, it will create a problem
>> as soon as someone gets arm hardware with more than ~1.5GB of RAM.  This
>> patch introduces a cap on the size of the kmem submap so that booting on
>> future larger-memory systems won't fail.
>>
>> 2. It appears that arm is more like sparc64 than x86 in the respect that
>> the ending address of the kernel address space can vary from machine to
>> machine.  To be more precise, I'm talking about the kernel address space
>> into which we can dynamically map and unmap code/data and I'm not
>> including regions for device access or direct maps.  Thus, the current
>> #define for VM_MAX_KERNEL_ADDRESS on arm is really incorrect or at least
>> inconsistent with how this is defined on other architectures.  This
>> patch borrows from sparc64 in defining a global variable,
>> vm_max_kernel_address, rather than a constant #define to represent the
>> end of the kernel address space.
>>
>> Thanks,
>> Alan
>>
> Took me a while to get to this, here's the results...
>
> DreamPlug (armv5)
>
>     real memory  = 536870912 (512 MB)
>     avail memory = 516612096 (492 MB)
>
>     before: vm.max_kernel_address: 4294967295 0xFFFFFFFF
>     after:  vm.max_kernel_address: 4026531840 0xF0000000
>
> rpi (armv6)
>
>     real memory  = 536870912 (512 MB)
>     avail memory = 386789376 (368 MB)
>
>     before: vm.max_kernel_address: 4294967295 0xFFFFFFFF
>     after:  vm.max_kernel_address: 3741319168 0xDF000000
>
> Beaglebone (armv7)
>
>     real memory  = 268435456 (256 MB)
>     avail memory = 254808064 (243 MB)
>
>     before: vm.max_kernel_address: 4294967295 0xFFFFFFFF
>     after:  vm.max_kernel_address: 3741319168 0xDF000000
>
> It appears that the values that go into making the max kernel address on
> various arm platforms may be choosen somewhat arbitrarily.  Most SoCs
> map the on-chip register space "somewhere" way up near the end of the
> 32-bit virtual address space, and set the max kernel address to that
> point.  0xD0000000, 0xE0000000, and 0xF0000000 all seem to be popular
> choices.  I just changed the value for DreamPlug, which only needs 1MB
> of register mappings, from 0xF0000000 to 0xFE000000 and it seems to be
> working fine.
>
> So, given that the variable ending address may be arbitrary and not much
> different than the current 0xFFFFFFFF constant, I wonder if your changes
> are going to have the intended effect?

There are two parts to my change.  One is making VM_MAX_KERNEL_ADDRESS
accurately reflect the end of the kernel's address space by making it a
variable.  The other is placing a cap on the size of the kernel's heap,
i.e., the kmem submap.  Without that cap, machines with larger physical
memories won't boot.  In fact, I just saw an e-mail a few hours ago from
Oleksandr Tymoshenko saying that his 1GB Beaglebone won't boot anymore.

> ...  It may be that the important
> part of the change would come next: pick better arbitrary ending
> addresses.

Is the location of the register mapping something that the kernel
chooses?  Or is it something that is passed to the kernel at boot-time?

>   Or do what sparc seems to be doing, and choose a size then
> set the address accordingly.
>
> Speaking of what sparc is doing, this sparc code looks suspicious:
>
> 	virtsz = roundup(5 / 3 * physsz, PAGE_SIZE_4M <<
> 	    (PAGE_SHIFT - TTE_SHIFT));
>
> I wonder if the intent there was "3 * physsz / 5"?
>
> -- Ian
>
>
>




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?511FD88E.2020403>