Date: Thu, 27 Dec 2012 18:46:54 -0800 From: Oleksandr Tymoshenko <gonzo@bluezbox.com> To: Alan Cox <alc@rice.edu> Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, Andre Oppermann <andre@freebsd.org> Subject: Re: svn commit: r243631 - in head/sys: kern sys Message-ID: <50DD081E.8000409@bluezbox.com> In-Reply-To: <50D03E83.8060908@rice.edu> References: <201211272119.qARLJxXV061083@svn.freebsd.org> <ABB3E29B-91F3-4C25-8FAB-869BBD7459E1@bluezbox.com> <50C1BC90.90106@freebsd.org> <50C25A27.4060007@bluezbox.com> <50C26331.6030504@freebsd.org> <50C26AE9.4020600@bluezbox.com> <50C3A3D3.9000804@freebsd.org> <50C3AF72.4010902@rice.edu> <330405A1-312A-45A5-BB86-4969478D8BBD@bluezbox.com> <50D03E83.8060908@rice.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
--------------000507020005010409010409
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
On 12/18/2012 1:59 AM, Alan Cox wrote:
> On 12/17/2012 23:40, Oleksandr Tymoshenko wrote:
>> On 2012-12-08, at 1:21 PM, Alan Cox <alc@rice.edu> wrote:
>>
>>> On 12/08/2012 14:32, Andre Oppermann wrote:
>> .. skipped ..
>>
>>>> The trouble seems to come from NSFBUFS which is (512 + maxusers * 16)
>>>> resulting in a kernel map of (512 + 400 * 16) * PAGE_SIZE = 27MB. This
>>>> seem to be pushing it with the smaller ARM kmap layout.
>>>>
>>>> Does it boot and run when you set the tunable kern.ipc.nsfbufs=3500?
>>>>
>>>> ARM does have a direct map mode as well which doesn't require the
>>>> allocation
>>>> of sfbufs. I'm not sure which other problems that approach has.
>>>>
>>> Only a few (3?) platforms use it. It reduces the size of the user
>>> address space, and translation between physical addresses and direct map
>>> addresses is not computationally trivial as it is on other
>>> architectures, e.g., amd64, ia64. However, it does try to use large
>>> page mappings.
>>>
>>>
>>>> Hopefully alc@ (added to cc) can answer that and also why the kmap of
>>>> 27MB
>>>> manages to wrench the ARM kernel.
>>>>
>>> Arm does not define caps on either the buffer map size (param.h) or the
>>> kmem map size (vmparam.h). It would probably make sense to copy these
>>> definitions from i386.
>> Adding caps didn't help. I did some digging and found out that although address range
>> 0xc0000000 .. 0xffffffff is indeed valid for ARM in general actual KVA space varies for
>> each specific hardware platform. This "real" KVA is defined by <virtual_avail, virtual_end>
>> pair and ifI use them instead of <VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS>
>> in init_param2 function my pandaboard successfully boots. Since former pair is used for defining
>> kernel_map boundaries I believe it should be used for auto tuning as well.
>
> That makes sense. However, "virtual_avail" isn't the start of the
> kernel address space. The kernel map always starts at
> VM_MIN_KERNEL_ADDRESS. (See kmem_init().) "virtual_avail" represents
> the next unallocated virtual address in the kernel address space at an
> early point in initialization. "virtual_avail" and "virtual_end" aren't
> used after that, or outside the VM system. Please use
> vm_map_min(kernel_map) and vm_map_max(kernel_map) instead.
I checked: kernel_map is not available (NULL) at this point. So we
can't use it to
determine real KVA size. Closest thing we can get is
virtual_avail/virtual_end pair.
Andre, could you approve attached patch for commit or suggest better
solution?
>
> That said, I would still add caps on the buffer map and kmem map size.
> As memory sizes on arm systems grow, you'll eventually need them, just
> like we did on i386.
What's the names for #defines for these?
--------------000507020005010409010409
Content-Type: text/plain; charset=windows-1252;
name="autotune-kva-fix.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="autotune-kva-fix.diff"
Index: kern/subr_param.c
===================================================================
--- kern/subr_param.c (revision 244665)
+++ kern/subr_param.c (working copy)
@@ -333,7 +333,7 @@
* At most it can be 3/4 of available kernel memory.
*/
realmem = qmin((quad_t)physpages * PAGE_SIZE,
- VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS);
+ virtual_end - virtual_avail);
maxmbufmem = realmem / 2;
TUNABLE_QUAD_FETCH("kern.maxmbufmem", &maxmbufmem);
if (maxmbufmem > (realmem / 4) * 3)
@@ -347,8 +347,8 @@
TUNABLE_LONG_FETCH("kern.ipc.maxpipekva", &maxpipekva);
if (maxpipekva < 512 * 1024)
maxpipekva = 512 * 1024;
- if (maxpipekva > (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS) / 64)
- maxpipekva = (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS) /
+ if (maxpipekva > (virtual_end - virtual_avail) / 64)
+ maxpipekva = (virtual_end - virtual_avail) /
64;
}
--------------000507020005010409010409--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50DD081E.8000409>