From owner-svn-src-all@FreeBSD.ORG Mon Jan 7 18:47:38 2013 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BA784CD6; Mon, 7 Jan 2013 18:47:38 +0000 (UTC) (envelope-from gonzo@id.bluezbox.com) Received: from id.bluezbox.com (id.bluezbox.com [88.198.91.248]) by mx1.freebsd.org (Postfix) with ESMTP id 40C02CF1; Mon, 7 Jan 2013 18:47:37 +0000 (UTC) Received: from [88.198.91.248] (helo=[IPv6:::1]) by id.bluezbox.com with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1TsHjS-000Ozc-3X; Mon, 07 Jan 2013 10:47:36 -0800 Message-ID: <50EB1841.5030006@bluezbox.com> Date: Mon, 07 Jan 2013 10:47:29 -0800 From: Oleksandr Tymoshenko User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Alan Cox Subject: Re: svn commit: r243631 - in head/sys: kern sys References: <201211272119.qARLJxXV061083@svn.freebsd.org> <50C1BC90.90106@freebsd.org> <50C25A27.4060007@bluezbox.com> <50C26331.6030504@freebsd.org> <50C26AE9.4020600@bluezbox.com> <50C3A3D3.9000804@freebsd.org> <50C3AF72.4010902@rice.edu> <330405A1-312A-45A5-BB86-4969478D8BBD@bluezbox.com> <50D03E83.8060908@rice.edu> <50DD081E.8000409@bluezbox.com> In-Reply-To: <50DD081E.8000409@bluezbox.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: gonzo@id.bluezbox.com X-Spam-Level: -- X-Spam-Report: Spam detection software, running on the system "id.bluezbox.com", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: On 12/27/2012 6:46 PM, Oleksandr Tymoshenko wrote: > On 12/18/2012 1:59 AM, Alan Cox wrote: >> On 12/17/2012 23:40, Oleksandr Tymoshenko wrote: >>> On 2012-12-08, at 1:21 PM, Alan Cox wrote: >>> >>>> On 12/08/2012 14:32, Andre Oppermann wrote: >>> .. skipped .. >>> >>>>> The trouble seems to come from NSFBUFS which is (512 + maxusers * 16) >>>>> resulting in a kernel map of (512 + 400 * 16) * PAGE_SIZE = 27MB. >>>>> This >>>>> seem to be pushing it with the smaller ARM kmap layout. >>>>> >>>>> Does it boot and run when you set the tunable kern.ipc.nsfbufs=3500? >>>>> >>>>> ARM does have a direct map mode as well which doesn't require the >>>>> allocation >>>>> of sfbufs. I'm not sure which other problems that approach has. >>>>> >>>> Only a few (3?) platforms use it. It reduces the size of the user >>>> address space, and translation between physical addresses and >>>> direct map >>>> addresses is not computationally trivial as it is on other >>>> architectures, e.g., amd64, ia64. However, it does try to use large >>>> page mappings. >>>> >>>> >>>>> Hopefully alc@ (added to cc) can answer that and also why the kmap of >>>>> 27MB >>>>> manages to wrench the ARM kernel. >>>>> >>>> Arm does not define caps on either the buffer map size (param.h) or >>>> the >>>> kmem map size (vmparam.h). It would probably make sense to copy these >>>> definitions from i386. >>> Adding caps didn't help. I did some digging and found out that >>> although address range >>> 0xc0000000 .. 0xffffffff is indeed valid for ARM in general actual >>> KVA space varies for >>> each specific hardware platform. This "real" KVA is defined by >>> >>> pair and ifI use them instead of >> VM_MAX_KERNEL_ADDRESS> >>> in init_param2 function my pandaboard successfully boots. Since >>> former pair is used for defining >>> kernel_map boundaries I believe it should be used for auto tuning as >>> well. >> >> That makes sense. However, "virtual_avail" isn't the start of the >> kernel address [...] Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, Andre Oppermann X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2013 18:47:38 -0000 On 12/27/2012 6:46 PM, Oleksandr Tymoshenko wrote: > On 12/18/2012 1:59 AM, Alan Cox wrote: >> On 12/17/2012 23:40, Oleksandr Tymoshenko wrote: >>> On 2012-12-08, at 1:21 PM, Alan Cox wrote: >>> >>>> On 12/08/2012 14:32, Andre Oppermann wrote: >>> .. skipped .. >>> >>>>> The trouble seems to come from NSFBUFS which is (512 + maxusers * 16) >>>>> resulting in a kernel map of (512 + 400 * 16) * PAGE_SIZE = 27MB. >>>>> This >>>>> seem to be pushing it with the smaller ARM kmap layout. >>>>> >>>>> Does it boot and run when you set the tunable kern.ipc.nsfbufs=3500? >>>>> >>>>> ARM does have a direct map mode as well which doesn't require the >>>>> allocation >>>>> of sfbufs. I'm not sure which other problems that approach has. >>>>> >>>> Only a few (3?) platforms use it. It reduces the size of the user >>>> address space, and translation between physical addresses and >>>> direct map >>>> addresses is not computationally trivial as it is on other >>>> architectures, e.g., amd64, ia64. However, it does try to use large >>>> page mappings. >>>> >>>> >>>>> Hopefully alc@ (added to cc) can answer that and also why the kmap of >>>>> 27MB >>>>> manages to wrench the ARM kernel. >>>>> >>>> Arm does not define caps on either the buffer map size (param.h) or >>>> the >>>> kmem map size (vmparam.h). It would probably make sense to copy these >>>> definitions from i386. >>> Adding caps didn't help. I did some digging and found out that >>> although address range >>> 0xc0000000 .. 0xffffffff is indeed valid for ARM in general actual >>> KVA space varies for >>> each specific hardware platform. This "real" KVA is defined by >>> >>> pair and ifI use them instead of >> VM_MAX_KERNEL_ADDRESS> >>> in init_param2 function my pandaboard successfully boots. Since >>> former pair is used for defining >>> kernel_map boundaries I believe it should be used for auto tuning as >>> well. >> >> That makes sense. However, "virtual_avail" isn't the start of the >> kernel address space. The kernel map always starts at >> VM_MIN_KERNEL_ADDRESS. (See kmem_init().) "virtual_avail" represents >> the next unallocated virtual address in the kernel address space at an >> early point in initialization. "virtual_avail" and "virtual_end" aren't >> used after that, or outside the VM system. Please use >> vm_map_min(kernel_map) and vm_map_max(kernel_map) instead. > > I checked: kernel_map is not available (NULL) at this point. So we > can't use it to > determine real KVA size. Closest thing we can get is > virtual_avail/virtual_end pair. > > Andre, could you approve attached patch for commit or suggest better > solution? Any update on this one? Can I proceed with commit?