From owner-freebsd-current@FreeBSD.ORG Thu Jan 31 17:13:43 2013 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 14175424 for ; Thu, 31 Jan 2013 17:13:43 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8504063F for ; Thu, 31 Jan 2013 17:13:42 +0000 (UTC) Received: (qmail 36811 invoked from network); 31 Jan 2013 18:33:30 -0000 Received: from unknown (HELO [62.48.0.94]) ([62.48.0.94]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 31 Jan 2013 18:33:30 -0000 Message-ID: <510AA63D.2080301@freebsd.org> Date: Thu, 31 Jan 2013 18:13:33 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Ian Lepore Subject: Re: Trouble with recent auto-tuning changes References: <1359310302.93359.48.camel@revolution.hippie.lan> <1359382968.93359.66.camel@revolution.hippie.lan> <5106CF85.8060004@rice.edu> In-Reply-To: <5106CF85.8060004@rice.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: alc@FreeBSD.org, freebsd-current@FreeBSD.org, Alan Cox X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jan 2013 17:13:43 -0000 On 28.01.2013 20:20, Alan Cox wrote: > On 01/28/2013 08:22, Ian Lepore wrote: >> On Mon, 2013-01-28 at 00:09 -0600, Alan Cox wrote: >>> On Sun, Jan 27, 2013 at 12:11 PM, Ian Lepore wrote: >>> >>>> I ran into a panic while attempting to un-tar a large file on a >>>> DreamPlug (arm-based system) running -current. The source and dest of >>>> the un-tar is the root filesystem on sdcard, and I get this: >>>> >>>> panic: kmem_malloc(4096): kmem_map too small: 12582912 total allocated >>>> >>>> Just before the panic I see the tar process get hung in a "nokva" wait. >>>> 12582912 is the value of VM_KMEM_SIZE from arm/include/vmparam.h. >>>> >>>> In r245575 the init order for mbuf limits was changed from >>>> SI_SUB_TUNABLES to SI_SUB_KMEM so that mbuf limits could be based on the >>>> results of sizing kernel memory. Unfortunately, the process of sizing >>>> kernel memory relies on the mbuf limits; in kmeminit(): >>>> >>>> vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; >>>> >>>> Since r245575, nmbclusters is zero when this line of code runs. If I >>>> manually plugin "32768" (the number tunable_mbinit() comes up with for >>>> this platform) in that line, the panic stops happening. >>>> >>>> So we've got two problems here... one is the circular dependency in >>>> calculating the mbuf limits. The other is the fact that some >>>> non-trivial amount of kernel memory we're allowing for mbufs is actually >>>> being used for other things. That is, if my system was actually using >>>> all the mbufs that tunable_mbinit() allowed for, then this panic while >>>> untarring a huge file would still have happened. >>>> >>>> >>> All of this is factually correct. However, it's a red herring. The real >>> problem is that arm, unlike every other architecture in the tree, does not >>> enable auto-sizing of the kmem map based on the physical memory size. >>> Specifically, you'll find VM_KMEM_SIZE_SCALE defined in >>> "arch"/include/vmparam.h on every other architecture, just not on arm. >>> This auto-sizing overrides the value of VM_KMEM_SIZE. >>> >> Aha. I'll investigate what other architectures do with that and try to >> get the same thing going for arm. >> > > i386 or (32-bit) MIPS would be the most similar. Also, I would > encourage you to look for other definitions that those architectures > have that arm doesn't. As physical memory sizes continue to grow on > arm-based systems, they may require other changes in vmparam.h and the > machine-dependent param.h that were made on those other architectures > year ago. Ian, The patch below should do the trick. Can you please test? > If you have any questions about any of the definitions, feel free to > e-mail me. > > Alan > > P.S. When I get a little more free time, I intend to get in touch with > Andre to address the apparent circular dependency. For now just know > that unless that circular dependency is combined with a lack of kmem map > auto-sizing, like arm, it's basically harmless. I'm working myself through it and will post a patch shortly that untangles a lot of the obscure VM initialization stuff and moves it into the modern SYSINIT world. -- Andre Index: arm/include/vmparam.h =================================================================== --- arm/include/vmparam.h (revision 246082) +++ arm/include/vmparam.h (working copy) @@ -134,13 +134,16 @@ #endif #define VM_MAX_KERNEL_ADDRESS 0xffffffff + /* * Virtual size (bytes) for various kernel submaps. */ - #ifndef VM_KMEM_SIZE -#define VM_KMEM_SIZE (12*1024*1024) +#define VM_KMEM_SIZE (12*1024*1024) #endif +#ifndef VM_KMEM_SIZE_SCALE +#define VM_KMEM_SIZE_SCALE (2) +#endif #define MAXTSIZ (16*1024*1024) #ifndef DFLDSIZ