From owner-freebsd-current@FreeBSD.ORG Mon Jan 28 19:20:45 2013 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 62BFD526; Mon, 28 Jan 2013 19:20:45 +0000 (UTC) (envelope-from alc@rice.edu) Received: from mh1.mail.rice.edu (mh1.mail.rice.edu [128.42.201.20]) by mx1.freebsd.org (Postfix) with ESMTP id 2B41D67E; Mon, 28 Jan 2013 19:20:45 +0000 (UTC) Received: from mh1.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh1.mail.rice.edu (Postfix) with ESMTP id 8F76A4601E3; Mon, 28 Jan 2013 13:20:38 -0600 (CST) Received: from mh1.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh1.mail.rice.edu (Postfix) with ESMTP id 8DE88460172; Mon, 28 Jan 2013 13:20:38 -0600 (CST) X-Virus-Scanned: by amavis-2.7.0 at mh1.mail.rice.edu, auth channel Received: from mh1.mail.rice.edu ([127.0.0.1]) by mh1.mail.rice.edu (mh1.mail.rice.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id v4gHhvJd-nor; Mon, 28 Jan 2013 13:20:38 -0600 (CST) Received: from adsl-216-63-78-18.dsl.hstntx.swbell.net (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) (Authenticated sender: alc) by mh1.mail.rice.edu (Postfix) with ESMTPSA id 0BD684601D7; Mon, 28 Jan 2013 13:20:37 -0600 (CST) Message-ID: <5106CF85.8060004@rice.edu> Date: Mon, 28 Jan 2013 13:20:37 -0600 From: Alan Cox User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:17.0) Gecko/20130127 Thunderbird/17.0.2 MIME-Version: 1.0 To: Ian Lepore Subject: Re: Trouble with recent auto-tuning changes References: <1359310302.93359.48.camel@revolution.hippie.lan> <1359382968.93359.66.camel@revolution.hippie.lan> In-Reply-To: <1359382968.93359.66.camel@revolution.hippie.lan> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: alc@FreeBSD.org, freebsd-current@FreeBSD.org, Andre Oppermann X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2013 19:20:45 -0000 On 01/28/2013 08:22, Ian Lepore wrote: > On Mon, 2013-01-28 at 00:09 -0600, Alan Cox wrote: >> On Sun, Jan 27, 2013 at 12:11 PM, Ian Lepore wrote: >> >>> I ran into a panic while attempting to un-tar a large file on a >>> DreamPlug (arm-based system) running -current. The source and dest of >>> the un-tar is the root filesystem on sdcard, and I get this: >>> >>> panic: kmem_malloc(4096): kmem_map too small: 12582912 total allocated >>> >>> Just before the panic I see the tar process get hung in a "nokva" wait. >>> 12582912 is the value of VM_KMEM_SIZE from arm/include/vmparam.h. >>> >>> In r245575 the init order for mbuf limits was changed from >>> SI_SUB_TUNABLES to SI_SUB_KMEM so that mbuf limits could be based on the >>> results of sizing kernel memory. Unfortunately, the process of sizing >>> kernel memory relies on the mbuf limits; in kmeminit(): >>> >>> vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; >>> >>> Since r245575, nmbclusters is zero when this line of code runs. If I >>> manually plugin "32768" (the number tunable_mbinit() comes up with for >>> this platform) in that line, the panic stops happening. >>> >>> So we've got two problems here... one is the circular dependency in >>> calculating the mbuf limits. The other is the fact that some >>> non-trivial amount of kernel memory we're allowing for mbufs is actually >>> being used for other things. That is, if my system was actually using >>> all the mbufs that tunable_mbinit() allowed for, then this panic while >>> untarring a huge file would still have happened. >>> >>> >> All of this is factually correct. However, it's a red herring. The real >> problem is that arm, unlike every other architecture in the tree, does not >> enable auto-sizing of the kmem map based on the physical memory size. >> Specifically, you'll find VM_KMEM_SIZE_SCALE defined in >> "arch"/include/vmparam.h on every other architecture, just not on arm. >> This auto-sizing overrides the value of VM_KMEM_SIZE. >> > Aha. I'll investigate what other architectures do with that and try to > get the same thing going for arm. > i386 or (32-bit) MIPS would be the most similar. Also, I would encourage you to look for other definitions that those architectures have that arm doesn't. As physical memory sizes continue to grow on arm-based systems, they may require other changes in vmparam.h and the machine-dependent param.h that were made on those other architectures year ago. If you have any questions about any of the definitions, feel free to e-mail me. Alan P.S. When I get a little more free time, I intend to get in touch with Andre to address the apparent circular dependency. For now just know that unless that circular dependency is combined with a lack of kmem map auto-sizing, like arm, it's basically harmless.