Date: Fri, 01 Feb 2013 11:30:40 +0100 From: Andre Oppermann <andre@freebsd.org> To: Ian Lepore <ian@FreeBSD.org> Cc: alc@FreeBSD.org, freebsd-current@FreeBSD.org, Alan Cox <alc@rice.edu> Subject: Re: Trouble with recent auto-tuning changes Message-ID: <510B9950.9080308@freebsd.org> In-Reply-To: <1359671120.93359.343.camel@revolution.hippie.lan> References: <1359310302.93359.48.camel@revolution.hippie.lan> <CAJUyCcOGE-86Y_fMkFyDwORd%2Bve0mpwrePv5uSey0L-mfD-9bA@mail.gmail.com> <1359382968.93359.66.camel@revolution.hippie.lan> <5106CF85.8060004@rice.edu> <510AA63D.2080301@freebsd.org> <1359671120.93359.343.camel@revolution.hippie.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On 31.01.2013 23:25, Ian Lepore wrote: > On Thu, 2013-01-31 at 18:13 +0100, Andre Oppermann wrote: >> On 28.01.2013 20:20, Alan Cox wrote: >>> On 01/28/2013 08:22, Ian Lepore wrote: >>>> On Mon, 2013-01-28 at 00:09 -0600, Alan Cox wrote: >>>>> On Sun, Jan 27, 2013 at 12:11 PM, Ian Lepore <ian@freebsd.org> wrote: >>>>> >>>>>> I ran into a panic while attempting to un-tar a large file on a >>>>>> DreamPlug (arm-based system) running -current. The source and dest of >>>>>> the un-tar is the root filesystem on sdcard, and I get this: >>>>>> >>>>>> panic: kmem_malloc(4096): kmem_map too small: 12582912 total allocated >>>>>> >>>>>> Just before the panic I see the tar process get hung in a "nokva" wait. >>>>>> 12582912 is the value of VM_KMEM_SIZE from arm/include/vmparam.h. >>>>>> >>>>>> In r245575 the init order for mbuf limits was changed from >>>>>> SI_SUB_TUNABLES to SI_SUB_KMEM so that mbuf limits could be based on the >>>>>> results of sizing kernel memory. Unfortunately, the process of sizing >>>>>> kernel memory relies on the mbuf limits; in kmeminit(): >>>>>> >>>>>> vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE; >>>>>> >>>>>> Since r245575, nmbclusters is zero when this line of code runs. If I >>>>>> manually plugin "32768" (the number tunable_mbinit() comes up with for >>>>>> this platform) in that line, the panic stops happening. >>>>>> >>>>>> So we've got two problems here... one is the circular dependency in >>>>>> calculating the mbuf limits. The other is the fact that some >>>>>> non-trivial amount of kernel memory we're allowing for mbufs is actually >>>>>> being used for other things. That is, if my system was actually using >>>>>> all the mbufs that tunable_mbinit() allowed for, then this panic while >>>>>> untarring a huge file would still have happened. >>>>>> >>>>>> >>>>> All of this is factually correct. However, it's a red herring. The real >>>>> problem is that arm, unlike every other architecture in the tree, does not >>>>> enable auto-sizing of the kmem map based on the physical memory size. >>>>> Specifically, you'll find VM_KMEM_SIZE_SCALE defined in >>>>> "arch"/include/vmparam.h on every other architecture, just not on arm. >>>>> This auto-sizing overrides the value of VM_KMEM_SIZE. >>>>> >>>> Aha. I'll investigate what other architectures do with that and try to >>>> get the same thing going for arm. >>>> >>> >>> i386 or (32-bit) MIPS would be the most similar. Also, I would >>> encourage you to look for other definitions that those architectures >>> have that arm doesn't. As physical memory sizes continue to grow on >>> arm-based systems, they may require other changes in vmparam.h and the >>> machine-dependent param.h that were made on those other architectures >>> year ago. >> >> Ian, >> >> The patch below should do the trick. Can you please test? > > Yep, that fixed the problem with untarring the large file. Here are > some before/after numbers from sysctl, converted from bytes to KBytes > for readability: > > vm.kmem_size_scale: 0 2 > vm.kmem_map_free: 5740 246440 > vm.kmem_map_size: 6548 7176 > vm.kmem_size: 12288 253616 > > real memory = 536870912 (512 MB) > avail memory = 516718592 (492 MB) Thank you for testing. Committed as r246204. If any other problems come up please report. -- Andre
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?510B9950.9080308>