From owner-freebsd-current@FreeBSD.ORG Fri Jun 21 18:50:21 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4143578F for ; Fri, 21 Jun 2013 18:50:21 +0000 (UTC) (envelope-from zbb@semihalf.com) Received: from mail-bk0-x231.google.com (mail-bk0-x231.google.com [IPv6:2a00:1450:4008:c01::231]) by mx1.freebsd.org (Postfix) with ESMTP id B39851FB4 for ; Fri, 21 Jun 2013 18:50:20 +0000 (UTC) Received: by mail-bk0-f49.google.com with SMTP id mz10so3496268bkb.36 for ; Fri, 21 Jun 2013 11:50:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:organization:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type:x-gm-message-state; bh=yvFjdlRzTLubNvlq/m2Yapvhhaxs0P7N4Ni78If8QBo=; b=NG/ieXEQjqkcV4CthorwwYBujCzmykqsX/ewn0bSPk5atxbBfikmk9kxcxZd+zsvgb ULGJx1/b4caZj5w/c+CEu0M8VMWyemqwiHgMyYcX6HoCMv/rfxtN4AxR+H6dIekpqCd6 8jQReghghQEeMIdPUkC3UVAOBJZ2+h1NysDmBGXOqngA9eNoytojX/RJU7sfJtyB7z+A a2zZlNd4pk+SNS+v9TFJ0obV9IvpQocyxq825J8zainRqKPtXQA9ff67YbCX78hM+Yvf RXbnPPaKXsjcVzZ3qV9+hYUo954OzJp++kZD+/b1VkesMJpSm6EtZyw4sXhid9b6qqIy q2HQ== X-Received: by 10.204.54.137 with SMTP id q9mr1935089bkg.80.1371840619574; Fri, 21 Jun 2013 11:50:19 -0700 (PDT) Received: from [10.0.2.117] (cardhu.semihalf.com. [213.17.239.108]) by mx.google.com with ESMTPSA id qw6sm2009033bkb.4.2013.06.21.11.50.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 21 Jun 2013 11:50:18 -0700 (PDT) Message-ID: <51C4A067.7010203@semihalf.com> Date: Fri, 21 Jun 2013 20:50:15 +0200 From: Zbyszek Bodek Organization: Semihalf User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130510 Thunderbird/17.0.6 MIME-Version: 1.0 To: Jeff Roberson Subject: Re: Kernel build fails on ARM: Cannot fork: Cannot allocate memory References: <51C1F53B.2080502@semihalf.com> In-Reply-To: Content-Type: multipart/mixed; boundary="------------040207010309000602070803" X-Gm-Message-State: ALoCoQnGTtWl+YM+EbJ4nPS0d3DiCk+C6L7pnngT7tNAf2tqBEZZgL5y9KYwS7Vc8U8mE7Awkh/4 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-arm@FreeBSD.org, freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 18:50:21 -0000 This is a multi-part message in MIME format. --------------040207010309000602070803 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 21.06.2013 01:56, Jeff Roberson wrote: > On Thu, 20 Jun 2013, Jeff Roberson wrote: > >> On Wed, 19 Jun 2013, Zbyszek Bodek wrote: >> >>> Hello, >>> >>> I've been trying to compile the kernel on my ARMv7 platform using the >>> sources from the current FreeBSD HEAD. >>> >>> make buildkernel <.....> -j5 >>> >>> 1/2 builds fails in the way described below: >>> -------------------------------------------------------------------------- >>> >>> ing-include-dirs -fdiagnostics-show-option -nostdinc -I. >>> -I/root/src/freebsd-arm-superpages/sys >>> -I/root/src/freebsd-arm-superpages/sys/contrib/altq >>> -I/root/src/freebsd-arm-superpages/sys/contrib/libfdt -D_KERNEL >>> -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common >>> -finline-limit=8000 --param inline-unit-growth=100 --param >>> large-function-growth=1000 -mno-thumb-interwork -ffreestanding -Werror >>> /root/src/freebsd-arm-superpages/sys/ufs/ffs/ffs_snapshot.c >>> Cannot fork: Cannot allocate memory >>> *** [ffs_snapshot.o] Error code 2 >>> 1 error >>> *** [buildkernel] Error code 2 >>> 1 error >>> *** [buildkernel] Error code 2 >>> 1 error >>> 5487.888u 481.569s 7:35.65 1310.0% 1443+167k 1741+5388io 221pf+0w >>> -------------------------------------------------------------------------- >>> >>> >>> The warning from std err is: >>> -------------------------------------------------------------------------- >>> >>> vm_thread_new: kstack allocation failed >>> vm_thread_new: kstack allocation failed >>> -------------------------------------------------------------------------- >>> >>> >>> I was trying to find out which commit is causing this (because I was >>> previously working on some older revision) and using bisect I got to: >>> >>> -------------------------------------------------------------------------- >>> >>> Author: jeff >>> Date: Tue Jun 18 04:50:20 2013 +0000 >>> >>> Refine UMA bucket allocation to reduce space consumption and improve >>> performance. >>> >>> - Always free to the alloc bucket if there is space. This gives >>> LIFO >>> allocation order to improve hot-cache performance. This also >>> allows >>> for zones with a single bucket per-cpu rather than a pair if the >>> entire >>> working set fits in one bucket. >>> - Enable per-cpu caches of buckets. To prevent recursive bucket >>> allocation one bucket zone still has per-cpu caches disabled. >>> - Pick the initial bucket size based on a table driven maximum size >>> per-bucket rather than the number of items per-page. This gives >>> more sane initial sizes. >>> - Only grow the bucket size when we face contention on the zone >>> lock, this >>> causes bucket sizes to grow more slowly. >>> - Adjust the number of items per-bucket to account for the header >>> space. >>> This packs the buckets more efficiently per-page while making them >>> not quite powers of two. >>> - Eliminate the per-zone free bucket list. Always return buckets >>> back >>> to the bucket zone. This ensures that as zones grow into larger >>> bucket sizes they eventually discard the smaller sizes. It >>> persists >>> fewer buckets in the system. The locking is slightly trickier. >>> - Only switch buckets in zalloc, not zfree, this eliminates >>> pathological >>> cases where we ping-pong between two buckets. >>> - Ensure that the thread that fills a new bucket gets to allocate >>> from >>> it to give a better upper bound on allocation time. >>> >>> Sponsored by: EMC / Isilon Storage Division >>> -------------------------------------------------------------------------- >>> >>> >>> I checked this several times and this commits seems to be causing this. >> >> Can you tell me how many cores and how much memory you have? And >> paste the output of vmstat -z when you see this error. >> >> You can try changing bucket_select() at line 339 in uma_core.c to read: >> >> static int >> bucket_select(int size) >> { >> return (MAX(PAGE_SIZE / size, 1)); >> } >> >> This will approximate the old bucket sizing behavior. > > Just to add some more information; On my machine with 16GB of ram the > handful of recent UMA commits save about 20MB of kmem on boot. There > are 30% fewer buckets allocated. And all of the malloc zones have > similar amounts of cached space. Actually the page size malloc bucket > is taking up much less space. > > I don't know if the problem is unique to arm but I have tested x86 > limited to 512MB of ram without trouble. I will need the stats I > mentioned before to understand what has happened. > Hello Jeff, Thank you for your interest in my problem. My system is a quad-core ARMv7 with 2048 MB of RAM on board. Please see attachment for the output from vmstat -z when the error occurs. Changing bucket_select() to static int bucket_select(int size) { return (MAX(PAGE_SIZE / size, 1)); } as you suggested helps for the problem. I've performed numerous attempts to build the kernel and none of them failed. Best regards Zbyszek Bodek --------------040207010309000602070803--