From owner-freebsd-current@FreeBSD.ORG Thu Jun 20 18:05:53 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 96801508 for ; Thu, 20 Jun 2013 18:05:53 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pb0-x236.google.com (mail-pb0-x236.google.com [IPv6:2607:f8b0:400e:c01::236]) by mx1.freebsd.org (Postfix) with ESMTP id 71FAF19B1 for ; Thu, 20 Jun 2013 18:05:53 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id ro2so6500381pbb.13 for ; Thu, 20 Jun 2013 11:05:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:x-x-sender:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type:x-gm-message-state; bh=u1b/ZB3LKPj64lS/Jv3m1lZLv5F/0fk+SdRIL0juyb4=; b=AhW/y7v1CBUQdMJRSB+jz4UOo1z5qjENIodEM8YaME571FQ1SqxijcvGcjCUd106cX WplpqpBK4JfXLhvTCamczk4iMbXnsFnq6mWWxesLQzH0FSNrxWp9KCrXPkbrJ25bZemi KCSBAbWh9BS8nDur9uaCzQjWWPtpYIMVDpVpRpUtnajheutRv7AnJ15QVhTk5Q3b2RiX y+FvCqjBfy9GzMI9qZfY40Y+NNCzzHdLZ7ghM00bVKD77YBpC4oC/Ur8F9+B8gEp/Wx2 gPy+UZEF969Dy8mcPdCyxJjFdCYPS6Ql/1szO5VgHkI7HD/yzdMjlux00VATN+PM1mvA skNw== X-Received: by 10.66.118.104 with SMTP id kl8mr12317044pab.182.1371751553188; Thu, 20 Jun 2013 11:05:53 -0700 (PDT) Received: from rrcs-66-91-135-210.west.biz.rr.com (rrcs-66-91-135-210.west.biz.rr.com. [66.91.135.210]) by mx.google.com with ESMTPSA id vb8sm1171354pbc.11.2013.06.20.11.05.51 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 20 Jun 2013 11:05:52 -0700 (PDT) Date: Thu, 20 Jun 2013 07:58:34 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Zbyszek Bodek Subject: Re: Kernel build fails on ARM: Cannot fork: Cannot allocate memory In-Reply-To: <51C1F53B.2080502@semihalf.com> Message-ID: References: <51C1F53B.2080502@semihalf.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Gm-Message-State: ALoCoQmKTtV3PNnoUMO2uFAW+/6yUTxdNK+dSX4z6dZvWlT6WMXZHtMDMdO+x1UABdConv2SVQJs Cc: freebsd-arm@FreeBSD.org, freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 18:05:53 -0000 On Wed, 19 Jun 2013, Zbyszek Bodek wrote: > Hello, > > I've been trying to compile the kernel on my ARMv7 platform using the > sources from the current FreeBSD HEAD. > > make buildkernel <.....> -j5 > > 1/2 builds fails in the way described below: > -------------------------------------------------------------------------- > ing-include-dirs -fdiagnostics-show-option -nostdinc -I. > -I/root/src/freebsd-arm-superpages/sys > -I/root/src/freebsd-arm-superpages/sys/contrib/altq > -I/root/src/freebsd-arm-superpages/sys/contrib/libfdt -D_KERNEL > -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common > -finline-limit=8000 --param inline-unit-growth=100 --param > large-function-growth=1000 -mno-thumb-interwork -ffreestanding -Werror > /root/src/freebsd-arm-superpages/sys/ufs/ffs/ffs_snapshot.c > Cannot fork: Cannot allocate memory > *** [ffs_snapshot.o] Error code 2 > 1 error > *** [buildkernel] Error code 2 > 1 error > *** [buildkernel] Error code 2 > 1 error > 5487.888u 481.569s 7:35.65 1310.0% 1443+167k 1741+5388io 221pf+0w > -------------------------------------------------------------------------- > > The warning from std err is: > -------------------------------------------------------------------------- > vm_thread_new: kstack allocation failed > vm_thread_new: kstack allocation failed > -------------------------------------------------------------------------- > > I was trying to find out which commit is causing this (because I was > previously working on some older revision) and using bisect I got to: > > -------------------------------------------------------------------------- > Author: jeff > Date: Tue Jun 18 04:50:20 2013 +0000 > > Refine UMA bucket allocation to reduce space consumption and improve > performance. > > - Always free to the alloc bucket if there is space. This gives LIFO > allocation order to improve hot-cache performance. This also allows > for zones with a single bucket per-cpu rather than a pair if the > entire > working set fits in one bucket. > - Enable per-cpu caches of buckets. To prevent recursive bucket > allocation one bucket zone still has per-cpu caches disabled. > - Pick the initial bucket size based on a table driven maximum size > per-bucket rather than the number of items per-page. This gives > more sane initial sizes. > - Only grow the bucket size when we face contention on the zone > lock, this > causes bucket sizes to grow more slowly. > - Adjust the number of items per-bucket to account for the header > space. > This packs the buckets more efficiently per-page while making them > not quite powers of two. > - Eliminate the per-zone free bucket list. Always return buckets back > to the bucket zone. This ensures that as zones grow into larger > bucket sizes they eventually discard the smaller sizes. It persists > fewer buckets in the system. The locking is slightly trickier. > - Only switch buckets in zalloc, not zfree, this eliminates > pathological > cases where we ping-pong between two buckets. > - Ensure that the thread that fills a new bucket gets to allocate from > it to give a better upper bound on allocation time. > > Sponsored by: EMC / Isilon Storage Division > -------------------------------------------------------------------------- > > I checked this several times and this commits seems to be causing this. Can you tell me how many cores and how much memory you have? And paste the output of vmstat -z when you see this error. You can try changing bucket_select() at line 339 in uma_core.c to read: static int bucket_select(int size) { return (MAX(PAGE_SIZE / size, 1)); } This will approximate the old bucket sizing behavior. Thanks, Jeff > > Does anyone observe similar behavior or have a solution? > > Best regards > Zbyszek Bodek >