From owner-freebsd-current@FreeBSD.ORG Sun Jun 23 06:57:12 2013 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7D7BADB4; Sun, 23 Jun 2013 06:57:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id E73C511A0; Sun, 23 Jun 2013 06:57:11 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r5N6v7bb006214; Sun, 23 Jun 2013 09:57:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r5N6v7bb006214 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r5N6v6dI006213; Sun, 23 Jun 2013 09:57:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 23 Jun 2013 09:57:06 +0300 From: Konstantin Belousov To: Zbyszek Bodek Subject: Re: Kernel build fails on ARM: Cannot fork: Cannot allocate memory Message-ID: <20130623065706.GV91021@kib.kiev.ua> References: <51C1F53B.2080502@semihalf.com> <51C4A067.7010203@semihalf.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WZjgKYXV2jnEACro" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-arm@FreeBSD.org, freebsd-current@freebsd.org, Jeff Roberson X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jun 2013 06:57:12 -0000 --WZjgKYXV2jnEACro Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 22, 2013 at 04:37:24PM -1000, Jeff Roberson wrote: > On Fri, 21 Jun 2013, Zbyszek Bodek wrote: >=20 > > On 21.06.2013 01:56, Jeff Roberson wrote: > >> On Thu, 20 Jun 2013, Jeff Roberson wrote: > >> > >>> On Wed, 19 Jun 2013, Zbyszek Bodek wrote: > >>> > >>>> Hello, > >>>> > >>>> I've been trying to compile the kernel on my ARMv7 platform using the > >>>> sources from the current FreeBSD HEAD. > >>>> > >>>> make buildkernel <.....> -j5 > >>>> > >>>> 1/2 builds fails in the way described below: > >>>> --------------------------------------------------------------------= ------ > >>>> > >>>> ing-include-dirs -fdiagnostics-show-option -nostdinc -I. > >>>> -I/root/src/freebsd-arm-superpages/sys > >>>> -I/root/src/freebsd-arm-superpages/sys/contrib/altq > >>>> -I/root/src/freebsd-arm-superpages/sys/contrib/libfdt -D_KERNEL > >>>> -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common > >>>> -finline-limit=3D8000 --param inline-unit-growth=3D100 --param > >>>> large-function-growth=3D1000 -mno-thumb-interwork -ffreestanding -W= error > >>>> /root/src/freebsd-arm-superpages/sys/ufs/ffs/ffs_snapshot.c > >>>> Cannot fork: Cannot allocate memory > >>>> *** [ffs_snapshot.o] Error code 2 > >>>> 1 error > >>>> *** [buildkernel] Error code 2 > >>>> 1 error > >>>> *** [buildkernel] Error code 2 > >>>> 1 error > >>>> 5487.888u 481.569s 7:35.65 1310.0% 1443+167k 1741+5388io 221pf+= 0w > >>>> --------------------------------------------------------------------= ------ > >>>> > >>>> > >>>> The warning from std err is: > >>>> --------------------------------------------------------------------= ------ > >>>> > >>>> vm_thread_new: kstack allocation failed > >>>> vm_thread_new: kstack allocation failed > >>>> --------------------------------------------------------------------= ------ > >>>> > >>>> > >>>> I was trying to find out which commit is causing this (because I was > >>>> previously working on some older revision) and using bisect I got to: > >>>> > >>>> --------------------------------------------------------------------= ------ > >>>> > >>>> Author: jeff > >>>> Date: Tue Jun 18 04:50:20 2013 +0000 > >>>> > >>>> Refine UMA bucket allocation to reduce space consumption and impr= ove > >>>> performance. > >>>> > >>>> - Always free to the alloc bucket if there is space. This gives > >>>> LIFO > >>>> allocation order to improve hot-cache performance. This also > >>>> allows > >>>> for zones with a single bucket per-cpu rather than a pair if t= he > >>>> entire > >>>> working set fits in one bucket. > >>>> - Enable per-cpu caches of buckets. To prevent recursive bucket > >>>> allocation one bucket zone still has per-cpu caches disabled. > >>>> - Pick the initial bucket size based on a table driven maximum s= ize > >>>> per-bucket rather than the number of items per-page. This giv= es > >>>> more sane initial sizes. > >>>> - Only grow the bucket size when we face contention on the zone > >>>> lock, this > >>>> causes bucket sizes to grow more slowly. > >>>> - Adjust the number of items per-bucket to account for the header > >>>> space. > >>>> This packs the buckets more efficiently per-page while making = them > >>>> not quite powers of two. > >>>> - Eliminate the per-zone free bucket list. Always return buckets > >>>> back > >>>> to the bucket zone. This ensures that as zones grow into larg= er > >>>> bucket sizes they eventually discard the smaller sizes. It > >>>> persists > >>>> fewer buckets in the system. The locking is slightly trickier. > >>>> - Only switch buckets in zalloc, not zfree, this eliminates > >>>> pathological > >>>> cases where we ping-pong between two buckets. > >>>> - Ensure that the thread that fills a new bucket gets to allocate > >>>> from > >>>> it to give a better upper bound on allocation time. > >>>> > >>>> Sponsored by: EMC / Isilon Storage Division > >>>> --------------------------------------------------------------------= ------ > >>>> > >>>> > >>>> I checked this several times and this commits seems to be causing th= is. > >>> > >>> Can you tell me how many cores and how much memory you have? And > >>> paste the output of vmstat -z when you see this error. > >>> > >>> You can try changing bucket_select() at line 339 in uma_core.c to rea= d: > >>> > >>> static int > >>> bucket_select(int size) > >>> { > >>> return (MAX(PAGE_SIZE / size, 1)); > >>> } > >>> > >>> This will approximate the old bucket sizing behavior. > >> > >> Just to add some more information; On my machine with 16GB of ram the > >> handful of recent UMA commits save about 20MB of kmem on boot. There > >> are 30% fewer buckets allocated. And all of the malloc zones have > >> similar amounts of cached space. Actually the page size malloc bucket > >> is taking up much less space. > >> > >> I don't know if the problem is unique to arm but I have tested x86 > >> limited to 512MB of ram without trouble. I will need the stats I > >> mentioned before to understand what has happened. > >> > > > > Hello Jeff, > > > > Thank you for your interest in my problem. > > > > My system is a quad-core ARMv7 with 2048 MB of RAM on board. > > Please see attachment for the output from vmstat -z when the error occu= rs. > > > > Changing bucket_select() to > > > > static int > > bucket_select(int size) > > { > > return (MAX(PAGE_SIZE / size, 1)); > > } > > > > as you suggested helps for the problem. I've performed numerous attempts > > to build the kernel and none of them failed. > > >=20 > I don't really see a lot of wasted memory in the zones. There is=20 > certainly some. Can you give me sysctl vm from both a working and=20 > non-working kernel after the build is done or fails? Try this: http://people.freebsd.org/~kib/misc/arm_bcache.1.patch Please _do_ notify me whether it compiled and helped with your problem. --WZjgKYXV2jnEACro Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJRxpxBAAoJEJDCuSvBvK1BaV0P/0dcOgHNcxsL0rv7T2YJXi2Z 9ie3nlIHBQvpA4mfbC2KAGOTX//3CFX+ZHotdRCdypN768h6QuZXfPawohy4Qjem q0lajL5rtzDjBgR+zmhmtnMGkqsew0o3UENEBzsbRmspDQIEccoxxQgkbzX8BtsF hi7EPVsoKw4CNkP/4puQmmTmWFS63AxCLrDYeqyNp/FMJBsffq+GLmRT4VhAiREr 5QbWm4OiXgONg8SDkcp6akbsn8Tpybfszaoj6N3RsGq6tE3gWO4UFVLqT8Wmrwhm gb90Q/N48gFJHCSyde62u4TEfgVozcSa3G2y3Jw9tjSHPPSqICyrwT/azTYn8TgM ypwaZ1W0NAEchAh8IXcc7YVYMeSoMaG0yY53X3gbWZLHVjdgJdoP3dwpAE65meoj jWJW0CXAQKSqJFd5DeCOcqIh0dhMUfTIa18NrUIUi7Uwm3MStvGLwjbYFXZVkdQL tJIHaWTm+XRvdPMpyYji/FBuIQomkcPS9KMlCXypvYjlCNRhj6HzlXw7dDnfYs44 WOGY81btBHdfw2vMwapoRqa+EWmucO29V9+GBlUpSF48mQiLyk0a5XkEXIg+Xo/T dKGTpgDjjBh6J5mqBpQCLrXf5wfvYRflK2BSNY8vQnRuybbCgb9jK0YzDxcPZ0zu FMQZnqqgZhk6XUsYBIvy =ig/L -----END PGP SIGNATURE----- --WZjgKYXV2jnEACro--