From owner-freebsd-current@FreeBSD.ORG Fri Apr 20 19:49:55 2012 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08F3E1065673; Fri, 20 Apr 2012 19:49:55 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id B76578FC0A; Fri, 20 Apr 2012 19:49:54 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:5452:d047:6eb8:2b2e] (unknown [IPv6:2001:7b8:3a7:0:5452:d047:6eb8:2b2e]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id EE3E55C37; Fri, 20 Apr 2012 21:49:52 +0200 (CEST) Message-ID: <4F91BDE1.4080802@FreeBSD.org> Date: Fri, 20 Apr 2012 21:49:53 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120410 Thunderbird/12.0 MIME-Version: 1.0 To: Jason Evans References: <20120420125718.GD1582@albert.catwhisker.org> <20120420165558.b51c8b66.misho@aitbg.com> In-Reply-To: <20120420165558.b51c8b66.misho@aitbg.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Michael Pounov , current@FreeBSD.org Subject: Re: /usr/bin/as: out of memory allocating 4194304 bytes after a total of 524288000 bytes X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Apr 2012 19:49:55 -0000 On 2012-04-20 15:55, Michael Pounov wrote: > On Fri, 20 Apr 2012 05:57:18 -0700 > David Wolfskill wrote: ... >> The update after 234416 was to 234454; the attempted buildworld failed: ... >> /usr/bin/as: out of memory allocating 4194304 bytes after a total of 524288000 bytes > yep, I sent PR for this issue;) > > http://www.freebsd.org/cgi/query-pr.cgi?pr=167064 The root cause of this is the new jemalloc import in r234370. In contrib/jemalloc/src/chunk.c, this defines a global variable called 'chunksize'. At run-time, this turns out to have a value of 4194304, at least on my i386 system. However, GNU as *also* has a global variable called 'chunksize', with a very different intent: it is the default chunk size for its so-called "obstacks", an internal implementation detail. It is set to zero in contrib/binutils/gas/as.c, but normally ends up as 4096 during further initialization. Now, because the variable from jemalloc ends up in libc.a, and /usr/bin/as is statically linked, the jemalloc variable overrides the one from GNU as. This causes as to allocate 4 MiB chunks instead of 4 KiB chunks for all its internal processing, and you can guess what happens with a reasonably large input file. :) I think the best solution would be for jemalloc to avoid using obvious names like "chunksize" for its globals, because it is basically a library that could be linked to any sort of program out there. For example, it could prefix all its internal-use only globals with "jemalloc_" or some other mangling scheme. Jason, any thoughts?