From owner-freebsd-current@FreeBSD.ORG Fri Apr 20 19:54:25 2012 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5A329106568F; Fri, 20 Apr 2012 19:54:25 +0000 (UTC) (envelope-from jasone@freebsd.org) Received: from canonware.com (10140.x.rootbsd.net [204.109.63.53]) by mx1.freebsd.org (Postfix) with ESMTP id 343218FC12; Fri, 20 Apr 2012 19:54:25 +0000 (UTC) Received: from [172.25.16.174] (unknown [173.252.71.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by canonware.com (Postfix) with ESMTPSA id A80AB28419; Fri, 20 Apr 2012 12:54:18 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=iso-8859-1 From: Jason Evans In-Reply-To: <4F91BDE1.4080802@FreeBSD.org> Date: Fri, 20 Apr 2012 12:54:17 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120420125718.GD1582@albert.catwhisker.org> <20120420165558.b51c8b66.misho@aitbg.com> <4F91BDE1.4080802@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1257) Cc: Michael Pounov , current@FreeBSD.org Subject: Re: /usr/bin/as: out of memory allocating 4194304 bytes after a total of 524288000 bytes X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Apr 2012 19:54:25 -0000 On Apr 20, 2012, at 12:49 PM, Dimitry Andric wrote: > On 2012-04-20 15:55, Michael Pounov wrote: >> On Fri, 20 Apr 2012 05:57:18 -0700 >> David Wolfskill wrote: > ... >>> The update after 234416 was to 234454; the attempted buildworld = failed: > ... >>> /usr/bin/as: out of memory allocating 4194304 bytes after a total of = 524288000 bytes >> yep, I sent PR for this issue;) >>=20 >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D167064 >=20 > The root cause of this is the new jemalloc import in r234370. In > contrib/jemalloc/src/chunk.c, this defines a global variable called > 'chunksize'. At run-time, this turns out to have a value of 4194304, = at > least on my i386 system. >=20 > However, GNU as *also* has a global variable called 'chunksize', with = a > very different intent: it is the default chunk size for its so-called > "obstacks", an internal implementation detail. It is set to zero in > contrib/binutils/gas/as.c, but normally ends up as 4096 during further > initialization. >=20 > Now, because the variable from jemalloc ends up in libc.a, and > /usr/bin/as is statically linked, the jemalloc variable overrides the > one from GNU as. This causes as to allocate 4 MiB chunks instead of 4 > KiB chunks for all its internal processing, and you can guess what > happens with a reasonably large input file. :) >=20 > I think the best solution would be for jemalloc to avoid using obvious > names like "chunksize" for its globals, because it is basically a > library that could be linked to any sort of program out there. >=20 > For example, it could prefix all its internal-use only globals with > "jemalloc_" or some other mangling scheme. Jason, any thoughts? jemalloc has optional namespace mangling support built in for just this = reason. I'll turn it on, hopefully today. Thanks, Jason=