From owner-svn-src-all@FreeBSD.ORG Mon Jun 3 17:44:52 2013 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 53903397 for ; Mon, 3 Jun 2013 17:44:52 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-vb0-x22c.google.com (mail-vb0-x22c.google.com [IPv6:2607:f8b0:400c:c02::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 0F8B91CC3 for ; Mon, 3 Jun 2013 17:44:51 +0000 (UTC) Received: by mail-vb0-f44.google.com with SMTP id i3so2879743vbh.3 for ; Mon, 03 Jun 2013 10:44:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MDhzXJw+ynQXmKfwjeVoDlsILxLv5zJ0pQ45ncPBn3c=; b=BVn017CY+hH1nALzw64dTlqin6W9JiiFGBnUUY7VYIlCSvwZSuh7CmzgOq0EWXtnKQ QKNTwmHmW1Njosdmnb5bwClMVv5FH6ripp2n2JG7keXUqp3QGhg7wo6GJs0z7lslEbxM 31y5PqJ9f1vAGe4+GLQzM1rIMQIWtBeyXmvH0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=MDhzXJw+ynQXmKfwjeVoDlsILxLv5zJ0pQ45ncPBn3c=; b=c2k1zH1NGMEn5jdeula6PVc+CT3hAHxuXXmHCO6u3HHUxhrnr/KE/obXeG+pVaPWnn g37QIr9Gg6aU2T2nqiROFtc8kwSoZa2yPeD8blCYofo0o8WT3hwPeirlSdlNIVlqrhPr n086ps4dtlmFdX3YH081nVv00Kd5YIdq9qf1QX7jWlLx+dF20My6mqxbl+sqogvDXWw+ Fx7dPjUvK4PhX/USjtzmbtrpviBBHgdrK+HhxcZJ0yLKE+jD5AFwrl2kffA0jG2kGr5V 2grohBxQQdcVJxMEGwp2WWqkOqg6ql6Ya7ayOQrdGHdX25i9hi5PKAvakSeo6mqjZFjf alEA== MIME-Version: 1.0 X-Received: by 10.220.48.201 with SMTP id s9mr2732244vcf.9.1370281491568; Mon, 03 Jun 2013 10:44:51 -0700 (PDT) Received: by 10.220.20.133 with HTTP; Mon, 3 Jun 2013 10:44:51 -0700 (PDT) In-Reply-To: References: <201305251859.r4PIxChc053341@svn.freebsd.org> <51AC9933.7050201@FreeBSD.org> Date: Mon, 3 Jun 2013 10:44:51 -0700 Message-ID: Subject: Re: svn commit: r250991 - in head: contrib/jemalloc/include/jemalloc include lib/libc/stdlib/jemalloc From: Peter Wemm To: Marcel Moolenaar Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQl4FAryTGmXrno88BkNDJoE47vflIaYjE6eT+KUxMTLaRbDY8iE+oHM7CHCowDr3ALIu32M Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, Marcel Moolenaar , Dimitry Andric , src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jun 2013 17:44:52 -0000 On Mon, Jun 3, 2013 at 7:50 AM, Marcel Moolenaar wrote: > > On Jun 3, 2013, at 6:25 AM, Dimitry Andric wrote: > >> On 2013-05-25 20:59, Marcel Moolenaar wrote: >>> Author: marcel >>> Date: Sat May 25 18:59:11 2013 >>> New Revision: 250991 >>> URL: http://svnweb.freebsd.org/changeset/base/250991 >>> >>> Log: >>> Make the malloc(3) family of functions weak and make their non-weak >>> implementations visible for use by applications. The functions $F that >>> are now weak symbols are: >>> allocm, calloc, dallocm, free, malloc, malloc_usable_size, >>> nallocm, posix_memalign, rallocm, realloc, sallocm >>> >>> The non-weak implementations of $F are exported as __$F. >> >> Hi Marcel, >> >> This commit seems to lead to various problems in ports, and possibly >> other software. For example, with Firefox I now always get a few >> messages like: >> >> firefox in free(): warning: malloc() has never been called >> >> Another example is the devel/talloc port, which now dies with: > > It looks like the python build is broken. The net effect is that _ctypes.so > has a strong definition of malloc, free, et al. This is not intentional, > but a side-effect of the what seems to be a bug in fficonfig.py.in (see > patch below). To elaborate: > > _ctypes.so incorporates the libffi functionality for what I presume is > the basis for Python bindings. libffi includes dlmalloc.c, an open source > allocator. dlmalloc.c is incuded by closures.c and closures.c defines > USE_DL_PREFIX. On top of that closures.c makes all allocator functions > static. This, by design there's no problem. In short: dlmalloc.c never > gets compiler separately/independently. > > The python build however compiles dlmalloc.c separately/independently. > As such, dlmalloc.c now defines and exports malloc et al and it also > get linked into _ctypes.so. > > Once _ctypes.so gets dynamically loaded, things start to break. > > Unfortunately, this means python needs to be recompiled from ports with > the following fix: > > Index: files/patch-Modules-_ctypes-libffi-fficonfig.py.in > =================================================================== > --- files/patch-Modules-_ctypes-libffi-fficonfig.py.in (revision 0) > +++ files/patch-Modules-_ctypes-libffi-fficonfig.py.in (working copy) > @@ -0,0 +1,10 @@ > +--- Modules/_ctypes/libffi/fficonfig.py.in.orig 2013-06-03 07:16:44.000000000 -0700 > ++++ Modules/_ctypes/libffi/fficonfig.py.in 2013-06-03 07:17:03.000000000 -0700 > +@@ -1,7 +1,6 @@ > + ffi_sources = """ > + src/prep_cif.c > + src/closures.c > +-src/dlmalloc.c > + """.split() > + > + ffi_platforms = { > > This has been tested with python-2.7.5. I can't say anything about > other versions. > > Do people concur that this is the right fix? Forgive me for asking dumb questions, but: * Does this fix the firefox, chromium etc problems? (were they linked to python?) * Even if the python code is wrong, is this one of those things that's going to keep biting us going forward? Interactions between dlopen/dlsym/rtld/library dependencies/embedded malloc replacements/etc has been accumulated a large number of casualties over the years. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV