From owner-freebsd-current@FreeBSD.ORG Wed Jun 13 15:31:22 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 56BF6106564A; Wed, 13 Jun 2012 15:31:22 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by mx1.freebsd.org (Postfix) with ESMTP id 2B8308FC1A; Wed, 13 Jun 2012 15:31:22 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8A0FCB945; Wed, 13 Jun 2012 11:31:21 -0400 (EDT) To: jasone@freebsd.org From: John Baldwin Date: Wed, 13 Jun 2012 11:31:21 -0400 MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201206131131.21059.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 13 Jun 2012 11:31:21 -0400 (EDT) Cc: current@freebsd.org Subject: jemalloc() assumes DSS is aligned X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2012 15:31:22 -0000 I tracked down a weird bug at work on the older jemalloc in FreeBSD 8/9 that a co-worker tripped over. Specifically, if you build the program below and link it with gold, the program will have an _end symbol that is on an odd address (std::nothrow results in some single-byte symbol being added to the end of the BSS). This causes the first arena allocated by jemalloc to use an odd address, and the rbt_nil structures for that arena's embedded trees (like runs_avail) to be allocated on odd addresses. This interferes with the RB trees using the low bit to distinguish red vs black. Specifically, the program ends up setting the right node of rbt_nil to an incorrect pointer value (the low bit gets cleared) resulting in an eventual segfault. Looking at phkmalloc, it always applied round_page() to the results from sbrk(). I believe that for jemalloc only the very first allocation from the DSS needs to check for misalignment, and the patch below does fix the segfault on FreeBSD 8. I have a stab at porting the change to jemalloc 3.0.0 in HEAD, but I'm not sure if it is quite correct. Also, I only made the DSS align on the quantum boundary rather than a page boundary. BTW, I filed a bug with the binutils folks as I initially thought this was a gold bug. However, POSIX doesn't make any guarantees about the return value of sbrk(), so I think gold is not broken. Test program: #include #include void foo() { char *c = new(std::nothrow) char[10]; delete c; } int main() { printf("Hello world\n"); } Tested patch against FreeBSD 8: Index: malloc.c =================================================================== --- malloc.c (revision 225507) +++ malloc.c (working copy) @@ -5132,6 +5132,9 @@ MALLOC_OUT: #ifdef MALLOC_DSS malloc_mutex_init(&dss_mtx); dss_base = sbrk(0); + i = (uintptr_t)dss_base & QUANTUM_MASK; + if (i != 0) + dss_base = sbrk(QUANTUM - i); dss_prev = dss_base; dss_max = dss_base; extent_tree_szad_new(&dss_chunks_szad); Untested forward port to jemalloc 3.0.0: Index: chunk_dss.c =================================================================== --- chunk_dss.c (revision 235919) +++ chunk_dss.c (working copy) @@ -123,12 +123,16 @@ chunk_in_dss(void *chunk) bool chunk_dss_boot(void) { + uintptr_t off; cassert(config_dss); if (malloc_mutex_init(&dss_mtx)) return (true); dss_base = sbrk(0); + off = (uintptr_t)dss_base & QUANTUM_MASK; + if (off != 0) + dss_base = sbrk(QUANTUM - off); dss_prev = dss_base; dss_max = dss_base; binutils ld.gold PR: http://sourceware.org/bugzilla/show_bug.cgi?id=14149 -- John Baldwin