From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 01:43:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B547D888 for ; Wed, 6 Nov 2013 01:43:10 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5F71C2577 for ; Wed, 6 Nov 2013 01:43:10 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0LhOSG-1VzkYx0smu-00mbRr for ; Wed, 06 Nov 2013 02:43:03 +0100 Message-ID: <52799E85.4050002@gmx.com> Date: Wed, 06 Nov 2013 02:42:29 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: FreeBSD Hackers , jasone@freebsd.org Subject: alignment of thread-local storage Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:c2+iTcFPHKObqtyUri/PxnV9rFifdr8ObV5TVajoRFY6WRXXy/f UhvzAUktV+SIEn6RvdCYxPupA8rfuu2uSZ5OSKJZHccyOwRwux8CM7sTgfygdpz7fR/59oa Jgao/9Di+5ur2XPdnXd+mDMrpvTXxTyEXRiSDFJRZgb710i/o42lBkdQbxF+v6CZI6PIRvM NAFEj7Hlsun3hmC4g8Nuw== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 01:43:10 -0000 Starting with revision 191847 of Clang/LLVM, a bus error tends to happen in realloc() under special circumstances. To reproduce: (1) Compile the following program and link it with the cURL library (or see (3b)): #include #include int main(void) { getpwuid(0); } (2) Compile libc (I use -CURRENT), but when compiling jemalloc.c, specifically use Clang, revision >=191847, and use -march=prescott (or similar) and at least -O1. (3) Run the program with the libc just built. The program will hopefully stop with a bus error. (3b) If choosing not to link with any library, then run the program through gdb(1). The program will hopefully also hit the bus error. In other words: # echo 'CPUTYPE=prescott' >> /etc/make.conf # echo 'CFLAGS=-g -O1' >> /etc/make.conf # cd /usr/src/lib/libc && make # cd # cat > x.c <<-EOF #include #include int main(void) { getpwuid(0); } EOF # clang x.c -L /usr/local/lib -lcurl # env LD_LIBRARY_PATH=/usr/src/lib/libc ./a.out The last 2 command lines can also be: # clang x.c # env LD_LIBRARY_PATH=/usr/src/lib/libc gdb ./a.out (gdb) run The backtrace is: 0x281d4235 in __realloc (ptr=0x282db7d0, size=) at jemalloc_jemalloc.c:1249 1249 ta->allocated += usize; (gdb) bt #0 0x281d4235 in __realloc (ptr=0x282db7d0, size=) at jemalloc_jemalloc.c:1249 #1 0x2826c119 in yygrowstack (data=0x282f5144) at nsparser.c:411 #2 0x2826b7e6 in _nsyyparse () at nsparser.c:470 #3 0x28276d34 in nss_configure () at /usr/src/lib/libc/net/nsdispatch.c:372 #4 0x28276301 in _nsdispatch (retval=0xbfbfdbe4, disp_tab=0x282dafb4, database=0x282d4982 "passwd", method_name=0x282d49b1 "getpwuid_r", defaults=0x282da594) at /usr/src/lib/libc/net/nsdispatch.c:645 #5 0x28254e9d in getpwuid_r (uid=0, pwd=0x282f4f50, buffer=0x28c0c400 "", bufsize=1024, result=0xbfbfdbe4) at /usr/src/lib/libc/gen/getpwent.c:609 #6 0x28255208 in wrap_getpwuid_r (key=..., pwd=0x282f4f50, buffer=0x28c0c400 "", bufsize=1024, res=0xbfbfdbe4) at /usr/src/lib/libc/gen/getpwent.c:686 #7 0x28254fda in getpw (fn=0x282551b0 , key=...) at /usr/src/lib/libc/gen/getpwent.c:654 #8 0x282551a3 in getpwuid (uid=0) at /usr/src/lib/libc/gen/getpwent.c:714 #9 0x0804860a in main () The current understanding (sort of) of the problem is: - The __jemalloc_thread_allocated_tls variable is updated using a processor instruction that requires alignment (paddq), that is, as of Clang/LLVM r191847. The variable is defined as: __thread thread_allocated_t __attribute__((tls_model("initial-exec"))) thread_allocated_tls = {0,0}; - The variable turns out to be insufficiently aligned, having only 4-byte alignment. the thread_allocated_tls object *should* be 16-byte aligned is it? (if not, that's the bug; the generated code looks correct and good) in the IR we have @thread_allocated_tls = global ..., align 16 how is storage for TLS variables allocated in the first place? compilers like to pretend that they exist magically, but they do not yeah, Clang emits the TLS variable as a 16-byte aligned symbol zygoloid: how are TLS variable actually allocated though? it can't be done once at load time like for globals so I'm guessing it must be malloc()ed or something if so, suspect your malloc So, what could be the bottom line of this? That is, which one is WRONG -- FreeBSD or Clang (or both)?