From owner-freebsd-toolchain@FreeBSD.ORG Tue Aug 23 21:35:04 2011 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AD571065675 for ; Tue, 23 Aug 2011 21:35:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9ED928FC17 for ; Tue, 23 Aug 2011 21:35:03 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:2cd0:314:6286:119c] (unknown [IPv6:2001:7b8:3a7:0:2cd0:314:6286:119c]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 837635C59; Tue, 23 Aug 2011 23:35:02 +0200 (CEST) Message-ID: <4E541D02.1040506@FreeBSD.org> Date: Tue, 23 Aug 2011 23:34:58 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Test Rat References: <86fwkvt9me.fsf@gmail.com> In-Reply-To: <86fwkvt9me.fsf@gmail.com> Content-Type: multipart/mixed; boundary="------------020601090600040307020106" Cc: freebsd-toolchain@freebsd.org Subject: Re: [clang] rtld-elf/rtld.c and stack traces in gdb(1) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Aug 2011 21:35:04 -0000 This is a multi-part message in MIME format. --------------020601090600040307020106 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 2011-08-21 11:08, Test Rat wrote: > I often get corrupted traces with clang world, the cause seems to be in rtld. ... > (gdb) bt > #0 0x00000008009455ac in ?? () > #1 0x0000000800944fa7 in ?? () After some digging, this turned out to be caused by the empty function r_debug_state() in libexec/rtld-elf/rtld.c. This function is just a necessary hook for gdb, but since it is completely empty, calls to it in the same compilation unit simply don't generate any code, even if the function is marked as __noinline. The attached patch fixes this, by marking the function __noinline, and inserting an empty asm statement, that pretends to clobber memory. It generates no extra code, and forces clang to emit calls to r_debug_state throughout rtld.c. It looks rather hackish, though. An alternative solution would be to move the r_debug_state() function to another .c file, which should work OK, until we eventually start using link time optimization... :) > And compiling rtld with clang + -O0 makes it crash. This is caused by yet another interesting problem, which is in the _rtld() function in rtld.c. It is run at the very beginning of rtld, when relocations have not yet been processed. This initial code must be very careful to *not* use any relocated symbols, or problems will arise. The early initialization goes like: ... /* Initialize and relocate ourselves. */ assert(aux_info[AT_BASE] != NULL); init_rtld((caddr_t) aux_info[AT_BASE]->a_un.a_ptr, aux_info); __progname = obj_rtld.path; argv0 = argv[0] != NULL ? argv[0] : "(null)"; environ = env; The init_rtld() function takes care of the initial relocations, after which 'global' symbols like __progname and environ can be used. However, at -O0, clang still reorders the retrieval of the __progname offset to just *before* the init_rtld() call, and assigns it afterwards: ... .LBB0_16: # %cond.end movq __progname@GOTPCREL(%rip), %rax <-- gets the offset here leaq -224(%rbp), %rsi .loc 1 329 5 # /usr/src/libexec/rtld-elf/rtld.c:329:5 movq -168(%rbp), %rcx movq 8(%rcx), %rdi movq %rax, -1504(%rbp) # 8-byte Spill <-- saves offset on stack callq init_rtld .loc 1 331 5 # /usr/src/libexec/rtld-elf/rtld.c:331:5 movq obj_rtld+24(%rip), %rax movq -1504(%rbp), %rcx # 8-byte Reload <-- loads offset from stack movq %rax, (%rcx) <-- stores value in __progname It's not clear to me why clang does this reordering even when optimization is off, but it is normally legal, and quite usual. However, in case of this early initialization, it is fatal, as __progname@GOTPCREL(%rip) will still be junk, or zero... With optimization, such reorderings are even more likely, but for some reason, we have always been lucky that it turned out OK. A possible solution would be to move the code after the init_rtld() call to another function, and call that, but this could also be defeated again by inlining. :( --------------020601090600040307020106 Content-Type: text/plain; name="fix-rtld-backtrace-2.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="fix-rtld-backtrace-2.diff" Index: libexec/rtld-elf/rtld.c =================================================================== --- libexec/rtld-elf/rtld.c (revision 225105) +++ libexec/rtld-elf/rtld.c (working copy) @@ -143,7 +143,7 @@ static void ld_utrace_log(int, void *, void *, siz static void rtld_fill_dl_phdr_info(const Obj_Entry *obj, struct dl_phdr_info *phdr_info); -void r_debug_state(struct r_debug *, struct link_map *); +void r_debug_state(struct r_debug *, struct link_map *) __noinline; /* * Data declarations. @@ -2780,6 +2780,14 @@ linkmap_delete(Obj_Entry *obj) void r_debug_state(struct r_debug* rd, struct link_map *m) { + /* + * The following is a hack to force the compiler to emit calls to + * this function, even when optimizing. If the function is empty, + * the compiler is not obliged to emit any code for calls to it, + * even when marked __noinline. However, gdb depends on those + * calls being made. + */ + __asm __volatile("" : : : "memory"); } /* --------------020601090600040307020106--