From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 8 21:22:42 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 170C5106567D for ; Sun, 8 Jun 2008 21:22:42 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (smtp5-g19.free.fr [212.27.42.35]) by mx1.freebsd.org (Postfix) with ESMTP id 9E3B18FC12 for ; Sun, 8 Jun 2008 21:22:41 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (localhost.localdomain [127.0.0.1]) by smtp5-g19.free.fr (Postfix) with ESMTP id CB6D53F6393; Sun, 8 Jun 2008 23:22:39 +0200 (CEST) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp5-g19.free.fr (Postfix) with ESMTP id AA3913F634B; Sun, 8 Jun 2008 23:22:39 +0200 (CEST) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id 8FA929B497; Sun, 8 Jun 2008 21:18:51 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id 80EC04089; Sun, 8 Jun 2008 23:18:51 +0200 (CEST) Date: Sun, 8 Jun 2008 23:18:51 +0200 From: Jeremie Le Hen To: Kostik Belousov Message-ID: <20080608211851.GA52350@obiwan.tataz.chchile.org> References: <20080517091740.GI70896@obiwan.tataz.chchile.org> <20080517102653.GI18958@deviant.kiev.zoral.com.ua> <20080517173525.GJ70896@obiwan.tataz.chchile.org> <20080517174637.GL18958@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080517174637.GL18958@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.5.15 (2007-04-06) Cc: freebsd-hackers@freebsd.org Subject: Re: Debugging rtld X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2008 21:22:42 -0000 Hi Kostik, hackers, On Sat, May 17, 2008 at 08:46:37PM +0300, Kostik Belousov wrote: > On Sat, May 17, 2008 at 07:35:25PM +0200, Jeremie Le Hen wrote: > > Hi, > > > > On Sat, May 17, 2008 at 01:26:53PM +0300, Kostik Belousov wrote: > > > On Sat, May 17, 2008 at 11:17:40AM +0200, Jeremie Le Hen wrote: > > > > I tried to compile my source tree with -fstack-protector-all, and it > > > > happens that rtld breaks with this: once the new rtld is installed every > > > > single problem coredumps. I tried to compile rtld-elf without SSP, but > > > > it didn't solve the problem. Then I had to compile libc_pic.a without > > > > SSP and it worked, but I don't understand the root of the problem. > > > > So I want to use the generated coredump for post-mortem analysis with > > > > gdb. > > > > > > > > I compiled world with DEBUG_FLAGS=-g. But GDB gives me a backtrace so > > > > long that it can't be real. Moreoever it doesn't seem to bring in the > > > > required symbols. I'm a GDB novice, so I'd like some help. > > > > > > > > chroot> ===> libexec/rtld-elf (install) > > > > chroot> chflags noschg /usr/libexec/ld-elf.so.1 > > > > chroot> install -s -o root -g wheel -m 555 -C -b -fschg -S ld-elf.so.1 /libexec > > > > chroot> install -o root -g wheel -m 444 rtld.1.gz /usr/share/man/man1 > > > > chroot> *** Signal 11 > > > > chroot> > > > > chroot> jarjarbinks# cd /root; ls > > > > chroot> Segmentation fault > > > > > > > > host> jarjarbinks:145# ls -l /space/chroot/root/ls.core > > > > host> -rw------- 1 root wheel 184320 May 17 10:19 /space/chroot/root/ls.core > > > > host> jarjarbinks:149# gdb -c /space/chroot/root/ls.core -e /space/chroot/bin/ls > > > > host> GNU gdb 6.1.1 [FreeBSD] > > > > host> [...] > > > > host> This GDB was configured as "i386-marcel-freebsd". > > > > host> Core was generated by `ls'. > > > > host> Program terminated with signal 11, Segmentation fault. > > > > host> #0 0x280583e4 in ?? () > > > > host> (gdb) bt > > > > host> #0 0x280583e4 in ?? () > > > > host> #1 0x00000000 in ?? () > > > > host> #2 0x00000000 in ?? () > > > > host> #3 0x00000000 in ?? () > > > > host> #4 0x00000000 in ?? () > > > > host> #5 0x00000000 in ?? () > > > > host> #6 0x00000000 in ?? () > > > > host> #7 0x00000000 in ?? () > > > > host> #8 0x00000000 in ?? () > > > > host> #9 0x00000000 in ?? () > > > > host> #10 0x00000000 in ?? () > > > > host> #11 0xffffffff in ?? () > > > > host> #12 0x00001000 in ?? () > > > > host> [...] > > > > host> #359 0x73763a68 in ?? () > > > > host> #360 0x5b455c3d in ?? () > > > > host> [...] > > > > host> #855 0x00000000 in ?? () > > > > host> [...] > > > > > > > > Any hint on how to proceed would be welcome. > > > > > > I usually add the CFLAGS+=-g to the rtld-elf/Makefile. Also, you do not > > > need to bring down the whole host by the broken ld.so.1. Do not install > > > it at all, and specify the path to the rtld by the --dynamic-linker switch, > > > see into ld. > Hmm, ^^^^ info > > > > Thank you for this tip. However the backtrace is still unusable. > > I've recompiled libc_pic.a with -g, then rtld-elf with -g and finally > > /bin/ls with you tip and -g. > > > > I'm really brought to a standstill here. > > Looks like you have a stack corruption, that is reasonable given the > matters you touching. The easiest, althought somewhat time-consuming way > of searching the point where the things break is to insert some break > into the code of the rtld, "int3" may be good, and moving it forward > until you start hitting the breakage instead of the breakpoint. I've naively added asm("int3;"); to the rtld-elf source, but I definitely can't debug rtld as an usual program. According to the rtld source, I'd say GDB needs some cooperation from rtld itself. Chicken and egg problem here. I've tested various methods to achieve debugging, such as explicitely use the "add-symbol-file" GDB command to load ld-elf.so.1 at its entry point. Given that the backtrace is unusable, I've tried to dump the stack starting from $ebp, and I think I could identify one or two pointers into the text, but GDB says there is nothing to disassemble at those address. I don't have enough {linker,loader,gdb}-fu to go on this way. FWIW, I succeded to understand where is the segmentation fault by dichotomy placing "assert(NULL)" in the code. Interestingly, when libc is compiled *with* -fstack-protector-all (but sys/stack_protector.c), rtld is compiled *without* stack protection, the call to mmap(2) in libexec/rtld-elf/i386/reloc.c:reloc_non_plt() triggers a SIGSEGV. Any hint from a rtld guru would be welcome... Thanks. -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org >