From owner-freebsd-toolchain@freebsd.org Mon Jan 16 19:43:12 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D369CB3411 for ; Mon, 16 Jan 2017 19:43:12 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (mail.vlakno.cz [91.217.96.224]) by mx1.freebsd.org (Postfix) with ESMTP id C3DE91366; Mon, 16 Jan 2017 19:43:10 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: by vlakno.cz (Postfix, from userid 1002) id 2CB3C12CCC9; Mon, 16 Jan 2017 20:40:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=vlakno.cz; s=mail; t=1484595635; bh=Nk8NQg1fkT2R3U7i7ChgRdV2Q72Y9eXNOZya47k3a3A=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=Cs8jQX6WqfRy2vvZcuEr1JgMvAbRk0bBqs0rY4lNHLfUopOxWajgLeULyP3nd9KTV pVSrSbAWJFmMcmV72MWMa2/7yFRWxOqrSGBbeoaM+DNJLgKVkD+KQJ3HGOpi6ZgZT/ pwObkIVgwcbSDA9wGhf68emVXV+sppNcHuBMOgPk= Date: Mon, 16 Jan 2017 20:40:35 +0100 From: Roman Divacky To: Mark Millard Cc: Ed Maste , FreeBSD Toolchain Subject: Re: /usr/bin/ld.lld on powerpc64: produces a.out for which: ld-elf.so.1: assert failed: /usr/src/libexec/rtld-elf/powerpc64/reloc.c:374 Message-ID: <20170116194035.GA20175@vlakno.cz> References: <20170111194844.GA16135@vlakno.cz> <8242A7B9-7ED3-4861-8209-F3728113D188@dsl-only.net> <20170111210658.GA20265@vlakno.cz> <20170112192223.GA49469@vlakno.cz> <932E3C38-B226-4BF1-B587-5A2D5EA19300@dsl-only.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2017 19:43:12 -0000 Mark, I think the TOC (.got + .plt) has to be contiguous in memory. The on-disk layout is not that important. Can you check whats the difference of the in-memory TOC between lld and ld.bfd? Thanks, Roman On Fri, Jan 13, 2017 at 02:07:00PM -0800, Mark Millard wrote: > Just an FYI: > > elfdump -a (from -r311950) does not dump .plt or .got.plt or .toc : > > # elfdump -a a.out | egrep "(got|toc|plt|:$)" | more > elf header: > program header: > section header: > sh_name: .rela.plt > sh_name: .plt > sh_name: .got > sh_name: .got.plt > sh_name: .toc > interp: > symbol table (.dynsym): > relocation with addend (.rela.plt): > dynamic: > global offset table: > symbol table (.symtab): > > (The "global offset table" was empty but its title was listed.) > > === > Mark Millard > markmi at dsl-only.net > > On 2017-Jan-12, at 5:58 PM, Mark Millard wrote: > > On 2017-Jan-12, at 11:22 AM, Roman Divacky wrote: > > > Can you check if the TOC is correct? LLD assumes this: > > > > static uint64_t PPC64TocOffset = 0x8000; > > > > uint64_t getPPC64TocBase() { > > // The TOC consists of sections .got, .toc, .tocbss, .plt in that order. The > > // TOC starts where the first of these sections starts. We always create a > > // .got when we see a relocation that uses it, so for us the start is always > > // the .got. > > uint64_t TocVA = In::Got->getVA(); > > > > // Per the ppc64-elf-linux ABI, The TOC base is TOC value plus 0x8000 > > // thus permitting a full 64 Kbytes segment. Note that the glibc startup > > // code (crt1.o) assumes that you can get from the TOC base to the > > // start of the .toc section with only a single (signed) 16-bit relocation. > > return TocVA + PPC64TocOffset; > > } > > [I warn that I'm outside familiar territory here.] > > If I understand the 1st comment right the following does not look > like a match for -fuse-dl=lld (readelf -a output): > > Section Headers: > [Nr] Name Type Address Offset > Size EntSize Flags Link Info Align > [ 0] NULL 0000000000000000 00000000 > 0000000000000000 0000000000000000 0 0 0 > . . . > [10] .rela.plt RELA 0000000010000420 00000420 > 0000000000000048 0000000000000018 A 5 0 8 > . . . > [15] .plt PROGBITS 0000000010010560 00010560 > 0000000000000060 0000000000000000 AX 0 0 16 > . . . > [20] .got PROGBITS 0000000010020138 00020138 > 0000000000000000 0000000000000000 WA 0 0 8 > . . . > [22] .got.plt PROGBITS 0000000010030020 00030020 > 0000000000000030 0000000000000000 WA 0 0 8 > . . . > [23] .toc PROGBITS 0000000010030050 00030050 > 0000000000000050 0000000000000000 WA 0 0 8 > > Possibly contributing reasons: > > A) .got is not "first" of the 4 sections (by Address or by [Nr]). > (.got is listed as zero size as well) > B) There is no reference to .got.plt in the comment. > C) .got and .toc have .got.plt and other things between > -- and .got and .got.plt have stuff between. > D) There is no .tocbss at all (guess: optional so possibly okay). > E) .plt is before .got by address and by [Nr] > (it is als not next to .got or .got.plt or .toc). > F) There is no reference to .got.plt in the comment. > G) In general there are other things between the sections > making them spread over a wider address range. > > [I guess that .rela.plt does not matter but I showed it > in case I'm wrong.] > > Another potential issue is .plt being PROGBITS instead of > NOBITS (see below). Related is AX flags above vs. WA > flags below being a potential issue. > > > By contrast for -fuse-dl-bfd I see: > > Section Headers: > [Nr] Name Type Address Offset > Size EntSize Flags Link Info Align > [ 0] NULL 0000000000000000 00000000 > 0000000000000000 0000000000000000 0 0 0 > . . . > [ 8] .rela.plt RELA 0000000010000370 00000370 > 0000000000000048 0000000000000018 A 4 22 8 > . . . > [21] .got PROGBITS 0000000010010c48 00000c48 > 0000000000000058 0000000000000008 WA 0 0 8 > [22] .plt NOBITS 0000000010010ca0 00000ca0 > 0000000000000060 0000000000000018 WA 0 0 8 > > So no .toc or .tocbase sections. > > But .got and .plt are next to each other with .got first > (by address and by [Nr]). This would fit the comments if > .toc and .tocbss are optional --and apparently they are. > > So my guess is that -fuse-dl-bfd looks to be as expected, > unlike -fuse-dl=lld . > > > > Perhaps thats not true on FreeBSD? Especially the hardcoded constant seems suspicious. > > When it comes to the actual PLT entry, there's this comment in the code: > > > > // FIXME: What we should do, in theory, is get the offset of the function > > // descriptor in the .opd section, and use that as the offset from %r2 (the > > // TOC-base pointer). Instead, we have the GOT-entry offset, and that will > > // be a pointer to the function descriptor in the .opd section. Using > > // this scheme is simpler, but requires an extra indirection per PLT dispatch. > > > > So I think that while it's different it might not be wrong. What might be wrong > > is the TOC entry (either it's content or it's position). > > > > I suspect there might be some Linux vs FreeBSD difference that prevents this from working. > > > > Roman > > === > Mark Millard > markmi at dsl-only.net > > On Thu, Jan 12, 2017 at 12:37:53AM -0800, Mark Millard wrote: > > On 2017-Jan-11, at 1:23 PM, Ed Maste wrote: > > > >> On 11 January 2017 at 21:06, Roman Divacky wrote: > >>> Looks like a progress :) Three questions... > >>> > >>> Is the readelf -a reasonable now? > >> > >> FYI, I just committed an ELF Tool Chain fix (r311941) so readelf > >> should display the relocation types properly now. > > > > Thanks. I updated to -r311950 to pick this up. > > > >>> If you compile with -g, does the > >>> backtrace make a bit more sense? And finally, can you try to "nexti/stepi" in gdb from > >>> _start to see where things go wrong? Possibly doing it both for ld linked a.out > >>> and lld linked a.out and compare where things differ. > > > > I had compiled with -g. It never gets to main. . . > > > > # /usr/local/bin/gdb a.out > > . . . > > Reading symbols from a.out...done. > > (gdb) start > > Temporary breakpoint 1 at 0x1001045c: file main.c, line 3. > > Starting program: /root/c_tests/a.out > > > > Program received signal SIGSEGV, Segmentation fault. > > 0x000000001001056c in ?? () > > > > Note that the temporary breakpoint is never hit. > > > > (gdb) bt > > #0 0x000000001001056c in ?? () > > #1 0x00000000100100d8 in ?? () > > #2 0x00000000500279b0 in ._rtld_start () at /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104 > > Backtrace stopped: frame did not save the PC > > > > (gdb) up 2 > > #2 0x00000000500279b0 in ._rtld_start () at /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104 > > 104 blrl /* _start(argc, argv, envp, obj, cleanup, ps_strings) */ > > (gdb) disass > > Dump of assembler code for function ._rtld_start: > > 0x0000000050027930 <+0>: stdu r1,-144(r1) > > 0x0000000050027934 <+4>: std r3,96(r1) > > 0x0000000050027938 <+8>: std r4,104(r1) > > 0x000000005002793c <+12>: std r5,112(r1) > > 0x0000000050027940 <+16>: std r8,136(r1) > > 0x0000000050027944 <+20>: bl 0x50027950 <._rtld_start+32> > > 0x0000000050027948 <+24>: .long 0x0 > > 0x000000005002794c <+28>: .long 0x30e40 > > 0x0000000050027950 <+32>: mflr r3 > > 0x0000000050027954 <+36>: ld r4,0(r3) > > 0x0000000050027958 <+40>: add r3,r4,r3 > > 0x000000005002795c <+44>: ld r4,-32768(r2) > > 0x0000000050027960 <+48>: subf r4,r4,r2 > > 0x0000000050027964 <+52>: bl 0x50027c64 > > 0x0000000050027968 <+56>: nop > > 0x000000005002796c <+60>: ld r4,104(r1) > > 0x0000000050027970 <+64>: addi r3,r4,-8 > > 0x0000000050027974 <+68>: addi r4,r1,128 > > 0x0000000050027978 <+72>: addi r5,r1,120 > > 0x000000005002797c <+76>: bl 0x50028608 <_rtld> > > 0x0000000050027980 <+80>: nop > > 0x0000000050027984 <+84>: ld r2,8(r3) > > 0x0000000050027988 <+88>: ld r11,16(r3) > > 0x000000005002798c <+92>: ld r3,0(r3) > > 0x0000000050027990 <+96>: mtlr r3 > > 0x0000000050027994 <+100>: ld r3,96(r1) > > 0x0000000050027998 <+104>: ld r4,104(r1) > > 0x000000005002799c <+108>: ld r5,112(r1) > > 0x00000000500279a0 <+112>: ld r6,120(r1) > > 0x00000000500279a4 <+116>: ld r7,128(r1) > > 0x00000000500279a8 <+120>: ld r8,136(r1) > > 0x00000000500279ac <+124>: blrl > > => 0x00000000500279b0 <+128>: li r0,1 > > 0x00000000500279b4 <+132>: sc > > 0x00000000500279b8 <+136>: nop > > 0x00000000500279bc <+140>: nop > > End of assembler dump. > > > > So setting a breakpoint at 0x00000000500279ac and > > trying again: > > > > (gdb) run > > Starting program: /root/c_tests/a.out > > > > Breakpoint 3, ._rtld_start () at /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104 > > 104 blrl /* _start(argc, argv, envp, obj, cleanup, ps_strings) */ > > (gdb) info registers > > r0 0x50027980 1342339456 > > r1 0xffffffffffffdaf0 18446744073709542128 > > r2 0x10028138 268599608 > > r3 0x1 1 > > r4 0xffffffffffffdbb8 18446744073709542328 > > r5 0xffffffffffffdbc8 18446744073709542344 > > r6 0x5004c000 1342488576 > > r7 0x50058b30 1342540592 > > r8 0x0 0 > > r9 0x0 0 > > r10 0x0 0 > > r11 0x0 0 > > r12 0x20000000 536870912 > > r13 0x50057010 1342533648 > > r14 0x0 0 > > r15 0x0 0 > > r16 0x0 0 > > r17 0x0 0 > > r18 0x0 0 > > r19 0x0 0 > > r20 0x0 0 > > r21 0x0 0 > > r22 0x0 0 > > r23 0x0 0 > > r24 0x0 0 > > r25 0x0 0 > > r26 0x0 0 > > r27 0x0 0 > > r28 0x0 0 > > r29 0x0 0 > > r30 0x0 0 > > r31 0x0 0 > > pc 0x500279ac 0x500279ac <._rtld_start+124> > > msr > > cr 0x22000c00 570428416 > > lr 0x10010000 0x10010000 > > ctr 0x50043a80 1342454400 > > xer 0x20000000 536870912 > > (gdb) stepi > > 0x0000000010010000 in ?? () > > > > and that is effectively at ._start . > > > > NOTE: There is no ._start name in the disassembly > > listed by objdump. > > > > By contrast for -fuse-ld=bfd building a.out objdump shows: > > > > 0000000010000438 <._start> mflr r0 > > 000000001000043c <._start+0x4> mfcr r12 > > 0000000010000440 <._start+0x8> std r31,-8(r1) > > 0000000010000444 <._start+0xc> std r0,16(r1) > > 0000000010000448 <._start+0x10> stw r12,8(r1) > > 000000001000044c <._start+0x14> stdu r1,-176(r1) > > . . . > > > > > > In gdb for ld.lld used: > > > > Reading symbols from a.out...done. > > (gdb) br *0x00000000500279ac > > Breakpoint 1 at 0x500279ac > > (gdb) run > > Starting program: /root/c_tests/a.out > > > > Breakpoint 1, ._rtld_start () at /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104 > > 104 blrl /* _start(argc, argv, envp, obj, cleanup, ps_strings) */ > > (gdb) stepi > > 0x0000000010010000 in ?? () > > (gdb) > > 0x0000000010010004 in ?? () > > (gdb) display/i $pc > > 1: x/i $pc > > => 0x10010004: mfcr r12 > > (gdb) stepi > > 0x0000000010010008 in ?? () > > 1: x/i $pc > > => 0x10010008: std r31,-8(r1) > > (gdb) > > 0x000000001001000c in ?? () > > 1: x/i $pc > > => 0x1001000c: std r0,16(r1) > > > > . . . > > > > (gdb) > > 0x00000000100100a0 in ?? () > > 1: x/i $pc > > => 0x100100a0: beq 0x100100ac > > (gdb) > > 0x00000000100100ac in ?? () > > 1: x/i $pc > > => 0x100100ac: cmpldi r8,0 > > (gdb) > > 0x00000000100100b0 in ?? () > > 1: x/i $pc > > => 0x100100b0: beq 0x100100c0 > > (gdb) > > 0x00000000100100c0 in ?? () > > 1: x/i $pc > > => 0x100100c0: addis r3,r2,0 > > (gdb) > > 0x00000000100100c4 in ?? () > > 1: x/i $pc > > => 0x100100c4: ld r3,32552(r3) > > (gdb) > > 0x00000000100100c8 in ?? () > > 1: x/i $pc > > => 0x100100c8: cmpldi r3,0 > > (gdb) > > 0x00000000100100cc in ?? () > > 1: x/i $pc > > => 0x100100cc: beq 0x100100e0 > > (gdb) > > 0x00000000100100d0 in ?? () > > 1: x/i $pc > > => 0x100100d0: mr r3,r7 > > (gdb) > > 0x00000000100100d4 in ?? () > > 1: x/i $pc > > => 0x100100d4: bl 0x10010560 > > > > Note: Below is from plt : > > > > Disassembly of section .plt: > > 0000000010010560 <.plt> std r2,40(r1) > > 0000000010010564 <.plt+0x4> addis r11,r2,0 > > 0000000010010568 <.plt+0x8> ld r12,32512(r11) > > 000000001001056c <.plt+0xc> ld r11,0(r12) <<<<<===== Fails here. > > 0000000010010570 <.plt+0x10> mtctr r11 > > 0000000010010574 <.plt+0x14> ld r2,8(r12) > > 0000000010010578 <.plt+0x18> ld r11,16(r12) > > 000000001001057c <.plt+0x1c> bctr > > > > (By setting breakpoints in the 3 such .plt code blocks: > > this is the first .plt code block executed and it fails.) > > > > The .plt is different from what ld.bfd generates: > > no __glink_PLTresolve or its use and the code does > > not appear strictly equivalent to me. > > > > Back to gdb based information: > > > > (gdb) info registers > > r0 0x500279b0 1342339504 > > r1 0xffffffffffffda40 18446744073709541952 > > r2 0x10028138 268599608 > > r3 0x50058b30 1342540592 > > r4 0x0 0 > > r5 0xffffffffffffdbc8 18446744073709542344 > > r6 0x5004c000 1342488576 > > r7 0x50058b30 1342540592 > > r8 0x0 0 > > r9 0x0 0 > > r10 0x0 0 > > r11 0x0 0 > > r12 0x22000c00 570428416 > > r13 0x50057010 1342533648 > > r14 0x0 0 > > r15 0x0 0 > > r16 0x0 0 > > r17 0x0 0 > > r18 0x0 0 > > r19 0x0 0 > > r20 0x0 0 > > r21 0x0 0 > > r22 0x0 0 > > r23 0x0 0 > > r24 0x0 0 > > r25 0x10028138 268599608 > > r26 0x0 0 > > r27 0x0 0 > > r28 0x1 1 > > r29 0xffffffffffffdbb8 18446744073709542328 > > r30 0xffffffffffffdbc8 18446744073709542344 > > r31 0xffffffffffffda40 18446744073709541952 > > pc 0x10010560 0x10010560 > > msr > > cr 0x42000c00 1107299328 > > lr 0x100100d8 0x100100d8 > > ctr 0x50043a80 1342454400 > > xer 0x20000000 536870912 > > > > (gdb) > > 0x0000000010010560 in ?? () > > 1: x/i $pc > > => 0x10010560: std r2,40(r1) > > (gdb) > > 0x0000000010010564 in ?? () > > 1: x/i $pc > > => 0x10010564: addis r11,r2,0 > > (gdb) > > 0x0000000010010568 in ?? () > > 1: x/i $pc > > => 0x10010568: ld r12,32512(r11) > > (gdb) > > 0x000000001001056c in ?? () > > 1: x/i $pc > > => 0x1001056c: ld r11,0(r12) > > (gdb) > > > > Program received signal SIGSEGV, Segmentation fault. > > 0x000000001001056c in ?? () > > 1: x/i $pc > > => 0x1001056c: ld r11,0(r12) > > > > The source code (from lib/csu/powerpc64/crt1.c ) is: > > > > void > > _start(int argc, char **argv, char **env, > > const struct Struct_Obj_Entry *obj __unused, void (*cleanup)(void), > > struct ps_strings *ps_strings) > > { > > > > handle_argv(argc, argv, env); > > > > if (ps_strings != (struct ps_strings *)0) > > __ps_strings = ps_strings; > > > > if (&_DYNAMIC != NULL) > > atexit(cleanup); > > else > > _init_tls(); > > > > #ifdef GCRT > > atexit(_mcleanup); > > monstartup(&eprol, &etext); > > #endif > > > > handle_static_init(argc, argv, env); > > exit(main(argc, argv, env)); > > } > > > > The 3 plt code blocks are for: > > > > atexit > > _init_tls > > exit > > > > from what I can tell, possibly not in that order. > > > > Overall: The plt handling seems to be broken. > > > > > >> You can also build rtld with additional debugging by adding -DDEBUG to > >> CFLAGS. In libexec/rtld-elf/Makefile there's an example command line > >> for building it locally, but I've just added CFLAGS+=-DDEBUG to the > >> Makefile in my test tree and built it along with the rest of my full > >> cross build. > > > > # svnlite diff /usr/src/libexec/rtld-elf/Makefile > > Index: /usr/src/libexec/rtld-elf/Makefile > > =================================================================== > > --- /usr/src/libexec/rtld-elf/Makefile (revision 311950) > > +++ /usr/src/libexec/rtld-elf/Makefile (working copy) > > @@ -17,6 +17,7 @@ > > malloc.c xmalloc.c debug.c libmap.c > > MAN= rtld.1 > > CSTD?= gnu99 > > +CFLAGS+=-DDEBUG > > CFLAGS+= -Wall -DFREEBSD_ELF -DIN_RTLD -ffreestanding > > CFLAGS+= -I${SRCTOP}/lib/csu/common > > .if exists(${.CURDIR}/${MACHINE_ARCH}) > > > > The above did not seem to make much of a difference for the > > code involved, likely because crt1.c is from > > lib/csu/powerpc64/ instead of from libexec/rtld-elf/ . > > > > > > === > > Mark Millard > > markmi at dsl-only.net > > _______________________________________________ > freebsd-toolchain@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain > To unsubscribe, send any mail to "freebsd-toolchain-unsubscribe@freebsd.org"