Date: Thu, 12 Jan 2017 17:58:52 -0800 From: Mark Millard <markmi@dsl-only.net> To: Roman Divacky <rdivacky@vlakno.cz>, Ed Maste <emaste@freebsd.org> Cc: FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: Re: /usr/bin/ld.lld on powerpc64: produces a.out for which: ld-elf.so.1: assert failed: /usr/src/libexec/rtld-elf/powerpc64/reloc.c:374 Message-ID: <932E3C38-B226-4BF1-B587-5A2D5EA19300@dsl-only.net> In-Reply-To: <20170112192223.GA49469@vlakno.cz> References: <7139F615-8F18-4EDC-9051-5FFEC0C4057F@dsl-only.net> <CAPyFy2CSmNyUjQcXyq4qUWp_A=Qr81O7fpRbZ5GcfFHFhdSntw@mail.gmail.com> <CF2D86C8-8EEB-4536-8D83-6F8C676EFEF6@dsl-only.net> <20170111194844.GA16135@vlakno.cz> <8242A7B9-7ED3-4861-8209-F3728113D188@dsl-only.net> <20170111210658.GA20265@vlakno.cz> <CAPyFy2DG3ucUkxDCwRJ10a-nhC1=YvVrwR7v0dw6LJk=e61nvQ@mail.gmail.com> <EF97071B-AE4A-4520-A997-52249B8DAB5A@dsl-only.net> <20170112192223.GA49469@vlakno.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017-Jan-12, at 11:22 AM, Roman Divacky <rdivacky at vlakno.cz> =
wrote:
> Can you check if the TOC is correct? LLD assumes this:
>=20
> static uint64_t PPC64TocOffset =3D 0x8000;
>=20
> uint64_t getPPC64TocBase() {
> // The TOC consists of sections .got, .toc, .tocbss, .plt in that =
order. The
> // TOC starts where the first of these sections starts. We always =
create a
> // .got when we see a relocation that uses it, so for us the start is =
always
> // the .got.
> uint64_t TocVA =3D In<ELF64BE>::Got->getVA();
>=20
> // Per the ppc64-elf-linux ABI, The TOC base is TOC value plus 0x8000
> // thus permitting a full 64 Kbytes segment. Note that the glibc =
startup
> // code (crt1.o) assumes that you can get from the TOC base to the
> // start of the .toc section with only a single (signed) 16-bit =
relocation.
> return TocVA + PPC64TocOffset;
> }
[I warn that I'm outside familiar territory here.]
If I understand the 1st comment right the following does not look
like a match for -fuse-dl=3Dlld (readelf -a output):
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
. . .
[10] .rela.plt RELA 0000000010000420 00000420
0000000000000048 0000000000000018 A 5 0 8
. . .
[15] .plt PROGBITS 0000000010010560 00010560
0000000000000060 0000000000000000 AX 0 0 16
. . .
[20] .got PROGBITS 0000000010020138 00020138
0000000000000000 0000000000000000 WA 0 0 8
. . .
[22] .got.plt PROGBITS 0000000010030020 00030020
0000000000000030 0000000000000000 WA 0 0 8
. . .
[23] .toc PROGBITS 0000000010030050 00030050
0000000000000050 0000000000000000 WA 0 0 8
Possibly contributing reasons:
A) .got is not "first" of the 4 sections (by Address or by [Nr]).
(.got is listed as zero size as well)
B) There is no reference to .got.plt in the comment.
C) .got and .toc have .got.plt and other things between
-- and .got and .got.plt have stuff between.
D) There is no .tocbss at all (guess: optional so possibly okay).
E) .plt is before .got by address and by [Nr]
(it is als not next to .got or .got.plt or .toc).
F) There is no reference to .got.plt in the comment.
G) In general there are other things between the sections
making them spread over a wider address range.
[I guess that .rela.plt does not matter but I showed it
in case I'm wrong.]
Another potential issue is .plt being PROGBITS instead of
NOBITS (see below). Related is AX flags above vs. WA
flags below being a potential issue.
By contrast for -fuse-dl-bfd I see:
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
. . .
[ 8] .rela.plt RELA 0000000010000370 00000370
0000000000000048 0000000000000018 A 4 22 8
. . .
[21] .got PROGBITS 0000000010010c48 00000c48
0000000000000058 0000000000000008 WA 0 0 8
[22] .plt NOBITS 0000000010010ca0 00000ca0
0000000000000060 0000000000000018 WA 0 0 8
So no .toc or .tocbase sections.
But .got and .plt are next to each other with .got first
(by address and by [Nr]). This would fit the comments if
.toc and .tocbss are optional --and apparently they are.
So my guess is that -fuse-dl-bfd looks to be as expected,
unlike -fuse-dl=3Dlld .
> Perhaps thats not true on FreeBSD? Especially the hardcoded constant =
seems suspicious.
> When it comes to the actual PLT entry, there's this comment in the =
code:
>=20
> // FIXME: What we should do, in theory, is get the offset of the =
function
> // descriptor in the .opd section, and use that as the offset from =
%r2 (the
> // TOC-base pointer). Instead, we have the GOT-entry offset, and that =
will
> // be a pointer to the function descriptor in the .opd section. Using
> // this scheme is simpler, but requires an extra indirection per PLT =
dispatch.
>=20
> So I think that while it's different it might not be wrong. What might =
be wrong
> is the TOC entry (either it's content or it's position).
>=20
> I suspect there might be some Linux vs FreeBSD difference that =
prevents this from working.
>=20
> Roman
=3D=3D=3D
Mark Millard
markmi at dsl-only.net
On Thu, Jan 12, 2017 at 12:37:53AM -0800, Mark Millard wrote:
> On 2017-Jan-11, at 1:23 PM, Ed Maste <emaste at freebsd.org> wrote:
>=20
>> On 11 January 2017 at 21:06, Roman Divacky <rdivacky at vlakno.cz> =
wrote:
>>> Looks like a progress :) Three questions...
>>>=20
>>> Is the readelf -a reasonable now?
>>=20
>> FYI, I just committed an ELF Tool Chain fix (r311941) so readelf
>> should display the relocation types properly now.
>=20
> Thanks. I updated to -r311950 to pick this up.
>=20
>>> If you compile with -g, does the
>>> backtrace make a bit more sense? And finally, can you try to =
"nexti/stepi" in gdb from
>>> _start to see where things go wrong? Possibly doing it both for ld =
linked a.out
>>> and lld linked a.out and compare where things differ.
>=20
> I had compiled with -g. It never gets to main. . .
>=20
> # /usr/local/bin/gdb a.out
> . . .
> Reading symbols from a.out...done.
> (gdb) start
> Temporary breakpoint 1 at 0x1001045c: file main.c, line 3.
> Starting program: /root/c_tests/a.out=20
>=20
> Program received signal SIGSEGV, Segmentation fault.
> 0x000000001001056c in ?? ()
>=20
> Note that the temporary breakpoint is never hit.
>=20
> (gdb) bt
> #0 0x000000001001056c in ?? ()
> #1 0x00000000100100d8 in ?? ()
> #2 0x00000000500279b0 in ._rtld_start () at =
/usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> Backtrace stopped: frame did not save the PC
>=20
> (gdb) up 2
> #2 0x00000000500279b0 in ._rtld_start () at =
/usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> 104 blrl /* _start(argc, argv, envp, obj, cleanup, =
ps_strings) */
> (gdb) disass
> Dump of assembler code for function ._rtld_start:
> 0x0000000050027930 <+0>: stdu r1,-144(r1)
> 0x0000000050027934 <+4>: std r3,96(r1)
> 0x0000000050027938 <+8>: std r4,104(r1)
> 0x000000005002793c <+12>: std r5,112(r1)
> 0x0000000050027940 <+16>: std r8,136(r1)
> 0x0000000050027944 <+20>: bl 0x50027950 <._rtld_start+32>
> 0x0000000050027948 <+24>: .long 0x0
> 0x000000005002794c <+28>: .long 0x30e40
> 0x0000000050027950 <+32>: mflr r3
> 0x0000000050027954 <+36>: ld r4,0(r3)
> 0x0000000050027958 <+40>: add r3,r4,r3
> 0x000000005002795c <+44>: ld r4,-32768(r2)
> 0x0000000050027960 <+48>: subf r4,r4,r2
> 0x0000000050027964 <+52>: bl 0x50027c64 <reloc_non_plt_self>
> 0x0000000050027968 <+56>: nop
> 0x000000005002796c <+60>: ld r4,104(r1)
> 0x0000000050027970 <+64>: addi r3,r4,-8
> 0x0000000050027974 <+68>: addi r4,r1,128
> 0x0000000050027978 <+72>: addi r5,r1,120
> 0x000000005002797c <+76>: bl 0x50028608 <_rtld>
> 0x0000000050027980 <+80>: nop
> 0x0000000050027984 <+84>: ld r2,8(r3)
> 0x0000000050027988 <+88>: ld r11,16(r3)
> 0x000000005002798c <+92>: ld r3,0(r3)
> 0x0000000050027990 <+96>: mtlr r3
> 0x0000000050027994 <+100>: ld r3,96(r1)
> 0x0000000050027998 <+104>: ld r4,104(r1)
> 0x000000005002799c <+108>: ld r5,112(r1)
> 0x00000000500279a0 <+112>: ld r6,120(r1)
> 0x00000000500279a4 <+116>: ld r7,128(r1)
> 0x00000000500279a8 <+120>: ld r8,136(r1)
> 0x00000000500279ac <+124>: blrl
> =3D> 0x00000000500279b0 <+128>: li r0,1
> 0x00000000500279b4 <+132>: sc =20
> 0x00000000500279b8 <+136>: nop
> 0x00000000500279bc <+140>: nop
> End of assembler dump.
>=20
> So setting a breakpoint at 0x00000000500279ac and
> trying again:
>=20
> (gdb) run
> Starting program: /root/c_tests/a.out=20
>=20
> Breakpoint 3, ._rtld_start () at =
/usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> 104 blrl /* _start(argc, argv, envp, obj, cleanup, =
ps_strings) */
> (gdb) info registers
> r0 0x50027980 1342339456
> r1 0xffffffffffffdaf0 18446744073709542128
> r2 0x10028138 268599608
> r3 0x1 1
> r4 0xffffffffffffdbb8 18446744073709542328
> r5 0xffffffffffffdbc8 18446744073709542344
> r6 0x5004c000 1342488576
> r7 0x50058b30 1342540592
> r8 0x0 0
> r9 0x0 0
> r10 0x0 0
> r11 0x0 0
> r12 0x20000000 536870912
> r13 0x50057010 1342533648
> r14 0x0 0
> r15 0x0 0
> r16 0x0 0
> r17 0x0 0
> r18 0x0 0
> r19 0x0 0
> r20 0x0 0
> r21 0x0 0
> r22 0x0 0
> r23 0x0 0
> r24 0x0 0
> r25 0x0 0
> r26 0x0 0
> r27 0x0 0
> r28 0x0 0
> r29 0x0 0
> r30 0x0 0
> r31 0x0 0
> pc 0x500279ac 0x500279ac <._rtld_start+124>
> msr <unavailable>
> cr 0x22000c00 570428416
> lr 0x10010000 0x10010000
> ctr 0x50043a80 1342454400
> xer 0x20000000 536870912
> (gdb) stepi
> 0x0000000010010000 in ?? ()
>=20
> and that is effectively at ._start .
>=20
> NOTE: There is no ._start name in the disassembly
> listed by objdump.
>=20
> By contrast for -fuse-ld=3Dbfd building a.out objdump shows:
>=20
> 0000000010000438 <._start> mflr r0
> 000000001000043c <._start+0x4> mfcr r12
> 0000000010000440 <._start+0x8> std r31,-8(r1)
> 0000000010000444 <._start+0xc> std r0,16(r1)
> 0000000010000448 <._start+0x10> stw r12,8(r1)
> 000000001000044c <._start+0x14> stdu r1,-176(r1)
> . . .
>=20
>=20
> In gdb for ld.lld used:
>=20
> Reading symbols from a.out...done.
> (gdb) br *0x00000000500279ac
> Breakpoint 1 at 0x500279ac
> (gdb) run
> Starting program: /root/c_tests/a.out=20
>=20
> Breakpoint 1, ._rtld_start () at =
/usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> 104 blrl /* _start(argc, argv, envp, obj, cleanup, =
ps_strings) */
> (gdb) stepi
> 0x0000000010010000 in ?? ()
> (gdb)=20
> 0x0000000010010004 in ?? ()
> (gdb) display/i $pc
> 1: x/i $pc
> =3D> 0x10010004: mfcr r12
> (gdb) stepi
> 0x0000000010010008 in ?? ()
> 1: x/i $pc
> =3D> 0x10010008: std r31,-8(r1)
> (gdb)=20
> 0x000000001001000c in ?? ()
> 1: x/i $pc
> =3D> 0x1001000c: std r0,16(r1)
>=20
> . . .
>=20
> (gdb)=20
> 0x00000000100100a0 in ?? ()
> 1: x/i $pc
> =3D> 0x100100a0: beq 0x100100ac
> (gdb)=20
> 0x00000000100100ac in ?? ()
> 1: x/i $pc
> =3D> 0x100100ac: cmpldi r8,0
> (gdb)=20
> 0x00000000100100b0 in ?? ()
> 1: x/i $pc
> =3D> 0x100100b0: beq 0x100100c0
> (gdb)=20
> 0x00000000100100c0 in ?? ()
> 1: x/i $pc
> =3D> 0x100100c0: addis r3,r2,0
> (gdb)=20
> 0x00000000100100c4 in ?? ()
> 1: x/i $pc
> =3D> 0x100100c4: ld r3,32552(r3)
> (gdb)=20
> 0x00000000100100c8 in ?? ()
> 1: x/i $pc
> =3D> 0x100100c8: cmpldi r3,0
> (gdb)=20
> 0x00000000100100cc in ?? ()
> 1: x/i $pc
> =3D> 0x100100cc: beq 0x100100e0
> (gdb)=20
> 0x00000000100100d0 in ?? ()
> 1: x/i $pc
> =3D> 0x100100d0: mr r3,r7
> (gdb)=20
> 0x00000000100100d4 in ?? ()
> 1: x/i $pc
> =3D> 0x100100d4: bl 0x10010560
>=20
> Note: Below is from plt :
>=20
> Disassembly of section .plt:
> 0000000010010560 <.plt> std r2,40(r1)
> 0000000010010564 <.plt+0x4> addis r11,r2,0
> 0000000010010568 <.plt+0x8> ld r12,32512(r11)
> 000000001001056c <.plt+0xc> ld r11,0(r12) <<<<<=3D=3D=3D=3D=3D =
Fails here.
> 0000000010010570 <.plt+0x10> mtctr r11
> 0000000010010574 <.plt+0x14> ld r2,8(r12)
> 0000000010010578 <.plt+0x18> ld r11,16(r12)
> 000000001001057c <.plt+0x1c> bctr
>=20
> (By setting breakpoints in the 3 such .plt code blocks:
> this is the first .plt code block executed and it fails.)
>=20
> The .plt is different from what ld.bfd generates:
> no __glink_PLTresolve or its use and the code does
> not appear strictly equivalent to me.
>=20
> Back to gdb based information:
>=20
> (gdb) info registers
> r0 0x500279b0 1342339504
> r1 0xffffffffffffda40 18446744073709541952
> r2 0x10028138 268599608
> r3 0x50058b30 1342540592
> r4 0x0 0
> r5 0xffffffffffffdbc8 18446744073709542344
> r6 0x5004c000 1342488576
> r7 0x50058b30 1342540592
> r8 0x0 0
> r9 0x0 0
> r10 0x0 0
> r11 0x0 0
> r12 0x22000c00 570428416
> r13 0x50057010 1342533648
> r14 0x0 0
> r15 0x0 0
> r16 0x0 0
> r17 0x0 0
> r18 0x0 0
> r19 0x0 0
> r20 0x0 0
> r21 0x0 0
> r22 0x0 0
> r23 0x0 0
> r24 0x0 0
> r25 0x10028138 268599608
> r26 0x0 0
> r27 0x0 0
> r28 0x1 1
> r29 0xffffffffffffdbb8 18446744073709542328
> r30 0xffffffffffffdbc8 18446744073709542344
> r31 0xffffffffffffda40 18446744073709541952
> pc 0x10010560 0x10010560
> msr <unavailable>
> cr 0x42000c00 1107299328
> lr 0x100100d8 0x100100d8
> ctr 0x50043a80 1342454400
> xer 0x20000000 536870912
>=20
> (gdb)=20
> 0x0000000010010560 in ?? ()
> 1: x/i $pc
> =3D> 0x10010560: std r2,40(r1)
> (gdb)=20
> 0x0000000010010564 in ?? ()
> 1: x/i $pc
> =3D> 0x10010564: addis r11,r2,0
> (gdb)=20
> 0x0000000010010568 in ?? ()
> 1: x/i $pc
> =3D> 0x10010568: ld r12,32512(r11)
> (gdb)=20
> 0x000000001001056c in ?? ()
> 1: x/i $pc
> =3D> 0x1001056c: ld r11,0(r12)
> (gdb)=20
>=20
> Program received signal SIGSEGV, Segmentation fault.
> 0x000000001001056c in ?? ()
> 1: x/i $pc
> =3D> 0x1001056c: ld r11,0(r12)
>=20
> The source code (from lib/csu/powerpc64/crt1.c ) is:
>=20
> void
> _start(int argc, char **argv, char **env,
> const struct Struct_Obj_Entry *obj __unused, void (*cleanup)(void),
> struct ps_strings *ps_strings)
> {
>=20
> handle_argv(argc, argv, env);
>=20
> if (ps_strings !=3D (struct ps_strings *)0)
> __ps_strings =3D ps_strings;
>=20
> if (&_DYNAMIC !=3D NULL)
> atexit(cleanup);
> else
> _init_tls();
>=20
> #ifdef GCRT
> atexit(_mcleanup);
> monstartup(&eprol, &etext);
> #endif
>=20
> handle_static_init(argc, argv, env);
> exit(main(argc, argv, env));
> }
>=20
> The 3 plt code blocks are for:
>=20
> atexit
> _init_tls
> exit
>=20
> from what I can tell, possibly not in that order.
>=20
> Overall: The plt handling seems to be broken.
>=20
>=20
>> You can also build rtld with additional debugging by adding -DDEBUG =
to
>> CFLAGS. In libexec/rtld-elf/Makefile there's an example command line
>> for building it locally, but I've just added CFLAGS+=3D-DDEBUG to the
>> Makefile in my test tree and built it along with the rest of my full
>> cross build.
>=20
> # svnlite diff /usr/src/libexec/rtld-elf/Makefile
> Index: /usr/src/libexec/rtld-elf/Makefile
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- /usr/src/libexec/rtld-elf/Makefile (revision 311950)
> +++ /usr/src/libexec/rtld-elf/Makefile (working copy)
> @@ -17,6 +17,7 @@
> malloc.c xmalloc.c debug.c libmap.c
> MAN=3D rtld.1
> CSTD?=3D gnu99
> +CFLAGS+=3D-DDEBUG
> CFLAGS+=3D -Wall -DFREEBSD_ELF -DIN_RTLD -ffreestanding
> CFLAGS+=3D -I${SRCTOP}/lib/csu/common
> .if exists(${.CURDIR}/${MACHINE_ARCH})
>=20
> The above did not seem to make much of a difference for the
> code involved, likely because crt1.c is from
> lib/csu/powerpc64/ instead of from libexec/rtld-elf/ .
>=20
>=20
> =3D=3D=3D
> Mark Millard
> markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?932E3C38-B226-4BF1-B587-5A2D5EA19300>