Date: Tue, 8 May 2018 11:14:18 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Phil Shafer <phil@juniper.net> Cc: freebsd-arch@freebsd.org Subject: Re: initialization problem w/ thread-specific .tbss data on i386 Message-ID: <20180508081417.GL6887@kib.kiev.ua> In-Reply-To: <201805072127.w47LR3W5060281@idle.juniper.net> References: <201805072127.w47LR3W5060281@idle.juniper.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, May 07, 2018 at 05:27:03PM -0400, Phil Shafer wrote:
> I have a problem reported with libxo-based applications running
> under FreeBSD-11-stable on i386 boxes that I think is related
> to rtld:
>
> When I breakpoint on main() and dump the contents of my uninitialized
> thread-specific variable, it has not been initialized to zeroes.
>
> I don't see this problem on 64-bit systems, only on i386 ones.
>
> When I look at the rtld code, it appears to memset the .tbss to
> zero (/usr/src/libexec/rtld-elf/rtld.c:allocate_tls) in the
> non-arch-specific code so the arch shouldn't matter, but something
> is not working right.
>
> So I'm looking for a helpful clue, such as how to debug rtld to see
> why this isn't being zeroed. I thought I'd use:
>
> gdb /libexec/ld-elf.so.1
> run /usr/bin/uptime
>
> for this doesn't work for me (SEGV with a callstack that doesn't
> make sense).
You need to supply argv[0]. Read ld-elf.so.1(1), it has the whole
section about direct execution mode.
>
> For this instance, the work around is to initialize the contents
> of xo_default_handle to zero so it's not in the .tbss, but I'd like
> to understand what's failing. In truth, I just have a hard time
> blaming rtld, even though this is issue is an obscure intersection
> of weird things (.tbbs on i386). Perhaps it's something wrong with
> how the library is built or similar. But given that it's not zeroed
> when main() get control, something's clearly broken.
>
> Details follow:
>
> I declare my variable as:
>
> #define THREAD_LOCAL(_x) __thread _x
> ...
> static THREAD_LOCAL(xo_handle_t) xo_default_handle;
>
> To help debug this issue, I made the following change to the sources
> to help with gdb's inability to show thread-local variables ("Cannot
> find thread-local variables on this target"):
>
> --- contrib/libxo/libxo/libxo.c.save 2018-05-04 17:26:29.079500000 -0400
> +++ contrib/libxo/libxo/libxo.c 2018-05-04 17:28:06.570875000 -0400
> @@ -8349,3 +8349,11 @@
> xop->xo_style = XO_STYLE_ENCODER;
> xop->xo_encoder = encoder;
> }
> +
> +void xo_print_handle (void);
> +void
> +xo_print_handle (void)
> +{
> + fprintf(stderr, "xo_default_handle: %p %d\n",
> + &xo_default_handle, sizeof(xo_handle_t));
> +}
>
> When I run the failing command (uptime) under gdb and breakpoint
> on main, my thread-local variable is not set to zeroes:
>
> % gdb uptime
> GNU gdb 6.1.1 [FreeBSD]
> ...
> This GDB was configured as "i386-marcel-freebsd"...
> (gdb) b main
> Breakpoint 1 at 0x8049be5: file /usr/src/usr.bin/w/w.c, line 145.
> (gdb) run
> Starting program: /usr/home/phil/work/lib/uptime
>
> Breakpoint 1, main (argc=1, argv=0xbfbfe60c) at /usr/src/usr.bin/w/w.c:145
> 145 (void)setlocale(LC_ALL, "");
> Current language: auto; currently minimal
> (gdb) call xo_print_handle()
> xo_default_handle: 0x2806aff0 328
> $1 = 34
> (gdb) x/82x 0x2806aff0
> 0x2806aff0: 0x00000000 0x00000000 0x00000000 0x280601ef
> 0x2806b000: 0x2806b010 0x2806a200 0x00000001 0x280601ef
> 0x2806b010: 0x2806b020 0x2806a400 0x0000005d 0x280601ef
> 0x2806b020: 0x2806b030 0x2806a600 0x000000a1 0x280601ef
> 0x2806b030: 0x2806b040 0x2806a800 0x00000147 0x280601ef
> 0x2806b040: 0x00000000 0x2806aa00 0x00000164 0x280601ef
> 0x2806b050: 0x2806c000 0x00000000 0x28065e70 0x280601ef
> 0x2806b060: 0x2806b070 0x2806ac00 0x00000421 0x280601ef
> 0x2806b070: 0x00000000 0x2806aa00 0x0000042d 0x280601ef
> 0x2806b080: 0x00000000 0x2806aa00 0x000001ff 0x280601ef
> 0x2806b090: 0x2806b0a0 0x2806a800 0x00000976 0x280601ef
> 0x2806b0a0: 0x00000000 0x2806aa00 0x00000983 0x280601ef
> 0x2806b0b0: 0x00000000 0x2806aa00 0x00000a18 0x280601ef
> 0x2806b0c0: 0x00000000 0x2806aa00 0x00000571 0x280601ef
> 0x2806b0d0: 0x2806b0e0 0x2806a000 0x00000000 0x280601ef
> 0x2806b0e0: 0x2806b0f0 0x2806a200 0x00000000 0x280601ef
> 0x2806b0f0: 0x2806b100 0x2806a400 0x00000000 0x280601ef
> 0x2806b100: 0x2806b110 0x2806a600 0x00000000 0x280601ef
> 0x2806b110: 0x2806b120 0x2806a800 0x00000000 0x280601ef
> 0x2806b120: 0x2806b130 0x2806aa00 0x00000000 0x280601ef
> 0x2806b130: 0x00000000 0x2806ac00
> (gdb)
>
> objdump shows the lib does have a .tbbs:
>
> 14 .tbss 00000658 000181f8 000181f8 000171f8 2**3
> ALLOC, THREAD_LOCAL
Can you build the library and binary with clang 5 or even gcc, and
see how is it end up ?
Also, try to link in libpthread.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180508081417.GL6887>