Date: Fri, 27 Oct 2017 19:48:43 +0200 From: Michal Meloun <melounmichal@gmail.com> To: Brooks Davis <brooks@freebsd.org>, Konstantin Belousov <kostikbel@gmail.com> Cc: Dimitry Andric <dim@FreeBSD.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, kib@freebsd.org Subject: Re: svn commit: r324938 - head/contrib/jemalloc/include/jemalloc/internal Message-ID: <d75bb1fc-60c8-c9b8-eef8-fc36750dca18@freebsd.org> In-Reply-To: <20171027152154.GA31598@spindle.one-eyed-alien.net> References: <201710232131.v9NLV4Rb068825@repo.freebsd.org> <38db6f4e-72b8-6ffd-4529-f15ca32bad54@freebsd.org> <6FD27DFB-5039-4E33-B131-EF5391DD1630@FreeBSD.org> <6eff6e66-4987-8753-105f-b6a5b8234ff3@freebsd.org> <20171027150841.GH2566@kib.kiev.ua> <20171027152154.GA31598@spindle.one-eyed-alien.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 27.10.2017 17:21, Brooks Davis wrote:
> On Fri, Oct 27, 2017 at 06:08:41PM +0300, Konstantin Belousov
> wrote:
>> On Fri, Oct 27, 2017 at 02:53:26PM +0200, Michal Meloun wrote:
>>> Sorry for top posting That's pity, we have clear problem in
>>> rtld code :( See:
>>> ----------------------------------------------------- RESCUE
>>> WITHOUT JEMALLOC_ALIGNED(16);
>>> ----------------------------------------------------- Program
>>> Headers: TLS 0xa732b0 0x00a832b0 0x00a832b0 0x00b40
>>> 0x011bc R 0x8 Section Headers: 04 .tdata .tbss
>>> .init_array .fini_array .jcr .got Dump: 00a832b0
>>> <__je_tsd_tls+0xa832b0>: a832b0: 00000005
>>>
>>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08:
>>> tsd_fetch_impl. (6 locations) (gdb) r Starting program:
>>> /usr/src/rescue.noalign sh
>>>
>>> Breakpoint 1, tsd_fetch_impl (init=true, minimal=false) at
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261
>>> 261 tsd_t *tsd = tsd_get(init); (gdb) n 263
>>> if (!init && tsd_get_allocates() && tsd == NULL) {
>>>
>>> (gdb) p tsd $1 = (tsd_t *) 0x20c83008
>>>
>>> (gdb) p *tsd $2 = {state = 5 '\005', ....
>>>
>>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ...
>>>
>>>
>>>
>>> ----------------------------------------------------- RESCUE
>>> WITH JEMALLOC_ALIGNED(16);
>>> ----------------------------------------------------- Program
>>> Headers: TLS 0xa732b0 0x00a832b0 0x00a832b0 0x00b40
>>> 0x011bc R 0x10 Section Headers: 04 .tdata .tbss
>>> .init_array .fini_array .jcr .got Dump: 00a832b0
>>> <__je_tsd_tls+0xa832b0>: a832b0: 00000005
>>>
>>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08:
>>> tsd_fetch_impl. (6 locations) (gdb) r Starting program:
>>> /usr/obj/usr/src/rescue/rescue/rescue sh Breakpoint 1,
>>> tsd_fetch_impl (init=true, minimal=false) at
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261
>>> 261 tsd_t *tsd = tsd_get(init); (gdb) n 263
>>> if (!init && tsd_get_allocates() && tsd == NULL) {
>>>
>>> (gdb) p tsd $1 = (tsd_t *) 0x20c83010
>>>
>>> (gdb) p *tsd $2 = {state = 0 '\000', ...
>>>
>>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ...
>>>
>>> !!!! BUT p *(tsd - 8 bytes) !!!!!!!!!! (gdb) p *((tsd_t
>>> *)0x20c83008) $4 = {state = 5 '\005', ...
>>>
>>> ----------------------------------------------------- So it's
>>> clear that:
>>>
>>> - both binaries are valid, .tdata section have valid data. -
>>> requested alignment is propagated to binary. - .tdata section
>>> is properly loaded to memory because p *((tsd_t *)0x00a832b0)
>>> is {state = 5 '\005' in both cases
>>>
>>> - a per thread copy of .tdata respect requested alignment but
>>> the original data was copied to unaligned address. because for
>>> aligned binary p *tsd is {state = 0 '\000', ... p *(tsd - 8
>>> bytes) is {state = 5 '\005'
>>>
>>> I'm right? Kib, please, can you help us?
>>
>> Does it happen for rescue binary ?
>>
>> Note that the binary is linked static, so the problem is in
>> lib/libc/gen/tls.c and not in rtld. There, I do not see any real
>> use of the phdr' p_align value.
>>
Ahh, right, good catch.
>> BTW, is rescue linked to libthr ?
Imho not.
>
> There isn't alignment support for TLS in static binaries. I've
> fixed this in CheriBSD and am planning to upstream the fixes at
> some point. The fix for variant I is in:
>
> https://github.com/CTSRD-CHERI/cheribsd/commit/3cfb124ebb9fdb545dad8436a04dd58c05b33f4b
>
>
The real alignment have no effect for program itself, it's important
only for AddressSanitizer.
But there is something what's still missing me.
The rescue binary is statically linked, so whole program have only one
TLS section and full TLS layout is determined by linker.
The problematic structure, tsd_tls, is first one in TLS section group:
readelf -a /usr/obj/usr/src/rescue/rescue/rescue | grep TLS
518961: 0000000000000000 2880 TLS GLOBAL DEFAULT 9 __je_tsd_tls
and
objdump -d -j .tdata /usr/obj/usr/src/rescue/rescue/rescue
00a832b0 <__je_tsd_tls+0xa832b0>:
a832b0: 00000005
but disassembled code expect it at TLS ptr + 0x8 (for code without
JEMALLOC_ALIGNED) or at TLS ptr + 0x10 (for code with JEMALLOC_ALIGNED)
[without JEMALLOC_ALIGNED]
7c4d90: e59f0014 ldr r0, [pc, #20] ; 7c4dac
<tsd_get+0x80>
7c4d94: e58d0004 str r0, [sp, #4]
7c4d98: eb031d14 bl 88c1f0 <__aeabi_read_tp>
7c4d9c: e59dc004 ldr ip, [sp, #4]
7c4da0: e080000c add r0, r0, ip
7c4da4: e1a0d00b mov sp, fp
7c4da8: e8bd8800 pop {fp, pc}
7c4dac: 00000008 .word 0x00000008 [0x10 for code
with JEMALLOC_ALIGNED]
so someone other allocates at least one byte from start of TLS and
linker is aware about this. But again, .data is first member of TLS
and tsd_tls is first of .tdata and the 0x5 is value of first member of
tsd_tls structure (.state = tsd_state_uninitialized).
So who consumes the first bytes in TLS ????
Michal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d75bb1fc-60c8-c9b8-eef8-fc36750dca18>