From owner-svn-src-head@freebsd.org Fri Oct 27 18:22:28 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3E69E4DFE4; Fri, 27 Oct 2017 18:22:28 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BA6672B4C; Fri, 27 Oct 2017 18:22:28 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id r196so5057088wmf.2; Fri, 27 Oct 2017 11:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:reply-to:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=694EYRfM9ugfrs5+PVwsnUjBo590OKqHL4Lv8C9F0kY=; b=gmyCle3oUlW7p7W3fMKmCRNzLYkVAAvujmiuJ6P+R1G1O/jjRN9Jt06xNhIxskYKuF uHAz0BmFXz7t2kRdUtYkz7g2lbumSF7ta36PqV9ctu51sRsGm/Z7mW2MU/0np/oiWCI7 CaEhy+s3RvoS4J1EB3/ew6tFiqfYupmsl7qcO/7a7PaNkSGeNCieKYIa2u0yMGLxwxV+ XKHkHevv9nIxBxsQYMmmOnUdDA/UQVOSNJsctARcdjyvXoKESYbBtmI8VcRY3S4V1nX+ WOHYyx/Svtppq8QqTpPzSdpwriLdS2WhhvwykqptM7PhdJAEJMCu4i+mzIncOypOH90Z V5AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:reply-to:subject:to:cc:references :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=694EYRfM9ugfrs5+PVwsnUjBo590OKqHL4Lv8C9F0kY=; b=CmMFUpvK5FpJ7McckhBKaquGLPK7PCe8gczRdGcT5Nzzaro2Q+nfjVsLnLxnuY9G+N SzIK03Zm2GDZZtRkgxsOl5XNJXc5Cr8dAeGPGRSQPdaL4rcRXQARab4jYX8alTrCffaj n0XFYhHAmyM/duxhKK31gOZnOFMkRcPjQCCNpq3FgMWtkF/Tf3U3Zo3eUkhuxconPIHq 2PflWKfH2fr87LCteZZCU0haF0cGtpooB5UqWLlzXFxKP50J7zaSMKJrEAMRx9XvBv9b OOO52xgb0+5g2elf/WuOkMf90NHg0FllwG1h1qakxRsZRzN+Wc5YsfcJniBLMsuX1zoG WHMg== X-Gm-Message-State: AMCzsaXpBUxOz5F4kVr6g0yCS7+nYbHmfh8mBZStxE/8PZwnkTe2xtle wAYD8iaBG4ZdWe9l0YvB2Q82LTqES5o= X-Google-Smtp-Source: ABhQp+RspASLALiA988lygnA52s5rMT676wZHaORIYAVOX/6q0D/R95Mg34UWy8YMeuAIL72PGGObA== X-Received: by 10.80.141.79 with SMTP id t15mr1742426edt.70.1509128546624; Fri, 27 Oct 2017 11:22:26 -0700 (PDT) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id o60sm5599319eda.48.2017.10.27.11.22.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Oct 2017 11:22:26 -0700 (PDT) From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: svn commit: r324938 - head/contrib/jemalloc/include/jemalloc/internal To: Brooks Davis , Konstantin Belousov Cc: Dimitry Andric , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, kib@freebsd.org References: <201710232131.v9NLV4Rb068825@repo.freebsd.org> <38db6f4e-72b8-6ffd-4529-f15ca32bad54@freebsd.org> <6FD27DFB-5039-4E33-B131-EF5391DD1630@FreeBSD.org> <6eff6e66-4987-8753-105f-b6a5b8234ff3@freebsd.org> <20171027150841.GH2566@kib.kiev.ua> <20171027152154.GA31598@spindle.one-eyed-alien.net> Message-ID: <9b85bde5-c450-94c7-a31e-f2eb7f6ab9ee@freebsd.org> Date: Fri, 27 Oct 2017 20:22:25 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20171027152154.GA31598@spindle.one-eyed-alien.net> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2017 18:22:29 -0000 On Fri, Oct 27, 2017 at 19:48:43 +0200, Michal Meloun wrote: > On 27.10.2017 17:21, Brooks Davis wrote: >> On Fri, Oct 27, 2017 at 06:08:41PM +0300, Konstantin Belousov >> wrote: >>> On Fri, Oct 27, 2017 at 02:53:26PM +0200, Michal Meloun wrote: >>>> Sorry for top posting That's pity, we have clear problem in >>>> rtld code :( See: >>>> ----------------------------------------------------- RESCUE >>>> WITHOUT JEMALLOC_ALIGNED(16); >>>> ----------------------------------------------------- Program >>>> Headers: TLS 0xa732b0 0x00a832b0 0x00a832b0 0x00b40 >>>> 0x011bc R 0x8 Section Headers: 04 .tdata .tbss >>>> .init_array .fini_array .jcr .got Dump: 00a832b0 >>>> <__je_tsd_tls+0xa832b0>: a832b0: 00000005 >>>> >>>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08: >>>> tsd_fetch_impl. (6 locations) (gdb) r Starting program: >>>> /usr/src/rescue.noalign sh >>>> >>>> Breakpoint 1, tsd_fetch_impl (init=true, minimal=false) at >>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261 >>>> 261 tsd_t *tsd = tsd_get(init); (gdb) n 263 >>>> if (!init && tsd_get_allocates() && tsd == NULL) { >>>> >>>> (gdb) p tsd $1 = (tsd_t *) 0x20c83008 >>>> >>>> (gdb) p *tsd $2 = {state = 5 '\005', .... >>>> >>>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ... >>>> >>>> >>>> >>>> ----------------------------------------------------- RESCUE >>>> WITH JEMALLOC_ALIGNED(16); >>>> ----------------------------------------------------- Program >>>> Headers: TLS 0xa732b0 0x00a832b0 0x00a832b0 0x00b40 >>>> 0x011bc R 0x10 Section Headers: 04 .tdata .tbss >>>> .init_array .fini_array .jcr .got Dump: 00a832b0 >>>> <__je_tsd_tls+0xa832b0>: a832b0: 00000005 >>>> >>>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08: >>>> tsd_fetch_impl. (6 locations) (gdb) r Starting program: >>>> /usr/obj/usr/src/rescue/rescue/rescue sh Breakpoint 1, >>>> tsd_fetch_impl (init=true, minimal=false) at >>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261 >>>> 261 tsd_t *tsd = tsd_get(init); (gdb) n 263 >>>> if (!init && tsd_get_allocates() && tsd == NULL) { >>>> >>>> (gdb) p tsd $1 = (tsd_t *) 0x20c83010 >>>> >>>> (gdb) p *tsd $2 = {state = 0 '\000', ... >>>> >>>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ... >>>> >>>> !!!! BUT p *(tsd - 8 bytes) !!!!!!!!!! (gdb) p *((tsd_t >>>> *)0x20c83008) $4 = {state = 5 '\005', ... >>>> >>>> ----------------------------------------------------- So it's >>>> clear that: >>>> >>>> - both binaries are valid, .tdata section have valid data. - >>>> requested alignment is propagated to binary. - .tdata section >>>> is properly loaded to memory because p *((tsd_t *)0x00a832b0) >>>> is {state = 5 '\005' in both cases >>>> >>>> - a per thread copy of .tdata respect requested alignment but >>>> the original data was copied to unaligned address. because for >>>> aligned binary p *tsd is {state = 0 '\000', ... p *(tsd - 8 >>>> bytes) is {state = 5 '\005' >>>> >>>> I'm right? Kib, please, can you help us? >>> >>> Does it happen for rescue binary ? >>> >>> Note that the binary is linked static, so the problem is in >>> lib/libc/gen/tls.c and not in rtld. There, I do not see any real >>> use of the phdr' p_align value. >>> > Ahh, right, good catch. >>> BTW, is rescue linked to libthr ? > Imho not. My bad, rescue *is* linked with libthr. And after looking to libthr/arch/arm/pthread_md.h, I'm confused much much more... >> >> There isn't alignment support for TLS in static binaries. I've >> fixed this in CheriBSD and am planning to upstream the fixes at >> some point. The fix for variant I is in: >> >> https://github.com/CTSRD-CHERI/cheribsd/commit/3cfb124ebb9fdb545dad8436a04dd58c05b33f4b >> >> > The real alignment have no effect for program itself, it's important > only for AddressSanitizer. > > But there is something what's still missing me. > The rescue binary is statically linked, so whole program have only one > TLS section and full TLS layout is determined by linker. > The problematic structure, tsd_tls, is first one in TLS section group: > > readelf -a /usr/obj/usr/src/rescue/rescue/rescue | grep TLS > 518961: 0000000000000000 2880 TLS GLOBAL DEFAULT 9 __je_tsd_tls > and > > objdump -d -j .tdata /usr/obj/usr/src/rescue/rescue/rescue > 00a832b0 <__je_tsd_tls+0xa832b0>: > a832b0: 00000005 > > but disassembled code expect it at TLS ptr + 0x8 (for code without > JEMALLOC_ALIGNED) or at TLS ptr + 0x10 (for code with JEMALLOC_ALIGNED) > > [without JEMALLOC_ALIGNED] > 7c4d90: e59f0014 ldr r0, [pc, #20] ; 7c4dac > > 7c4d94: e58d0004 str r0, [sp, #4] > 7c4d98: eb031d14 bl 88c1f0 <__aeabi_read_tp> > 7c4d9c: e59dc004 ldr ip, [sp, #4] > 7c4da0: e080000c add r0, r0, ip > 7c4da4: e1a0d00b mov sp, fp > 7c4da8: e8bd8800 pop {fp, pc} > 7c4dac: 00000008 .word 0x00000008 [0x10 for code > with JEMALLOC_ALIGNED] > > so someone other allocates at least one byte from start of TLS and > linker is aware about this. But again, .data is first member of TLS > and tsd_tls is first of .tdata and the 0x5 is value of first member of > tsd_tls structure (.state = tsd_state_uninitialized). > > So who consumes the first bytes in TLS ???? > > Michal