From owner-svn-src-all@freebsd.org Fri Oct 27 17:48:46 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C67A5E4D49C; Fri, 27 Oct 2017 17:48:46 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 469D5719C8; Fri, 27 Oct 2017 17:48:46 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id b189so4869337wmd.4; Fri, 27 Oct 2017 10:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:reply-to:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=HCzH87EdiNHYw9MPluAfOpVwCnlNI4ZBWMd37ACgP8c=; b=SdcF8e/OUBkTouhAf161ASVOAnKqfxg13vQye9ooG72BZ1iYIG6KZSW9HFutieC2F8 loUNI3qSYKj6uw8Evfpr6xzhbsKdzibX4a2rqP4xaDrL7Ip95AmTtgcoxMTPMqmdrSzD qjtidg9R2j3/p/P7VYRETVKoX86cnliglhZMTBACDQ6RZJxQjsvJSL5ZcMMBtARKjDXz 79Y+S/HvfpL4pbD9alt/IjT9lU2C/o7HEnp/EJ+OEKJJqNyE0nNtTrLb9ZTW8JWfPd5d ibOOJHiE+4YiM8q56GjqeMF/yXDrZHU8vimVebbZivY2bLpSL2Xd/zWVCCogG1tur0+M bVgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:reply-to:subject:to:cc:references :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=HCzH87EdiNHYw9MPluAfOpVwCnlNI4ZBWMd37ACgP8c=; b=oEpw1XeMGXojarj6B9p5i4JK4Q6k182lPqNDrzptkPR4HUKTY2XYpgy/DuaUkcvTXY fKAn3vpW4CiD5oteUmrJXfnyI0fL9MSslKNCtQgy9fuKgZY4DbdxU5Bj5a4IsDc3B9z0 9WVwwjsNAJi1kiWPE7Ja/9CDtAF3kbjtHIZIS9s4MpIaiq+CFnrfZn5ncy18EjbghOOi Vt+cZm7JRQc9FBSNDkc27SnqRQ5H+8zUoVvnSyH8yJ/SfQmIO5vqKGwRVSpGjjl2i7HX 5nczrFOl1Y/bnILcAa9kfIEhRulrma79tS4LKvKUQ6+qlYX9lBZ9io9Q4uFcdru3vJge S/Yw== X-Gm-Message-State: AMCzsaWQqF7QfzfuJlRX+I7OHwWMlxLTw5usUrY+oBDzP9BtFgimA+2p EDKlbPT69+RUHf3ymdburBLdgk7ulQ4= X-Google-Smtp-Source: ABhQp+SRjXh6xrEsTtQN0yfPi/EHFfuwp9UypVVuBB/6Ztn+wcNmQcab1sHNMpERpqerFgfFbwU/5g== X-Received: by 10.80.182.118 with SMTP id c51mr1654622ede.204.1509126524314; Fri, 27 Oct 2017 10:48:44 -0700 (PDT) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id i6sm5792482edk.3.2017.10.27.10.48.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Oct 2017 10:48:43 -0700 (PDT) From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: svn commit: r324938 - head/contrib/jemalloc/include/jemalloc/internal To: Brooks Davis , Konstantin Belousov Cc: Dimitry Andric , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, kib@freebsd.org References: <201710232131.v9NLV4Rb068825@repo.freebsd.org> <38db6f4e-72b8-6ffd-4529-f15ca32bad54@freebsd.org> <6FD27DFB-5039-4E33-B131-EF5391DD1630@FreeBSD.org> <6eff6e66-4987-8753-105f-b6a5b8234ff3@freebsd.org> <20171027150841.GH2566@kib.kiev.ua> <20171027152154.GA31598@spindle.one-eyed-alien.net> Message-ID: Date: Fri, 27 Oct 2017 19:48:43 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20171027152154.GA31598@spindle.one-eyed-alien.net> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2017 17:48:46 -0000 On 27.10.2017 17:21, Brooks Davis wrote: > On Fri, Oct 27, 2017 at 06:08:41PM +0300, Konstantin Belousov > wrote: >> On Fri, Oct 27, 2017 at 02:53:26PM +0200, Michal Meloun wrote: >>> Sorry for top posting That's pity, we have clear problem in >>> rtld code :( See: >>> ----------------------------------------------------- RESCUE >>> WITHOUT JEMALLOC_ALIGNED(16); >>> ----------------------------------------------------- Program >>> Headers: TLS 0xa732b0 0x00a832b0 0x00a832b0 0x00b40 >>> 0x011bc R 0x8 Section Headers: 04 .tdata .tbss >>> .init_array .fini_array .jcr .got Dump: 00a832b0 >>> <__je_tsd_tls+0xa832b0>: a832b0: 00000005 >>> >>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08: >>> tsd_fetch_impl. (6 locations) (gdb) r Starting program: >>> /usr/src/rescue.noalign sh >>> >>> Breakpoint 1, tsd_fetch_impl (init=true, minimal=false) at >>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261 >>> 261 tsd_t *tsd = tsd_get(init); (gdb) n 263 >>> if (!init && tsd_get_allocates() && tsd == NULL) { >>> >>> (gdb) p tsd $1 = (tsd_t *) 0x20c83008 >>> >>> (gdb) p *tsd $2 = {state = 5 '\005', .... >>> >>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ... >>> >>> >>> >>> ----------------------------------------------------- RESCUE >>> WITH JEMALLOC_ALIGNED(16); >>> ----------------------------------------------------- Program >>> Headers: TLS 0xa732b0 0x00a832b0 0x00a832b0 0x00b40 >>> 0x011bc R 0x10 Section Headers: 04 .tdata .tbss >>> .init_array .fini_array .jcr .got Dump: 00a832b0 >>> <__je_tsd_tls+0xa832b0>: a832b0: 00000005 >>> >>> GDB (gdb) b tsd_fetch_impl Breakpoint 1 at 0x7c4c08: >>> tsd_fetch_impl. (6 locations) (gdb) r Starting program: >>> /usr/obj/usr/src/rescue/rescue/rescue sh Breakpoint 1, >>> tsd_fetch_impl (init=true, minimal=false) at >>> /usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:261 >>> 261 tsd_t *tsd = tsd_get(init); (gdb) n 263 >>> if (!init && tsd_get_allocates() && tsd == NULL) { >>> >>> (gdb) p tsd $1 = (tsd_t *) 0x20c83010 >>> >>> (gdb) p *tsd $2 = {state = 0 '\000', ... >>> >>> (gdb) p *((tsd_t *)0x00a832b0) $3 = {state = 5 '\005', ... >>> >>> !!!! BUT p *(tsd - 8 bytes) !!!!!!!!!! (gdb) p *((tsd_t >>> *)0x20c83008) $4 = {state = 5 '\005', ... >>> >>> ----------------------------------------------------- So it's >>> clear that: >>> >>> - both binaries are valid, .tdata section have valid data. - >>> requested alignment is propagated to binary. - .tdata section >>> is properly loaded to memory because p *((tsd_t *)0x00a832b0) >>> is {state = 5 '\005' in both cases >>> >>> - a per thread copy of .tdata respect requested alignment but >>> the original data was copied to unaligned address. because for >>> aligned binary p *tsd is {state = 0 '\000', ... p *(tsd - 8 >>> bytes) is {state = 5 '\005' >>> >>> I'm right? Kib, please, can you help us? >> >> Does it happen for rescue binary ? >> >> Note that the binary is linked static, so the problem is in >> lib/libc/gen/tls.c and not in rtld. There, I do not see any real >> use of the phdr' p_align value. >> Ahh, right, good catch. >> BTW, is rescue linked to libthr ? Imho not. > > There isn't alignment support for TLS in static binaries. I've > fixed this in CheriBSD and am planning to upstream the fixes at > some point. The fix for variant I is in: > > https://github.com/CTSRD-CHERI/cheribsd/commit/3cfb124ebb9fdb545dad8436a04dd58c05b33f4b > > The real alignment have no effect for program itself, it's important only for AddressSanitizer. But there is something what's still missing me. The rescue binary is statically linked, so whole program have only one TLS section and full TLS layout is determined by linker. The problematic structure, tsd_tls, is first one in TLS section group: readelf -a /usr/obj/usr/src/rescue/rescue/rescue | grep TLS 518961: 0000000000000000 2880 TLS GLOBAL DEFAULT 9 __je_tsd_tls and objdump -d -j .tdata /usr/obj/usr/src/rescue/rescue/rescue 00a832b0 <__je_tsd_tls+0xa832b0>: a832b0: 00000005 but disassembled code expect it at TLS ptr + 0x8 (for code without JEMALLOC_ALIGNED) or at TLS ptr + 0x10 (for code with JEMALLOC_ALIGNED) [without JEMALLOC_ALIGNED] 7c4d90: e59f0014 ldr r0, [pc, #20] ; 7c4dac 7c4d94: e58d0004 str r0, [sp, #4] 7c4d98: eb031d14 bl 88c1f0 <__aeabi_read_tp> 7c4d9c: e59dc004 ldr ip, [sp, #4] 7c4da0: e080000c add r0, r0, ip 7c4da4: e1a0d00b mov sp, fp 7c4da8: e8bd8800 pop {fp, pc} 7c4dac: 00000008 .word 0x00000008 [0x10 for code with JEMALLOC_ALIGNED] so someone other allocates at least one byte from start of TLS and linker is aware about this. But again, .data is first member of TLS and tsd_tls is first of .tdata and the 0x5 is value of first member of tsd_tls structure (.state = tsd_state_uninitialized). So who consumes the first bytes in TLS ???? Michal