From owner-svn-src-all@freebsd.org Mon Feb 4 08:10:20 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE78414C2B6D; Mon, 4 Feb 2019 08:10:19 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18E1972792; Mon, 4 Feb 2019 08:10:19 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wm1-x344.google.com with SMTP id t200so12376760wmt.0; Mon, 04 Feb 2019 00:10:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:reply-to:subject:to:cc:references:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language; bh=AOIaRjvSiujimDdLxR7bJ78wUB+WV7aWHfuybW8yeAo=; b=cNPCTmxV8W965ycyfVtszHOMKtUQRADLIWJL+uQyuBg647pSzyTrh/Xzl2Qh7188tg luPqlyF026zqxd5gMM9gKuPSlfi2AVegG6sSdmmKZDYZl+s9AwEnmXloAmRvkFboZx4t FO9ZoHultJTVgn9AGTZUaHY1dOlPhdlfy6+um7HumGno1l+zW6Pix4PetfndgYNQazIM +mJ87j3zPwTI0HlBayMr1AURO1eT5HmN0KuZlo4E3jXNEp4zJ/j8R/LeV/dywQsKXfeu z1cOAmrFQ1CyN1dgjG7ldKmcIPJmmGmWB5R0BTiPsklguaUkoQZcd8KqXstXPwEh+F78 MhvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:reply-to:subject:to:cc:references :openpgp:autocrypt:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=AOIaRjvSiujimDdLxR7bJ78wUB+WV7aWHfuybW8yeAo=; b=AxkkSV9i9IG6zYHtyADhp4VWagofkfOn268Mk1BqYHjb2YTq6KEdDQ/rJaQlob7iOg 9M434IKyekdk4Iml7erwzub+5PQNzuQxfgK8BTWsG1BD7x3IBuA4yHl3PXAVHbGSezH3 YKz8yHZMtghhbB8ow02misTSOm/4p0D+7FsYa/vrTKiefFtCU7+/qcnRlHYABmIwEaa0 plIJfxDi7jPEmzUSvV2SGVBlE3KQxqPlfBFk+zmP/4k+/+VQ/63YF+vDiDxqiUWQgv3x 5AfMtBc7ZqrB4zKwuPCPZ16PI5IE4zIVDYUTWQvHq/nM9MJbSjhMyNc1+Fpfr+kYxyKJ KmDw== X-Gm-Message-State: AHQUAuZUt6TnSP/X5IQ3ExZkAKN4vz/IZG6HE/nW3xNKj+zuhxK68ZZX p7ASSDxzat6GTCv9IXITyhwVTibJ X-Google-Smtp-Source: AHgI3IZFEXcTaQB1YEOdW1FX422HjU/dVQaItyZBxMnh0SQpX1wUDihdiQL/lEjfjcn+B5jINVn21w== X-Received: by 2002:a1c:cec1:: with SMTP id e184mr12609091wmg.75.1549267817658; Mon, 04 Feb 2019 00:10:17 -0800 (PST) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id m4sm10266168wmi.3.2019.02.04.00.10.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Feb 2019 00:10:16 -0800 (PST) Sender: Michal Meloun From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: svn commit: r343566 - in head/lib/libthr: . thread To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201901292246.x0TMkjQH074121@repo.freebsd.org> <20190204070017.GO24863@kib.kiev.ua> Openpgp: preference=signencrypt Autocrypt: addr=mmel@freebsd.org; prefer-encrypt=mutual; keydata= mQENBFYuVRkBCADZiwLCCne3wG9b9k+R2Neo5zVo2bLaZRfNNY/v9kg283i0sb1Da4EdEiNT 15El5UyozhphUIbIR/zrVpxF1TvvFdoCyzx6a68bNY2d9dBrDcNDZC+XnyDdHQoobN87DWT1 mRVkmbg9LHZ/SVUOkGYuWyE+8UYeDAcUizuXwSK5zFWmeTyIoWNa68ifrWLfQe0p4x5jC/AI VURCi17p360vU4fhgwoMvEEhrRBWCr4DYHToFjIt2WdBy3GR1qoO0+Xkd6G+OoBULo+XDfgu L2WdPvh0K69F9/LgHkMmG5Il7SCe62QGpG2vaCgRV7BQhLX+kxlvM+WrdRatWRml4Y/3ABEB AAG0IE1pY2hhbCBNZWxvdW4gPG1tZWxAZnJlZWJzZC5vcmc+iQFXBBMBCgBBAhsDBQsJCAcD BRUKCQgLBRYDAgEAAh4BAheAAhkBFiEEAN1KEEuAn+Apg413aR6ya17FqqoFAlw3aO8FCQ9v FVYACgkQaR6ya17FqqrIFwf/V/NRuAutw1cZ/HRCwfp00KY+8g730KwOaAMhPYvq8tqf8krD whFRWt9s4DSVTVJUC2dRZa5g1MS77iQL/WWzBUhJj7rW4uMpcKpi7UXi060sKrnE+1ZBvanG 8YCMTHKxQgQDD0cBqsyOAfrSA+Pyya3/HyAh1vvjQZfu5vWxA1X5NX5l5KtonprJ+ybVacwO W34yfWSX+ncx089JpUknenGdI+gVoKfa6uh/8YiCZTI5poxBttrOB8NGfylTsTTTbw2pkire Vkc87qq++I9bBNtTqWOZjB/3KBUd4q1veEUa4MS63TWtAvcEcsxHiBv4AH6npPO/iPycuGGh qldT1LkBDQRWLlUZAQgAuqBGR9LbUP86pYT2Zd+HZsiOun24LxPQwK6KZuysB0so02isa7ac T6t5uBfyfmdgzfvsc6dqU8bI65e8JtDwwVVF1Y4fF6cmNcmJqU10Gw5/bAu9re9JJWx2g/OJ tYMhwmqEE5DcS6GwgvJZI+9hgoSLL3mZ+GsO25NM4BhVH1o1AE4Q3O0HZMMnIzH+pma0augZ Xc3Qep17/Occ+1A7xEbn8VAYx8cwjkYUy7JyvVx4ecuj+i+uoCXhNhX5QdIcn4UMT1233Bcd yVZtZsLTI26ByFmrquGjkmN+/hra/s+nKfcegOKeHWVY/uFgDRk0TEYd6QuddZWo7+XMZLps wQARAQABiQElBBgBCgAPAhsMBQJZjBHDBQkHICOqAAoJEGkesmtexaqquysH/iDgRIrlMvFT GLyktZws/K/C03gWZ/tWwJV5+SFqRoasM+Z+9+cb44XA3rReqieLsYF5qSXTXHBPhEg1+9V9 KEPLpwektLtAv8zynN+6VVffwyRWQ4PIZmHCMT6PJP+Rk0DU6VrDc3ch58erduuyRDsbh32g 96nYMNvDsLhDvq44GPHLfglUHRY6wHo3IxAH2HosaWNC10R13JSrCdFnPMeIa+nSE+11hAdB mWpuPgVX2VNErtrwSBr7LsyCZ4yLO0bN5fKM0/Tavc9EcI9qDBrnts/EfZoCHhDtTMEFnQZM ybsMso20I/nxYPFuWuuFn7uQeoYrwWxt15SkN0RgtV2JATwEGAEKACYCGwwWIQQA3UoQS4Cf 4CmDjXdpHrJrXsWqqgUCXDdpmwUJD28WAgAKCRBpHrJrXsWqqtOrB/9Okk/dm2/mAs3tbA6l lKOZAwfUwBonwG7YhK9dOb5No/bJoY8G9s2IVXqwWKqCDv1qKC4M8pMXPD9fVsfNPG8GA1GZ rpZoZxHFteXZHoE1grpWypE0T3hp1W0hQNWXKv0MSuEMRVKU1m/iyLa10X7q8BaKtEDs35Xf DCh7eVg1vqiKtWyFn1u+y9Gy1Ztc/SKTs8mQvMYNUdBZuV6vN0VSz3I6wD9HVfo6Eo3M7cVp qqQ+KhrrGiCCYQPfZwpFAJSSgV3bR5O0BlNaGKJOoVTV1yexRbVEStswiFP8JFQp6U+S3nqE FFob7daQBLxi80oUVRn6eEdzR8tBhIy7sbeD Message-ID: <420b91ef-b318-100b-a0b3-99f29d03cea8@freebsd.org> Date: Mon, 4 Feb 2019 09:10:17 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.0 MIME-Version: 1.0 In-Reply-To: <20190204070017.GO24863@kib.kiev.ua> Content-Type: multipart/mixed; boundary="------------98DC485539ACAB25B79005DB" Content-Language: en-US X-Rspamd-Queue-Id: 18E1972792 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; TAGGED_FROM(0.00)[] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2019 08:10:20 -0000 This is a multi-part message in MIME format. --------------98DC485539ACAB25B79005DB Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 04.02.2019 8:00, Konstantin Belousov wrote: > On Mon, Feb 04, 2019 at 07:31:19AM +0100, Michal Meloun wrote: >> On 29.01.2019 23:46, Konstantin Belousov wrote: >>> Author: kib >>> Date: Tue Jan 29 22:46:44 2019 >>> New Revision: 343566 >>> URL: https://svnweb.freebsd.org/changeset/base/343566 >>> >>> Log: >>> Untangle jemalloc and mutexes initialization. >>> >>> The need to use libc malloc(3) from some places in libthr always >>> caused issues. For instance, per-thread key allocation was switched to >>> use plain mmap(2) to get storage, because some third party mallocs >>> used keys for implementation of calloc(3). >>> >>> Even more important, libthr calls calloc(3) during initialization of >>> pthread mutexes, and jemalloc uses pthread mutexes. Jemalloc provides >>> some way to both postpone the initialization, and to make >>> initialization to use specialized allocator, but this is very fragile >>> and often breaks. See the referenced PR for another example. >>> >>> Add the small malloc implementation used by rtld, to libthr. Use it in >>> thr_spec.c and for mutexes initialization. This avoids the issues with >>> mutual dependencies between malloc and libthr in principle. The >>> drawback is that some more allocations are not interceptable for >>> alternate malloc implementations. There should be not too much memory >>> use from this allocator, and the alternative, direct use of mmap(2) is >>> obviously worse. >>> >>> PR: 235211 >>> MFC after: 2 weeks >>> Sponsored by: The FreeBSD Foundation >>> Differential revision: https://reviews.freebsd.org/D18988 >>> >> This broke ARM static binaries (at least rescue/rescue). From first >> look it seems that __pthread_mutex_init() invoked by atomic_init() is >> called before curthread is set (before _thread_init_hack()). >> >> >> root@tegra124:/usr/src # gdb --args >> /usr/obj/usr/src/arm.armv7/rescue/rescue/rescue sh >> ... >> Reading symbols from /usr/obj/usr/src/arm.armv7/rescue/rescue/rescue...done. >> (gdb) b _thread_init_hack >> Breakpoint 1 at 0x67cad0: file /usr/src/lib/libthr/thread/thr_init.c, >> line 296. >> (gdb) r >> Starting program: /usr/obj/usr/src/arm.armv7/rescue/rescue/rescue sh >> >> Program received signal SIGSEGV, Segmentation fault. >> __thr_calloc (num=1, size=) at >> /usr/src/lib/libthr/thread/thr_malloc.c:82 >> 82 thr_malloc_lock(curthread); >> (gdb) bt >> #0 __thr_calloc (num=1, size=) at >> /usr/src/lib/libthr/thread/thr_malloc.c:82 >> #1 0x00676e1c in mutex_init (mutex=, >> mutex_attr=, calloc_cb=) >> at /usr/src/lib/libthr/thread/thr_mutex.c:294 >> #2 __pthread_mutex_init (mutex=0xc6e948 , >> mutex_attr=) at /usr/src/lib/libthr/thread/thr_mutex.c:393 >> #3 0x001d41f0 in handle_static_init (argc=2, argv=, >> env=) at /usr/src/lib/csu/common/ignore_init.c:124 >> #4 0x001d40e8 in __start (argc=2, argv=, env=> out>, ps_strings=, obj=0x0, cleanup=0x0) >> at /usr/src/lib/csu/arm/crt1.c:112 >> #5 0x001d4000 in ?? () >> Backtrace stopped: previous frame identical to this frame (corrupt stack?) >> (gdb) f 3 >> #3 0x001d41f0 in handle_static_init (argc=2, argv=, >> env=) at /usr/src/lib/csu/common/ignore_init.c:124 >> 124 fn(argc, argv, env); >> (gdb) list >> 119 _init(); >> 120 array_size = __init_array_end - __init_array_start; >> 121 for (n = 0; n < array_size; n++) { >> 122 fn = __init_array_start[n]; >> 123 if ((uintptr_t)fn != 0 && (uintptr_t)fn != 1) >> 124 fn(argc, argv, env); >> 125 } >> 126 } >> (gdb) p n >> $1 = 20 >> (gdb) p fn >> $6 = (void (*)(int, char **, char **)) 0x63f21c >> (gdb) p (void *)&__init_array_start >> $19 = (void *) 0xa7ab60 >> (gdb) p (void *)&__init_array_end >> $20 = (void *) 0xa7abc0 >> (gdb) x/24a &__init_array_start >> 0xa7ab60: 0x1d4270 >> 0x2d926c <_$$hide$$ ifconfig.lo ifconfig_ctor> >> 0x2d98e0 <_$$hide$$ ifconfig.lo link_ctor> >> 0x2d9cb4 <_$$hide$$ ifconfig.lo inet_ctor> >> 0xa7ab70: 0x2da2e0 <_$$hide$$ ifconfig.lo inet6_ctor> >> 0x2db790 <_$$hide$$ ifconfig.lo clone_ctor> >> 0x2dbb8c <_$$hide$$ ifconfig.lo mac_ctor> >> 0x2dbe70 <_$$hide$$ ifconfig.lo ifmedia_ctor> >> 0xa7ab80: 0x2dced0 <_$$hide$$ ifconfig.lo fib_ctor> >> 0x2dd118 <_$$hide$$ ifconfig.lo vlan_ctor> >> 0x2dd668 <_$$hide$$ ifconfig.lo vxlan_ctor> >> 0x2df45c <_$$hide$$ ifconfig.lo gre_ctor> >> 0xa7ab90: 0x2df6d8 <_$$hide$$ ifconfig.lo gif_ctor> >> 0x2df874 <_$$hide$$ ifconfig.lo ipsec_ctor> >> 0x2e2144 <_$$hide$$ ifconfig.lo ieee80211_ctor> >> 0x2f0a10 <_$$hide$$ ifconfig.lo carp_ctor> >> 0xa7aba0: 0x2f0e6c <_$$hide$$ ifconfig.lo group_ctor> >> 0x2f199c <_$$hide$$ ifconfig.lo pfsync_ctor> >> 0x2f1a04 <_$$hide$$ ifconfig.lo bridge_ctor> >> 0x2f35dc <_$$hide$$ ifconfig.lo lagg_ctor> >> 0xa7abb0: 0x63f21c >> 0x67cad0 <_thread_init_hack> >> 0x90e5b4 >> 0x9a6b98 >> >> So it's clear that order of static constructors is invalid (moreover, I >> thing that we are inside undefined area here). >> Any idea/hint how to fix this. > > Try the following (I did not even compiled). It might require additional > handling of NULL curthread in thr_malloc.c, in which case the locking > should be elided. > > diff --git a/lib/libthr/thread/thr_malloc.c b/lib/libthr/thread/thr_malloc.c > index 157c72f10d6..8b72a1840f7 100644 > --- a/lib/libthr/thread/thr_malloc.c > +++ b/lib/libthr/thread/thr_malloc.c > @@ -46,6 +46,8 @@ void > __thr_malloc_init(void) > { > > + if (npagesizes != 0) > + return; > npagesizes = getpagesizes(pagesizes_d, nitems(pagesizes_d)); > if (npagesizes == -1) { > npagesizes = 1; > diff --git a/lib/libthr/thread/thr_mutex.c b/lib/libthr/thread/thr_mutex.c > index f6f37c1264e..4db65384331 100644 > --- a/lib/libthr/thread/thr_mutex.c > +++ b/lib/libthr/thread/thr_mutex.c > @@ -390,6 +390,7 @@ __pthread_mutex_init(pthread_mutex_t * __restrict mutex, > } > if (mutex_attr == NULL || > (*mutex_attr)->m_pshared == PTHREAD_PROCESS_PRIVATE) { > + __thr_malloc_init(); > return (mutex_init(mutex, mutex_attr ? *mutex_attr : NULL, > __thr_calloc)); > } > Thanks for fast response. It works, but yes, additional handling of NULL curthread is required. The following patch fixes this issue for me (tested only on ARMv7). Can you, please, check it and eventually commit it? Thanks, Michal --------------98DC485539ACAB25B79005DB Content-Type: text/plain; charset=UTF-8; name="thread.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="thread.diff" SW5kZXg6IGxpYi9saWJ0aHIvdGhyZWFkL3Rocl9tYWxsb2MuYwo9PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0t LSBsaWIvbGlidGhyL3RocmVhZC90aHJfbWFsbG9jLmMJKHJldmlzaW9uIDM0MzczMikKKysr IGxpYi9saWJ0aHIvdGhyZWFkL3Rocl9tYWxsb2MuYwkod29ya2luZyBjb3B5KQpAQCAtNDUs NiArNDUsOCBAQAogdm9pZAogX190aHJfbWFsbG9jX2luaXQodm9pZCkKIHsKKwlpZiAobnBh Z2VzaXplcyAhPSAwKQorCQlyZXR1cm47CiAKIAlucGFnZXNpemVzID0gZ2V0cGFnZXNpemVz KHBhZ2VzaXplc19kLCBuaXRlbXMocGFnZXNpemVzX2QpKTsKIAlpZiAobnBhZ2VzaXplcyA9 PSAtMSkgewpAQCAtNTksNiArNjEsOCBAQAogdGhyX21hbGxvY19sb2NrKHN0cnVjdCBwdGhy ZWFkICpjdXJ0aHJlYWQpCiB7CiAKKwlpZiAoY3VydGhyZWFkID09IE5VTEwpCisJCXJldHVy bjsKIAljdXJ0aHJlYWQtPmxvY2tsZXZlbCsrOwogCV90aHJfdW11dGV4X2xvY2soJnRocl9t YWxsb2NfdW10eCwgVElEKGN1cnRocmVhZCkpOwogfQpAQCAtNjcsNiArNzEsOCBAQAogdGhy X21hbGxvY191bmxvY2soc3RydWN0IHB0aHJlYWQgKmN1cnRocmVhZCkKIHsKIAorCWlmIChj dXJ0aHJlYWQgPT0gTlVMTCkKKwkJcmV0dXJuOwogCV90aHJfdW11dGV4X3VubG9jaygmdGhy X21hbGxvY191bXR4LCBUSUQoY3VydGhyZWFkKSk7CiAJY3VydGhyZWFkLT5sb2NrbGV2ZWwt LTsKIAlfdGhyX2FzdChjdXJ0aHJlYWQpOwpJbmRleDogbGliL2xpYnRoci90aHJlYWQvdGhy X211dGV4LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gbGliL2xpYnRoci90aHJlYWQvdGhyX211dGV4 LmMJKHJldmlzaW9uIDM0MzczMikKKysrIGxpYi9saWJ0aHIvdGhyZWFkL3Rocl9tdXRleC5j CSh3b3JraW5nIGNvcHkpCkBAIC0zOTAsNiArMzkwLDcgQEAKIAl9CiAJaWYgKG11dGV4X2F0 dHIgPT0gTlVMTCB8fAogCSAgICAoKm11dGV4X2F0dHIpLT5tX3BzaGFyZWQgPT0gUFRIUkVB RF9QUk9DRVNTX1BSSVZBVEUpIHsKKwkJX190aHJfbWFsbG9jX2luaXQoKTsKIAkJcmV0dXJu IChtdXRleF9pbml0KG11dGV4LCBtdXRleF9hdHRyID8gKm11dGV4X2F0dHIgOiBOVUxMLAog CQkgICAgX190aHJfY2FsbG9jKSk7CiAJfQo= --------------98DC485539ACAB25B79005DB--