From nobody Sat May 3 05:46:21 2025 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZqGxV3pY9z5tk5T for ; Sat, 03 May 2025 05:46:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZqGxV2GV5z3FZ4 for ; Sat, 03 May 2025 05:46:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1746251182; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LfBGQOtRV7CYRaiBAetZa92focgL69h93R7H4BWrpf0=; b=MbFYkA9yNM0p9aRs4pF9cfGtyKgCYaNXHiENDbVPqJT/aO13KLL2Lu1NSGRKEW9d3UTaPn dBTzhcufJ51X4rB6HAFN2Z7DjrbCCEut+KyezAK7cCptAIFZxhngA8Rm7EbkS3Nnk264XG g6k16H0ImgRF5ToGVbgfkVL4cvpwqOEf1k1nXk+haUFUVS5GVdI1N18eRnJNViBwLiKftf vxGha9cIwPTX6GOH4rNwpVssynWTNmC0YH4ScHPLp0P5uny3jEAw+1pnFDBMLVncymMfhm 2ULaZff0pD3MJiaW60m0bTg8JGFwzMMJ/iJ68C5pA4xWNDj3qmkE+Qi17Fh1pA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1746251182; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LfBGQOtRV7CYRaiBAetZa92focgL69h93R7H4BWrpf0=; b=dKnRf4ZaUUj5vSCOv0ELQqAaJvywvjRPzgaBnGUjPktfGgzOhU3l9wioO9JAC4FmnyPZTY 7MLSU4+84ycDjFBMb7Z65dNrQhcpFMxOeAOV2lBAGEvNUjcEG+KY55MzE9BccMbiZ3Fwic ltC/ct/Eu3bluP9oT0KzSquNmpDakBpzmgPiI0g6uun/v+8pPWRIpF+hXFKvqwPINK0edD ckaV4CmjWiK55XGSKgHSIbgoGV10xqx47AgsJpz4vVNlgmNLfRAglKKBpMrxottwyNnH8g Kg2FlmwuJcKOUVte4DNGakPWtkR2/G7cMMLUXwtvJaMLvSpiLocu7bOjp7a1Gg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1746251182; a=rsa-sha256; cv=none; b=p+189I7aQa1rTaukFDuGqdupzloS3brkZY3n7WwuzZV0CcgOlBpL/1/HWbV3a8D3PSWm84 n6Bqrq+ISQ4T19pTgIZr43oGsi9MJSe7Z7ZMsc5baE6UzIU8RqpKT7JBpOas1dqwlJGd/g wO2LODiWEjnUT8kxx4R/zn8AqsiFSV/T24J5gvx1pEVRMFGcnmA6ZwZ9zblKDF1H23WtMk BqJb4wkn9jrt0K/WYEdLwwtghj8fg+wZ5/MnvD5t6EDRUUwR7hNLrVKDXIwZLKmEQ3EjBP M6Vfa2QV1FqAFpa7ibZjxDMgS4Mallm9TXBKGkGqVGTcv/+bAVBLU/fdA093LA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZqGxV1JzCzDZB for ; Sat, 03 May 2025 05:46:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 5435kMS7081742 for ; Sat, 3 May 2025 05:46:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 5435kMxQ081741 for bugs@FreeBSD.org; Sat, 3 May 2025 05:46:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 286537] segfault in libthr __Tthr_mutex_trylock then jmalloc.c when run qdrant Date: Sat, 03 May 2025 05:46:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 14.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: igor.polovykh@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D286537 Bug ID: 286537 Summary: segfault in libthr __Tthr_mutex_trylock then jmalloc.c when run qdrant Product: Base System Version: 14.2-STABLE Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bin Assignee: bugs@FreeBSD.org Reporter: igor.polovykh@gmail.com I get segfault when run qdrant. I tried to compile it from sources (github repo) and from ports/databases. Behaviour is the same. There was no any err= ors and warnings while compilation of binary from sources. Something wrong with jemalloc.c=20 This is a source code of BSD (/usr/src/lib/libthr/thread/thr_mutex.c) where segfault happens: 615 __Tthr_mutex_trylock(pthread_mutex_t *mutex) 616 { 617 struct pthread *curthread; 618 struct pthread_mutex *m; 619 uint32_t id; 620 int ret, robust; 621 622 ret =3D check_and_init_mutex(mutex, &m); 623 if (ret !=3D 0) 624 return (ret); 625 curthread =3D _get_curthread(); 626 id =3D TID(curthread); 627 if (m->m_flags & PMUTEX_FLAG_PRIVATE) 628 THR_CRITICAL_ENTER(curthread);=20=20=20 <------------------------------------------------------------- 629 robust =3D _mutex_enter_robust(curthread, m); 630 ret =3D _thr_umutex_trylock(&m->m_lock, id); 631 if (__predict_true(ret =3D=3D 0) || ret =3D=3D EOWNERDEAD) { 632 enqueue_mutex(curthread, m, ret); 633 if (ret =3D=3D EOWNERDEAD) 634 m->m_lock.m_flags |=3D UMUTEX_NONCONSISTENT; 635 } else if (PMUTEX_OWNER_ID(m) =3D=3D id) { 636 ret =3D mutex_self_trylock(m); 637 } /* else {} */ 638 if (robust) 639 _mutex_leave_robust(curthread, m); 640 if (ret !=3D 0 && ret !=3D EOWNERDEAD && 641 (m->m_flags & PMUTEX_FLAG_PRIVATE) !=3D 0) 642 THR_CRITICAL_LEAVE(curthread); 643 return (ret); 644 } and jmalloc.c 2111 static bool 2112 malloc_init_hard(void) { 2113 tsd_t *tsd; 2114 2115 #if defined(_WIN32) && _WIN32_WINNT < 0x0600 2116 _init_init_lock(); 2117 #endif 2118 malloc_mutex_lock(TSDN_NULL, &init_lock); 2119 2120 #define UNLOCK_RETURN(tsdn, ret, reentrancy) \ 2121 malloc_init_hard_cleanup(tsdn, reentrancy); \ 2122 return ret; 2123 2124 if (!malloc_init_hard_needed()) { 2125 UNLOCK_RETURN(TSDN_NULL, false, false) 2126 } 2127 2128 if (malloc_init_state !=3D malloc_init_a0_initialized && 2129 malloc_init_hard_a0_locked()) { 2130 UNLOCK_RETURN(TSDN_NULL, true, false) 2131 } 2132 2133 malloc_mutex_unlock(TSDN_NULL, &init_lock); 2134 /* Recursive allocation relies on functional tsd. */ 2135 tsd =3D malloc_tsd_boot0(); 2136 if (tsd =3D=3D NULL) { 2137 return true; 2138 } 2139 if (malloc_init_hard_recursible()) { 2140 return true; 2141 } 2142 2143 malloc_mutex_lock(tsd_tsdn(tsd), &init_lock); 2144 /* Set reentrancy level to 1 during init. */ 2145 pre_reentrancy(tsd, NULL); 2146 /* Initialize narenas before prof_boot2 (for allocation). */ 2147 if (malloc_init_narenas() 2148 || background_thread_boot1(tsd_tsdn(tsd), b0get())) { 2149 UNLOCK_RETURN(tsd_tsdn(tsd), true, true) 2150 } 2151 if (config_prof && prof_boot2(tsd, b0get())) { 2152 UNLOCK_RETURN(tsd_tsdn(tsd), true, true) 2153 } 2154 2155 malloc_init_percpu(); 2156 2157 if (malloc_init_hard_finish()) { 2158 UNLOCK_RETURN(tsd_tsdn(tsd), true, true) 2159 } 2160 post_reentrancy(tsd); 2161 malloc_mutex_unlock(tsd_tsdn(tsd), &init_lock); 2162 2163 witness_assert_lockless(witness_tsd_tsdn( 2164 tsd_witness_tsdp_get_unsafe(tsd))); 2165 malloc_tsd_boot1(); 2166 /* Update TSD after tsd_boot1. */ 2167 tsd =3D tsd_fetch(); 2168 if (opt_background_thread) { 2169 assert(have_background_thread); 2170 /* 2171 * Need to finish init & unlock first before creating background 2172 * threads (pthread_create depends on malloc). ctl_init (which 2173 * sets isthreaded) needs to be called without holding any lock. 2174 */ 2175 background_thread_ctl_init(tsd_tsdn(tsd)); 2176 if (background_thread_create(tsd, 0)) { <----------------------------------------------------- 2177 return true; 2178 } 2179 } 2180 #undef UNLOCK_RETURN 2181 return false; 2182 } I'd be very glad for any aid to solve this issue. this is a complete callstack of running application. The crash occurs during C++ static initialization (e.g., std::locale). (lldb) thread backtrace all * thread #1, name =3D 'qdrant', stop reason =3D signal SIGSEGV * frame #0: 0x000039c7a57951a0 libthr.so.3`__Tthr_mutex_trylock(mutex=3D) at thr_mutex.c:628:3 frame #1: 0x000039bf7e0ae2d9 qdrant`malloc_mutex_trylock_final(mutex=3D0x000039bf83b2cdb0) at mutex.h:15= 7:9 frame #2: 0x000039bf7e0ac7e2 qdrant`malloc_mutex_lock(tsdn=3D0x000042c09c083720, mutex=3D0x000039bf83b2c= db0) at mutex.h:216:7 frame #3: 0x000039bf7e0ac6dc qdrant`_rjem_je_background_thread_create(tsd=3D0x000042c09c083720, arena_in= d=3D0) at background_thread.c:519:2 frame #4: 0x000039bf7e09bf6b qdrant`malloc_init_hard at jemalloc.c:2176= :7 frame #5: 0x000039bf7e05b170 qdrant`calloc [inlined] malloc_init at jemalloc.c:298:41 frame #6: 0x000039bf7e05b152 qdrant`calloc [inlined] imalloc_init_check(sopts=3D0x000039c7a4805558, dopts=3D0x000039c7a4805520) = at jemalloc.c:2658:41 frame #7: 0x000039bf7e05b139 qdrant`calloc [inlined] imalloc(sopts=3D0x000039c7a4805558, dopts=3D0x000039c7a4805520) at jemalloc.c:2689:32 frame #8: 0x000039bf7e057d0a qdrant`calloc(num=3D1, size=3D1664) at jemalloc.c:2852:2 frame #9: 0x000039c7a57933fe libthr.so.3`_thr_alloc(curthread=3D0x0000000000000000) at thr_list.c:154:12 frame #10: 0x000039c7a579242f libthr.so.3`_libpthread_init(curthread=3D0x0000000000000000) at thr_init.c:= 336:15 frame #11: 0x000039c7a5794b85 libthr.so.3`__Tthr_mutex_lock [inlined] _thr_check_init at thr_private.h:927:3 frame #12: 0x000039c7a5794b74 libthr.so.3`__Tthr_mutex_lock(mutex=3D0x000039c7a6729fc0) at thr_mutex.c:74= 9:2 frame #13: 0x000039c7a66bc0a4 libc++.so.1`std::__1::__call_once(unsigned long volatile&, void*, void (*)(void*)) [inlined] std::__1::__libcpp_mutex_lock[abi:se180100](__m=3D) at __threading_support:280:57 frame #14: 0x000039c7a66bc098 libc++.so.1`std::__1::__call_once(flag=3D0x000039c7a672aa70, arg=3D, func=3D) at call_once.cpp:44:3 frame #15: 0x000039c7a66f45d9 libc++.so.1`std::__1::locale::__imp::__imp(unsigned long) [inlined] void std::__1::call_once[abi:se180100](__fla= g=3D0x000039c7a672aa70, __func=3D0x000039c7a4807e10) at once_flag.h:131:5 frame #16: 0x000039c7a66f45a8 libc++.so.1`std::__1::locale::__imp::__imp(unsigned long) [inlined] std::__1::locale::id::__get(this=3D0x000039c7a672aa70) at locale.cpp:598:3 frame #17: 0x000039c7a66f459d libc++.so.1`std::__1::locale::__imp::__imp(unsigned long) [inlined] void std::__1::locale::__imp::install>(this=3D0x000039c7= a672abb0, f=3D) at locale.cpp:155:22 frame #18: 0x000039c7a66f459d libc++.so.1`std::__1::locale::__imp::__imp(this=3D0x000039c7a672abb0, refs= =3D1) at locale.cpp:163:3 frame #19: 0x000039c7a66f6023 libc++.so.1`std::__1::locale::classic() [inlined] std::__1::locale::__imp& std::__1::__no_destroy::__emplace[abi:se180100](this=3D0x000039c7a672abb0, __args=3D) at no_destroy.h:47:= 19 frame #20: 0x000039c7a66f600f libc++.so.1`std::__1::locale::classic() [inlined] std::__1::locale::classic()::$_0::operator()(this=3D) const at locale.cpp:504:40 frame #21: 0x000039c7a66f600f libc++.so.1`std::__1::locale::classic() at locale.cpp:502:69 frame #22: 0x000039c7a66f87ab libc++.so.1`std::__1::locale::locale() [inlined] std::__1::locale::__global() at locale.cpp:511:33 frame #23: 0x000039c7a66f8796 libc++.so.1`std::__1::locale::locale(this=3D0x000039c7a672a518) at locale.cpp:525:39 frame #24: 0x000039c7a66cd0f0 libc++.so.1`std::__1::basic_streambuf>::basic_streambuf(this=3D0x000039c7a672a510) at streambuf:293:35 frame #25: 0x000039c7a66daa48 libc++.so.1`std::__1::DoIOSInit::DoIOSIni= t() [inlined] std::__1::__stdinbuf::__stdinbuf(this=3D0x000039c7a672a510, __fp=3D0x000039c7a9b1b4e0, __st=3D) at std_stream.h:72:21 frame #26: 0x000039c7a66daa39 libc++.so.1`std::__1::DoIOSInit::DoIOSInit(this=3D) at iostream.cpp:139:57 frame #27: 0x000039c7a66db09d libc++.so.1`std::__1::ios_base::Init::Init(this=3D) at iostream.cpp:174:20 frame #28: 0x000039c7a66dbef5 libc++.so.1`_GLOBAL__I_000100 [inlined] __cxx_global_var_init at iostream_init.h:2:31 frame #29: 0x000039c7a66dbee4 libc++.so.1`_GLOBAL__I_000100 at iostream.cpp:0 frame #30: 0x000048ca6fe5d1fd ld-elf.so.1`objlist_call_init(list=3D0x000039c7a4808ee0, lockstate=3D0x000039c7a4808ce0) at rtld.c:3125:7 frame #31: 0x000048ca6fe5be19 ld-elf.so.1`_rtld(sp=3D, exit_proc=3D0x000039c7a4808f60, objp=3D0x000039c7a4808f68) at rtld.c:965:5 frame #32: 0x000048ca6fe58ea9 ld-elf.so.1`.rtld_start at rtld_start.S:40 Complete issue description on on github. https://github.com/qdrant/qdrant/issues/6433 --=20 You are receiving this mail because: You are the assignee for the bug.=