From nobody Sat Apr 20 09:10:10 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VM5MV18hRz5HNHZ for ; Sat, 20 Apr 2024 09:10:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VM5MV0cc1z4FQN; Sat, 20 Apr 2024 09:10:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713604230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JgHpA8EGCsOur4D6Pb9TvPz9cGQlBO7x81zhqLm9O6I=; b=TRs2yeRp5coOrOQobCQG72AqsSIhufUb/dDBCfuiodFllhF22p6Rm2SPMXGVtxCufqA95q EeEM6TVPI+PejrePhyC5O8i7chZ9EUGvXt57cBunSUBlQz/4nd4AY6LisetRPfC0LS7IJg 2njUVy/yGxxciRB0LGUtsm64buksONnJieeRv01qccbzQS/fuuhFxi3E46JeYAaVhLkoY5 sjSj8tmCRGuViG1cfhkk3d9Xkb0CXIzbi5LslE1dzV99voICoglQZMRkeZhkqGqlMs3dHt Bdngv3e8B96bzX0bMvNIahgmY4Gf07mbIvm3BM0G9Dmbp5RR0tQbbA6OOg3CUQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713604230; a=rsa-sha256; cv=none; b=pOoCnv0v7u55QBvzv5pU8ToMIFRCuof4W5uCAiJydF6Jn7s6EUGnOJ2RqGCf0tlFdR2edg jagzwcxqj7rC1kVg05AvNPonZbD+Tbnx1WacEyth8CpoyC80HP4sO4KkRRR/0Ub/GbEyae NZsgZNfG+zEJxHxgYBZhvWikAqJHXB9QZjdMQeCqoEUZM9t66EFWB9lCEnsNGY7BwM2U3n s75P+uamSGop6o3BVi0f1HHQXfuV7l0JWMTAoVZm/35gLCqm0TBKVFrFyAt2imfxt9gRKh XkNjpmPGJsfndJcNyF5HkbXkdJh9VEjkHBjv2EKq7fWvBM0U1PJfvwDeia7ugQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713604230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JgHpA8EGCsOur4D6Pb9TvPz9cGQlBO7x81zhqLm9O6I=; b=XOLGOgCUAWx+BPqRHyaA6fN6zilDI6w5yl+naofDbX/PP+RyXDhZdOT5hdNEJHX/lz1H9o kpxtnkeNNjMAIpAVu4PKEdc7OmYSSxN5InrEKzOY/2ain4wmtVtbTFP8vc/P14oP9w+Zre D1V3oRkYvWDNCBCoyK8ujdN+XLlFyQQnnZ4QUthm92dN0SeDTvt93DN4Ywkyyu0t8WB7SR f4F65TaJxOQycha0AOuHS4+fv7qvCT5Ux4p/+Rvh9yWeaQ9n8rkzO7qz+Eu2CXFvmL0Dvi +RRACq4aUnIYxoiSZxRHR625xXAnRgDp6ISIws5YBwnAUnKLj2xrfnLS15BB+Q== Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4VM5MT6QNnz1Fxp; Sat, 20 Apr 2024 09:10:29 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 4C94562191; Sat, 20 Apr 2024 11:10:23 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: What does this error mean: No space available for static Thread Local Storage ? From: Juraj Lutter In-Reply-To: <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> Date: Sat, 20 Apr 2024 11:10:10 +0200 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <42774b55-241a-497b-816f-94b95187c3e6@FreeBSD.org> <6fd793ca-60ec-47e7-9ec9-4bcb56803426@FreeBSD.org> To: Yuri X-Mailer: Apple Mail (2.3774.500.171.1.1) > On 20 Apr 2024, at 10:37, Yuri wrote: >=20 > On 4/20/24 01:29, Gleb Popov wrote: >> This message probably comes from the library code. I'd start looking = there. >>=20 >=20 > No, it comes from the FreeBSD src tree. >=20 > The problem with this message is that it is cryptic, and doesn't lead = the user to a solution. When we=E2=80=99re at rust and memory (thread-local storage?) related: = I=E2=80=99ve tried to run databases/qdrant in production. It works until it=E2=80=99s idle. When some collections = are being created/loaded, it crashes. I have been able to track it down to malloc() called from within = strdup() called from within thr_set_name(). That internally calls, something like: frame #6: 0x00003f6e7cbc8955 = libc.so.7`__je_tsd_fetch_slow(tsd=3D0x00003f6faeb53090, = minimal=3D) at jemalloc_tsd.c:0 frame #7: 0x00003f6e7cb77e3a libc.so.7`__je_malloc_default [inlined] = tsd_fetch_impl(init=3Dtrue, minimal=3Dfalse) at tsd.h:355:10 frame #8: 0x00003f6e7cb77e30 libc.so.7`__je_malloc_default [inlined] = tsd_fetch at tsd.h:381:9 frame #9: 0x00003f6e7cb77e30 libc.so.7`__je_malloc_default [inlined] = imalloc(sopts=3D, dopts=3D) at = jemalloc_jemalloc.c:2256:15 frame #10: 0x00003f6e7cb77e30 libc.so.7`__je_malloc_default(size=3D40)= at jemalloc_jemalloc.c:2293:2 I can reliably reproduce the problem on both stable/13, stable/14 and = main. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org