From nobody Thu Aug 21 12:11:00 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4c72Gj6Yzfz64nRP for ; Thu, 21 Aug 2025 12:11:09 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4c72Gj1PXtz3bRy for ; Thu, 21 Aug 2025 12:11:09 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=FfKpYam8; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of joh.hendriks@gmail.com designates 2a00:1450:4864:20::634 as permitted sender) smtp.mailfrom=joh.hendriks@gmail.com Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-afcb7a3a085so154515966b.2 for ; Thu, 21 Aug 2025 05:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755778262; x=1756383062; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=rCX3hb1i3BAf7BvmZyML3r0Xg124o4yVJ28fj4aZIFo=; b=FfKpYam8Ozwo2Z/3hvpSo+aevKX7DGUkXyUgPDQEmbHcjrXlVaN+vVyxj4Z4oUUE8c 5JLqYBELBFxiZ38a9fDNjB2gc2+2kWplJ+wqaVu3M7t3FQKwbd+oFVAU8RH1bD9ZMzrk 90tJqGt5eQBq3KehE8i4AcJj8oaNurroqv4IMWTEgL0N2svdjpx2G0HHh5PwORZ+sQ4H XefFXV0xA6NJ3Vu4pR04xKTMj60l8ky2HtLyUwvVW/pMr+cADio1fxisSo0GV+U8n2dl 0U+c6NXzy3VNzDT85UxZcaTaUJkuDRQtT/BHBloXvIKZf5mwIfwA1tP6KhSkjfo+if7A yidg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755778262; x=1756383062; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rCX3hb1i3BAf7BvmZyML3r0Xg124o4yVJ28fj4aZIFo=; b=AlA8pCaTjD8O01fdIfMR9WHnPRl7fa6AoDtVbK/OKzjkQQ7G3vcOOEXEC8LBoWPfza CSCJMULLm326TY+wUa0TVVL07tFQj9e4qV++Oc5B8XEx/y92btyM59LcRvOEzIyxJ0iD NBDYJ1ZUgrj2bEOpNo0rTXzo3c9snuXsifkAsTc+R8Hzy4thnPXZb8nLs0TPGi+rUqkN fOAB9BHlAPSomFXIEhPPB3u6WHyX+9K6KfUaCBWRI0E5/qhdaEJn2qIkclxHivGAQFlO K9Rq08+gjjsWSSj7NeQw7UGomp46NFYYVTGc4qVK8mQk5CtTLqNH2CmCTfJtYX5ZItCp y7Gg== X-Gm-Message-State: AOJu0YyhIFA6TSnvnIOwZgEzw36QyohWfv0BOfM+4obNUDrm2VxI7cEq iox4TvageqsaiMn+JBhsCx1ZimEWhc7bEzbm7Def9B+cWGM34f4HknJ7QnZnYgNj X-Gm-Gg: ASbGncvMwmvc256f/OFUdbYpDg5QkZgXeylGXCECfyIiiGKhMBBE1OF0keSl5WQGYrZ tdXqWXqUULKXb8IHQamBdGPFmTvZALq8V0ihSvobq7Q0XDXIXkiBmxKk35B9rMu2cJgS6fqs62P pO9darvvfKGWNRviNubLOzZgLhokEh7vJvgKMe4GERmbeQapc0ZbjkUKzeKGFvbtVo5d5gfwy3i IPU52Av5PBCk6W22+WvJs7pWxWfRrXP8eut8D40qz68uKviu7hVvCSQiFEuzBmLi7+rU+A4D/us HJDox5PlfE4ash17eJG05SA5pyMYIxR7nbdH7aL1LHYhL3DfzZU95MVWs8Tq1Ec657e9Xa2Ky8i JmrVupBo7CCE9+NmvndXGmG0Hnp/XjYrdCxIEhHYU3sYFYeRaUgDxkLPjQqWSHu9Mi9rTDPjhdb rYQDPG5js5N7bjlL2+diHrTA== X-Google-Smtp-Source: AGHT+IGKHXdQhJKD0XmbN+fKKwLqfLbSa8uSkosco6kBAaWdVN1ZqMMEELuJbCbwlq8RGoxc3HhAAw== X-Received: by 2002:a17:906:8448:b0:afd:bc2d:f77b with SMTP id a640c23a62f3a-afe07b81a4dmr186291366b.30.1755778262194; Thu, 21 Aug 2025 05:11:02 -0700 (PDT) Received: from [192.168.178.122] (80-115-35-171.cable.dynamic.v4.ziggo.nl. [80.115.35.171]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-afded2bc53dsm386370366b.7.2025.08.21.05.11.01 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 21 Aug 2025 05:11:01 -0700 (PDT) Message-ID: <67d49c1e-37d2-423d-a16a-d5e966ee8d66@gmail.com> Date: Thu, 21 Aug 2025 14:11:00 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: jemalloc 5.3.0 upgrade To: FreeBSD Current References: Content-Language: en-US From: Johan Hendriks In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::634:from] X-Rspamd-Queue-Id: 4c72Gj1PXtz3bRy On 8/15/25 11:56 PM, Warner Losh wrote: > After much delay, I've landed jemalloc 5.3.0 into main. > > This is likely the last update of jemalloc since the upstream is, at > best, in turmoil, and at worst dead. > > I tried to completely automate all the details of the upgrade, but > only got so far. I did the rest of the upgrade by hand (described in > FREEBSD-upgrade). I'd held off landing this until I had that, but once > it was clear this was likely the last time we'd need this, I just did > the last few steps by hand. I did this to make it easier to audit to > ensure that the pull request we got for this (which I redid, but > compared to the original) didn't sneak something in. Others can audit > me as well. > > I've run this with a netflix workload and my developer workload with > no regressions. > > Please let me know if this causes problems for anybody. I'm sure glad > I'll not have to rebase the merge again (it was a pathological case > for the instructions in the handbook, so I'll update those). > > I've been coordinating this with the release engineer for a while now, > who gave me a go ahead for landing this during the freeze since I > couldn't finish before my vacation last month... > > Warner > > P.S. Here's the release notes: > +* 5.3.0 (May 6, 2022) > + > +  This release contains many speed and space optimizations, from micro > +  optimizations on common paths to rework of internal data structures and > +  locking schemes, and many more too detailed to list below. Multiple > percent > +  of system level metric improvements were measured in tested production > +  workloads.  The release has gone through large-scale production > testing. > + > +  New features: > +  - Add the thread.idle mallctl which hints that the calling thread > will be > +    idle for a nontrivial period of time.  (@davidtgoldblatt) > +  - Allow small size classes to be the maximum size class to cache in the > +    thread-specific cache, through the opt.[lg_]tcache_max option. >  (@interwq, > +    @jordalgo) > +  - Make the behavior of realloc(ptr, 0) configurable with > opt.zero_realloc. > +    (@davidtgoldblatt) > +  - Add 'make uninstall' support.  (@sangshuduo, @Lapenkov) > +  - Support C++17 over-aligned allocation.  (@marksantaniello) > +  - Add the thread.peak mallctl for approximate per-thread peak > memory tracking. > +    (@davidtgoldblatt) > +  - Add interval-based stats output opt.stats_interval.  (@interwq) > +  - Add prof.prefix to override filename prefixes for dumps. >  (@zhxchen17) > +  - Add high resolution timestamp support for profiling.  (@tyroguru) > +  - Add the --collapsed flag to jeprof for flamegraph generation. > +    (@igorwwwwwwwwwwwwwwwwwwww) > +  - Add the --debug-syms-by-id option to jeprof for debug symbols > discovery. > +    (@DeannaGelbart) > +  - Add the opt.prof_leak_error option to exit with error code when > leak is > +    detected using opt.prof_final.  (@yunxuo) > +  - Add opt.cache_oblivious as an runtime alternative to > config.cache_oblivious. > +    (@interwq) > +  - Add mallctl interfaces: > +    + opt.zero_realloc  (@davidtgoldblatt) > +    + opt.cache_oblivious  (@interwq) > +    + opt.prof_leak_error  (@yunxuo) > +    + opt.stats_interval  (@interwq) > +    + opt.stats_interval_opts  (@interwq) > +    + opt.tcache_max  (@interwq) > +    + opt.trust_madvise  (@azat) > +    + prof.prefix  (@zhxchen17) > +    + stats.zero_reallocs  (@davidtgoldblatt) > +    + thread.idle  (@davidtgoldblatt) > +    + thread.peak.{read,reset}  (@davidtgoldblatt) > + > +  Bug fixes: > +  - Fix the synchronization around explicit tcache creation which > could cause > +    invalid tcache identifiers.  This regression was first released > in 5.0.0. > +    (@yoshinorim, @davidtgoldblatt) > +  - Fix a profiling biasing issue which could cause incorrect heap > usage and > +    object counts.  This issue existed in all previous releases with > the heap > +    profiling feature.  (@davidtgoldblatt) > +  - Fix the order of stats counter updating on large realloc which > could cause > +    failed assertions.  This regression was first released in 5.0.0. >  (@azat) > +  - Fix the locking on the arena destroy mallctl, which could cause > concurrent > +    arena creations to fail.  This functionality was first introduced > in 5.0.0. > +    (@interwq) > + > +  Portability improvements: > +  - Remove nothrow from system function declarations on macOS and > FreeBSD. > +    (@davidtgoldblatt, @fredemmott, @leres) > +  - Improve overcommit and page alignment settings on NetBSD.  (@zoulasc) > +  - Improve CPU affinity support on BSD platforms.  (@devnexen) > +  - Improve utrace detection and support.  (@devnexen) > +  - Improve QEMU support with MADV_DONTNEED zeroed pages detection. >  (@azat) > +  - Add memcntl support on Solaris / illumos.  (@devnexen) > +  - Improve CPU_SPINWAIT on ARM.  (@AWSjswinney) > +  - Improve TSD cleanup on FreeBSD.  (@Lapenkov) > +  - Disable percpu_arena if the CPU count cannot be reliably > detected.  (@azat) > +  - Add malloc_size(3) override support.  (@devnexen) > +  - Add mmap VM_MAKE_TAG support.  (@devnexen) > +  - Add support for MADV_[NO]CORE.  (@devnexen) > +  - Add support for DragonFlyBSD.  (@devnexen) > +  - Fix the QUANTUM setting on MIPS64.  (@brooksdavis) > +  - Add the QUANTUM setting for ARC.  (@vineetgarc) > +  - Add the QUANTUM setting for LoongArch.  (@wangjl-uos) > +  - Add QNX support.  (@jqian-aurora) > +  - Avoid atexit(3) calls unless the relevant profiling features are > enabled. > +    (@BusyJay, @laiwei-rice, @interwq) > +  - Fix unknown option detection when using Clang.  (@Lapenkov) > +  - Fix symbol conflict with musl libc.  (@georgthegreat) > +  - Add -Wimplicit-fallthrough checks.  (@nickdesaulniers) > +  - Add __forceinline support on MSVC.  (@santagada) > +  - Improve FreeBSD and Windows CI support.  (@Lapenkov) > +  - Add CI support for PPC64LE architecture.  (@ezeeyahoo) > + > +  Incompatible changes: > +  - Maximum size class allowed in tcache (opt.[lg_]tcache_max) now > has an upper > +    bound of 8MiB.  (@interwq) > + > +  Optimizations and refactors (@davidtgoldblatt, @Lapenkov, @interwq): > +  - Optimize the common cases of the thread cache operations. > +  - Optimize internal data structures, including RB tree and pairing > heap. > +  - Optimize the internal locking on extent management. > +  - Extract and refactor the internal page allocator and interface > modules. > + > +  Documentation: > +  - Fix doc build with --with-install-suffix.  (@lawmurray, @interwq) > +  - Add PROFILING_INTERNALS.md.  (@davidtgoldblatt) > +  - Ensure the proper order of doc building and installation. >  (@Mingli-Yu) > > Just a thank you for all your work. regards, Johan Hendriks