From nobody Fri Aug 7 01:48:20 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hGRr53GlTz6ndlT for ; Fri, 07 Aug 2026 01:48:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hGRr50gBWz3pQk for ; Fri, 07 Aug 2026 01:48:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786067301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=P9GhrEs9tS+IoJc9rvQhDkHXDu5XxTL2VHBfv3K+sjU=; b=geGmZArqp2hX2dallJufEh2eReWQgHpla4oZiXpHO9Au5j5q2+urrAXXf+dBKHYqbuiZak yHKJzOgwksQj1oSJ7fTXPWszLKBak4AfFKLY2BbedgenZYz+RkcTX70C7MsaAp6QRIe6nD blJ8+D0fSLC4o1YXeCsJtBbaqtlZotPrDCuiYv9zzCN0iuK1fGxl57DdWX/wo5Qq9sueZj Bx+a0gMCadLNl9v4nNJmlfijgiEoQhzxAwSx4rJIctR8jv3eULUPzljJUmr8SKOT19bDj1 tpdqls6R/nHrh+Y9m1iV2vM+DfSOlNvVYB1Yrwcqh21RJHzMBfT2BL1Bjp6DPQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786067301; a=rsa-sha256; cv=none; b=rHi68iSK2zN1ccT46CkJwRKg0LLuGK4wd0+01zLhvt0C6aZbt3kKh2rLgJhI4DHT+eyvV0 tnUjApHzzmTnC4MdBADoWr0xQl1X0Z53fsschjz0jMBs255jcZk8QEgM6CWxR16CEoJOqM dkWeUVkF/HRKTWZDs/246Y2XX9iARnUxB7brhEoRreSOoneddQjUgWS5QEhAGkyDES8cIb 5cQPWYJ2aE+xOTWQ99egIP9ac5/geQ4xdRiKO3WnXVCoMWTDnGCgtnd8Pefcm90agPFTyK eWMJ5HVLqKrNJwMJV1MIddqt66tGCDiHvR8WzWLZTCaXPz56r3CZ6iPcs2F6lw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786067301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=P9GhrEs9tS+IoJc9rvQhDkHXDu5XxTL2VHBfv3K+sjU=; b=Ey27hb2Xl0urkNq3s/vzfO9Xv0I6s/xp3J44JW5llYri6A2EZLn0E8Gkt6VYq0ZnPuu+rc /GyfMSzrYxItHIBCgp6zZajfwAHKgZusUmS/9HelLRP9sUt58SAglSkJdK4ZcqKELEghJG PzfaZb93/9U9qpeZQ0rU7Ws1CX/vD+lLcLcrGZTiCiTvPtT07U/nbw2W9fPEojjdZHxXHv BAPNHxYSSYwVGdlJWAcSL0fwUvmAsVDM15+wj/HvFBoT51ZPW+T2YlMrWHKUQu/TjAuBXh Z6xVOcscp7yYSn+caUuIOfpZWRaP3J9Vqot046XzQP8wO9D74pv9wasgShXleg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hGRr46d7Lzmnt for ; Fri, 07 Aug 2026 01:48:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3bba7 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 07 Aug 2026 01:48:20 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kyle Evans Subject: git: 6ce4a4dea4df - stable/15 - stdio: *memstream: grow the buffer by 1.5x on write List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 6ce4a4dea4df18c1d91d4d5f2d98aa2ff54285d3 Auto-Submitted: auto-generated Date: Fri, 07 Aug 2026 01:48:20 +0000 Message-Id: <6a753964.3bba7.74d798e7@gitrepo.freebsd.org> The branch stable/15 has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=6ce4a4dea4df18c1d91d4d5f2d98aa2ff54285d3 commit 6ce4a4dea4df18c1d91d4d5f2d98aa2ff54285d3 Author: Kyle Evans AuthorDate: 2026-08-01 03:34:37 +0000 Commit: Kyle Evans CommitDate: 2026-08-06 23:37:49 +0000 stdio: *memstream: grow the buffer by 1.5x on write This improves performance by reducing the number of allocations as we write into the memstream, both in the fully buffered case with larger memstreams and also more trivially in the line- and un-buffered case as they flush back to the underlying buffer more often. The inspiration for this was taken from Apple's implementation in https://github.com/apple-oss-distributions/libc, but expanded to include wmemstream for consistency. I've added a test for the bug that I hit in libder that caused me to notice this in the first place, and fixed that bug in this version. Reviewed by: des, jhb (both slightly previous version) Sponsored by: Klara, Inc. (cherry picked from commit a3a884c0d43ab02187022be9ae9084e6c725ba68) --- lib/libc/stdio/open_memstream.c | 21 +++++++++++----- lib/libc/stdio/open_wmemstream.c | 22 +++++++++++----- lib/libc/tests/stdio/open_memstream2_test.c | 39 +++++++++++++++++++++++++++++ lib/libc/tests/stdio/open_wmemstream_test.c | 38 ++++++++++++++++++++++++++++ 4 files changed, 108 insertions(+), 12 deletions(-) diff --git a/lib/libc/stdio/open_memstream.c b/lib/libc/stdio/open_memstream.c index d0a22bcfe59f..fbff50001cda 100644 --- a/lib/libc/stdio/open_memstream.c +++ b/lib/libc/stdio/open_memstream.c @@ -28,9 +28,11 @@ */ #include "namespace.h" +#include #include #include #include +#include #ifdef DEBUG #include #endif @@ -55,13 +57,20 @@ static int memstream_grow(struct memstream *ms, fpos_t newoff) { char *buf; - ssize_t newsize; + ssize_t newlen; if (newoff < 0 || newoff >= SSIZE_MAX) - newsize = SSIZE_MAX - 1; + newlen = SSIZE_MAX - 1; else - newsize = newoff; - if (newsize > ms->size) { + newlen = newoff; + if (newlen > ms->size) { + size_t newsize; + + if (ckd_add(&newsize, ms->size, ms->size / 2)) + newsize = SSIZE_MAX - 1; + + newsize = MAX(newsize, newlen); + buf = realloc(*ms->bufp, newsize + 1); if (buf == NULL) return (0); @@ -75,8 +84,8 @@ memstream_grow(struct memstream *ms, fpos_t newoff) ms->size = newsize; } - if (newsize > ms->len) - ms->len = newsize; + if (newlen > ms->len) + ms->len = newlen; return (1); } diff --git a/lib/libc/stdio/open_wmemstream.c b/lib/libc/stdio/open_wmemstream.c index d1844e573b1b..0cadf892023e 100644 --- a/lib/libc/stdio/open_wmemstream.c +++ b/lib/libc/stdio/open_wmemstream.c @@ -28,9 +28,11 @@ */ #include "namespace.h" +#include #include #include #include +#include #ifdef DEBUG #include #endif @@ -56,13 +58,20 @@ static int wmemstream_grow(struct wmemstream *ms, fpos_t newoff) { wchar_t *buf; - ssize_t newsize; + ssize_t newlen; if (newoff < 0 || newoff >= SSIZE_MAX / sizeof(wchar_t)) - newsize = SSIZE_MAX / sizeof(wchar_t) - 1; + newlen = SSIZE_MAX / sizeof(wchar_t) - 1; else - newsize = newoff; - if (newsize > ms->size) { + newlen = newoff; + if (newlen > ms->size) { + size_t newsize; + + if (ckd_add(&newsize, ms->size, ms->size / 2)) + newsize = SSIZE_MAX - 1; + + newsize = MAX(newsize, newlen); + buf = reallocarray(*ms->bufp, newsize + 1, sizeof(wchar_t)); if (buf == NULL) return (0); @@ -74,8 +83,9 @@ wmemstream_grow(struct wmemstream *ms, fpos_t newoff) *ms->bufp = buf; ms->size = newsize; } - if (newsize > ms->len) - ms->len = newsize; + + if (newlen > ms->len) + ms->len = newlen; return (1); } diff --git a/lib/libc/tests/stdio/open_memstream2_test.c b/lib/libc/tests/stdio/open_memstream2_test.c index c9c6528832d8..13df97f12dcd 100644 --- a/lib/libc/tests/stdio/open_memstream2_test.c +++ b/lib/libc/tests/stdio/open_memstream2_test.c @@ -184,7 +184,45 @@ ATF_TC_BODY(seek_tests, tc) SEEK_OK(-1, SEEK_END, 2); SEEK_OK(OFF_MAX - 1, SEEK_SET, OFF_MAX - 1); SEEK_FAIL(2, SEEK_CUR, EOVERFLOW); + + fclose(fp); +} + +ATF_TC(resize_test); +ATF_TC_HEAD(resize_test, tc) +{ + atf_tc_set_md_var(tc, "descr", + "Tests that memstream growth doesn't move SEEK_END too far"); +} +ATF_TC_BODY(resize_test, tc) +{ + FILE *fp; + size_t wsz; + char addition[] = "Test String"; + + fp = open_memstream(&buf, &len); + ATF_REQUIRE_MSG(fp != NULL, "open_memstream failed: %d", errno); + + /* Doesn't matter what we write, we'll just want to trigger alloc 2x. */ + wsz = fprintf(fp, "%s", addition); + ATF_REQUIRE_EQ(wsz, sizeof(addition) - 1); + ATF_REQUIRE(fflush(fp) == 0); + + /* Trigger growth. */ + ATF_REQUIRE_EQ('a', fputc('a', fp)); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + /* + * Seeking to the end shouldn't put us past the last write, because + * we've only done a small series of writes. + */ + ATF_REQUIRE(fseek(fp, 0, SEEK_END) == 0); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + ATF_REQUIRE(!ferror(fp)); fclose(fp); + + ATF_REQUIRE_EQ(wsz + 1, len); } ATF_TP_ADD_TCS(tp) @@ -193,6 +231,7 @@ ATF_TP_ADD_TCS(tp) ATF_TP_ADD_TC(tp, open_group_test); ATF_TP_ADD_TC(tp, simple_tests); ATF_TP_ADD_TC(tp, seek_tests); + ATF_TP_ADD_TC(tp, resize_test); return (atf_no_error()); } diff --git a/lib/libc/tests/stdio/open_wmemstream_test.c b/lib/libc/tests/stdio/open_wmemstream_test.c index 7ab882a4740a..3fce061e832f 100644 --- a/lib/libc/tests/stdio/open_wmemstream_test.c +++ b/lib/libc/tests/stdio/open_wmemstream_test.c @@ -187,12 +187,50 @@ ATF_TC_BODY(seek_tests, tc) fclose(fp); } +ATF_TC(resize_test); +ATF_TC_HEAD(resize_test, tc) +{ + atf_tc_set_md_var(tc, "descr", + "Tests that wmemstream growth doesn't move SEEK_END too far"); +} +ATF_TC_BODY(resize_test, tc) +{ + FILE *fp; + size_t wsz; + wchar_t addition[] = L"Test String"; + + fp = open_wmemstream(&buf, &len); + ATF_REQUIRE_MSG(fp != NULL, "open_memstream failed: %d", errno); + + /* Doesn't matter what we write, we'll just want to trigger alloc 2x. */ + wsz = fwprintf(fp, L"%ls", addition); + ATF_REQUIRE_EQ(wsz, wcslen(addition)); + ATF_REQUIRE(fflush(fp) == 0); + + /* Trigger growth. */ + ATF_REQUIRE_EQ(L'a', fputwc(L'a', fp)); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + /* + * Seeking to the end shouldn't put us past the last write, because + * we've only done a small series of writes. + */ + ATF_REQUIRE(fseek(fp, 0, SEEK_END) == 0); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + ATF_REQUIRE(!ferror(fp)); + fclose(fp); + + ATF_REQUIRE_EQ(wsz + 1, len); +} + ATF_TP_ADD_TCS(tp) { ATF_TP_ADD_TC(tp, open_group_test); ATF_TP_ADD_TC(tp, simple_tests); ATF_TP_ADD_TC(tp, seek_tests); + ATF_TP_ADD_TC(tp, resize_test); return (atf_no_error()); }