From nobody Sat Aug 1 03:35:29 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hBpVV1McHz6mlHV for ; Sat, 01 Aug 2026 03:35:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hBpVT695cz3FNT for ; Sat, 01 Aug 2026 03:35:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785555329; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=360UFIr3nomCQDw+yOOxMPNh+5t39YE078ePJKLP97M=; b=fUagtWSU1m42YfAKy6uozVN4myIJf1iBalTJa+vIy8vx5pk79nWWbQQHNKa9eRYdfW2Bwb Kn44/XU4lCd3wmeVate2mAsXjW2BNrxaa4OtDsfqMBAwnI9tno1+MAcafKqZIVF/M20juK K4IMG/22O2mkI80AYdNgIW46bkxTM8t5ftTqXvySd6PJuSZG5PTT8I9oc0zhxOZNJq2i3s 0dyZNe1CGx+xGi32yw457fgZWHjhSYIOqOgk8BnAjSbC99meijXZ2gOgQIVOulo0NgKDfj JKSeX1PsnzoYneikqoybgaRmuhYQoLf4RCaRjxyYLPs8fAC6wZyh66TwSMZ1Jg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1785555329; a=rsa-sha256; cv=none; b=tDyzTc9mCpV+EsSzd3p4DpstYOsCoRiw7GjxKfp0Y6rpP5E6iY8ci+tiuxnXHF4IjgdICb 21lrjN8x6W7mRpbVS44NtjqZ9yCEQ2RkDSW3QF7qbYYVUxbpK5zWGjeOG8/ISOVI9t1bNu FTLDLRk3fUvqsDpbYeL7x3z0K5dKOk896X50hS5Cq6qLkrdNBK7Jy366Y6/VZKODofFID8 Q4QTyZ7l8Bpr2ZFuTFYilFIoNogDc40CjWrbunAVr5XLPNot5ic/N4GajVb9lK+vDZ4Jyq WyHRynIHQ2K7XTvNxNpyUIC0ewRDAOvihtZME5HDgNVOPIAJU0eXGBHk1ipgyw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785555329; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=360UFIr3nomCQDw+yOOxMPNh+5t39YE078ePJKLP97M=; b=ow5SkuXSnhojcLq8ZmkSXjPqYPv/JUSWBw/Tb5lC+SCYLgyY3+tBNC6a9ECnIfA0Cb8FD1 5vix2ScFmyvNTuehZX01L68OSrys9g3RK1v4IGKFVs7vpdWj2LlYG+jWPKZ6Cw4x8fr/35 ofc1W3JqeFfiqAiQcFHGZyjhEjLt0ybpi2qLXonTWWCCPYi7daxgjyrvO0iW3e0wxfYq2A P0Hhp6fNGKSWxINC6qc8NhZOzA3DXyfAuRUslxwDnKDgcylpM8rrFjdjvhLW+mAWvKVVo4 DIneNUgJJDivxSjFOBDud2xwWL7IjaKADz+9fJ+fybR2fTzRV/XV+Pzz46Y6Ew== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hBpVT4c3BzCln for ; Sat, 01 Aug 2026 03:35:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 367df by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sat, 01 Aug 2026 03:35:29 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kyle Evans Subject: git: a3a884c0d43a - main - stdio: *memstream: grow the buffer by 1.5x on write List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a3a884c0d43ab02187022be9ae9084e6c725ba68 Auto-Submitted: auto-generated Date: Sat, 01 Aug 2026 03:35:29 +0000 Message-Id: <6a6d6981.367df.741dd191@gitrepo.freebsd.org> The branch main has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=a3a884c0d43ab02187022be9ae9084e6c725ba68 commit a3a884c0d43ab02187022be9ae9084e6c725ba68 Author: Kyle Evans AuthorDate: 2026-08-01 03:34:37 +0000 Commit: Kyle Evans CommitDate: 2026-08-01 03:34:37 +0000 stdio: *memstream: grow the buffer by 1.5x on write This improves performance by reducing the number of allocations as we write into the memstream, both in the fully buffered case with larger memstreams and also more trivially in the line- and un-buffered case as they flush back to the underlying buffer more often. The inspiration for this was taken from Apple's implementation in https://github.com/apple-oss-distributions/libc, but expanded to include wmemstream for consistency. I've added a test for the bug that I hit in libder that caused me to notice this in the first place, and fixed that bug in this version. Reviewed by: des, jhb (both slightly previous version) Sponsored by: Klara, Inc. Differential Revision: https://reviews.freebsd.org/D57355 --- lib/libc/stdio/open_memstream.c | 21 +++++++++++----- lib/libc/stdio/open_wmemstream.c | 22 +++++++++++----- lib/libc/tests/stdio/open_memstream2_test.c | 39 +++++++++++++++++++++++++++++ lib/libc/tests/stdio/open_wmemstream_test.c | 38 ++++++++++++++++++++++++++++ 4 files changed, 108 insertions(+), 12 deletions(-) diff --git a/lib/libc/stdio/open_memstream.c b/lib/libc/stdio/open_memstream.c index d0a22bcfe59f..fbff50001cda 100644 --- a/lib/libc/stdio/open_memstream.c +++ b/lib/libc/stdio/open_memstream.c @@ -28,9 +28,11 @@ */ #include "namespace.h" +#include #include #include #include +#include #ifdef DEBUG #include #endif @@ -55,13 +57,20 @@ static int memstream_grow(struct memstream *ms, fpos_t newoff) { char *buf; - ssize_t newsize; + ssize_t newlen; if (newoff < 0 || newoff >= SSIZE_MAX) - newsize = SSIZE_MAX - 1; + newlen = SSIZE_MAX - 1; else - newsize = newoff; - if (newsize > ms->size) { + newlen = newoff; + if (newlen > ms->size) { + size_t newsize; + + if (ckd_add(&newsize, ms->size, ms->size / 2)) + newsize = SSIZE_MAX - 1; + + newsize = MAX(newsize, newlen); + buf = realloc(*ms->bufp, newsize + 1); if (buf == NULL) return (0); @@ -75,8 +84,8 @@ memstream_grow(struct memstream *ms, fpos_t newoff) ms->size = newsize; } - if (newsize > ms->len) - ms->len = newsize; + if (newlen > ms->len) + ms->len = newlen; return (1); } diff --git a/lib/libc/stdio/open_wmemstream.c b/lib/libc/stdio/open_wmemstream.c index d1844e573b1b..0cadf892023e 100644 --- a/lib/libc/stdio/open_wmemstream.c +++ b/lib/libc/stdio/open_wmemstream.c @@ -28,9 +28,11 @@ */ #include "namespace.h" +#include #include #include #include +#include #ifdef DEBUG #include #endif @@ -56,13 +58,20 @@ static int wmemstream_grow(struct wmemstream *ms, fpos_t newoff) { wchar_t *buf; - ssize_t newsize; + ssize_t newlen; if (newoff < 0 || newoff >= SSIZE_MAX / sizeof(wchar_t)) - newsize = SSIZE_MAX / sizeof(wchar_t) - 1; + newlen = SSIZE_MAX / sizeof(wchar_t) - 1; else - newsize = newoff; - if (newsize > ms->size) { + newlen = newoff; + if (newlen > ms->size) { + size_t newsize; + + if (ckd_add(&newsize, ms->size, ms->size / 2)) + newsize = SSIZE_MAX - 1; + + newsize = MAX(newsize, newlen); + buf = reallocarray(*ms->bufp, newsize + 1, sizeof(wchar_t)); if (buf == NULL) return (0); @@ -74,8 +83,9 @@ wmemstream_grow(struct wmemstream *ms, fpos_t newoff) *ms->bufp = buf; ms->size = newsize; } - if (newsize > ms->len) - ms->len = newsize; + + if (newlen > ms->len) + ms->len = newlen; return (1); } diff --git a/lib/libc/tests/stdio/open_memstream2_test.c b/lib/libc/tests/stdio/open_memstream2_test.c index c9c6528832d8..13df97f12dcd 100644 --- a/lib/libc/tests/stdio/open_memstream2_test.c +++ b/lib/libc/tests/stdio/open_memstream2_test.c @@ -184,7 +184,45 @@ ATF_TC_BODY(seek_tests, tc) SEEK_OK(-1, SEEK_END, 2); SEEK_OK(OFF_MAX - 1, SEEK_SET, OFF_MAX - 1); SEEK_FAIL(2, SEEK_CUR, EOVERFLOW); + + fclose(fp); +} + +ATF_TC(resize_test); +ATF_TC_HEAD(resize_test, tc) +{ + atf_tc_set_md_var(tc, "descr", + "Tests that memstream growth doesn't move SEEK_END too far"); +} +ATF_TC_BODY(resize_test, tc) +{ + FILE *fp; + size_t wsz; + char addition[] = "Test String"; + + fp = open_memstream(&buf, &len); + ATF_REQUIRE_MSG(fp != NULL, "open_memstream failed: %d", errno); + + /* Doesn't matter what we write, we'll just want to trigger alloc 2x. */ + wsz = fprintf(fp, "%s", addition); + ATF_REQUIRE_EQ(wsz, sizeof(addition) - 1); + ATF_REQUIRE(fflush(fp) == 0); + + /* Trigger growth. */ + ATF_REQUIRE_EQ('a', fputc('a', fp)); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + /* + * Seeking to the end shouldn't put us past the last write, because + * we've only done a small series of writes. + */ + ATF_REQUIRE(fseek(fp, 0, SEEK_END) == 0); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + ATF_REQUIRE(!ferror(fp)); fclose(fp); + + ATF_REQUIRE_EQ(wsz + 1, len); } ATF_TP_ADD_TCS(tp) @@ -193,6 +231,7 @@ ATF_TP_ADD_TCS(tp) ATF_TP_ADD_TC(tp, open_group_test); ATF_TP_ADD_TC(tp, simple_tests); ATF_TP_ADD_TC(tp, seek_tests); + ATF_TP_ADD_TC(tp, resize_test); return (atf_no_error()); } diff --git a/lib/libc/tests/stdio/open_wmemstream_test.c b/lib/libc/tests/stdio/open_wmemstream_test.c index 7ab882a4740a..3fce061e832f 100644 --- a/lib/libc/tests/stdio/open_wmemstream_test.c +++ b/lib/libc/tests/stdio/open_wmemstream_test.c @@ -187,12 +187,50 @@ ATF_TC_BODY(seek_tests, tc) fclose(fp); } +ATF_TC(resize_test); +ATF_TC_HEAD(resize_test, tc) +{ + atf_tc_set_md_var(tc, "descr", + "Tests that wmemstream growth doesn't move SEEK_END too far"); +} +ATF_TC_BODY(resize_test, tc) +{ + FILE *fp; + size_t wsz; + wchar_t addition[] = L"Test String"; + + fp = open_wmemstream(&buf, &len); + ATF_REQUIRE_MSG(fp != NULL, "open_memstream failed: %d", errno); + + /* Doesn't matter what we write, we'll just want to trigger alloc 2x. */ + wsz = fwprintf(fp, L"%ls", addition); + ATF_REQUIRE_EQ(wsz, wcslen(addition)); + ATF_REQUIRE(fflush(fp) == 0); + + /* Trigger growth. */ + ATF_REQUIRE_EQ(L'a', fputwc(L'a', fp)); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + /* + * Seeking to the end shouldn't put us past the last write, because + * we've only done a small series of writes. + */ + ATF_REQUIRE(fseek(fp, 0, SEEK_END) == 0); + ATF_REQUIRE_EQ(wsz + 1, ftello(fp)); + + ATF_REQUIRE(!ferror(fp)); + fclose(fp); + + ATF_REQUIRE_EQ(wsz + 1, len); +} + ATF_TP_ADD_TCS(tp) { ATF_TP_ADD_TC(tp, open_group_test); ATF_TP_ADD_TC(tp, simple_tests); ATF_TP_ADD_TC(tp, seek_tests); + ATF_TP_ADD_TC(tp, resize_test); return (atf_no_error()); }