From nobody Tue Aug 1 13:56:52 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RFc9W08y0z4qCjZ; Tue, 1 Aug 2023 13:56:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RFc9J3sj8z43dL; Tue, 1 Aug 2023 13:56:52 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1690898212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dULJ3LrMj9f0Wv6FzeZGehjhtLX8OlYEOxjx1Vg015U=; b=mTOlmXjXYjvacmOavS/fNhEO1lhn+4D9WxNv1gRb+18xGCQNC15Wz3wP9XyoJx5rSE8RUl m09SCK2CY8cm/NadlDb4IbEA1/birlMAQUWhAWR/pK8mRf2RX4LoON78EK8SO/zEQ23Klp U2RS3ajEDX+xMQxsfvhhEG9CQPF9z8Y0PLFbD53FNqf85Kg7sL9WkZcMPEXk1qZFhNncsN tb7jZOJwZj5bOPIvMvvGc+sJoR0MLjbvsAG2tqgA11AyID/K2cxfYOmdqfxURzi04X6Ya5 EflD/ljU1LJMuuBvcyIHNe6JopSEJOcp1P5vy44a7qVDrl4G6L/BUm/vFBrYDQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1690898212; a=rsa-sha256; cv=none; b=HgSczazGcTmZ9x7smZcYwwkClYFkSc5VolqjpBFjzx+EaR/MoojanAKOzGx1czgEOL61xz pO45ToyvpcnoMXDc6xi/4cZQwa0V04dBV/3KardzCCKTTbRf1oe+c+rJjig7N1sK2gzx/L u3SgeSt/Oz/lY044yx959FT8Llq0OZ1AgsFmAfAc+IZv1o6hHxxdVCrQ83tRQEL6aSLhOT +YvbrVT/nqd1TyWfC7yoFmIBieft04JKtWhoapRRjHope15SFmoUqwjxeurbTlTdlttsRU cPG5YWXHz2eu7s7Jqp+/+LyDjgb0wNHuSYIi0is9K/Bbd/afCdscXd0hsYLqtQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1690898212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dULJ3LrMj9f0Wv6FzeZGehjhtLX8OlYEOxjx1Vg015U=; b=Xw09Rooy5ntjotveD9lUkhSs7b+2MQXIIxY8CAUNH9YgcTADBaAts2yTGY6J3SmkFjEZqi c9PUYPpL9ZuyhffDqT+88Z6j/Fhj9qQcTWB0MRIutFG5YUHJBCnT30xJh4BgTsaWGyx6yd JP3MsSwjq6Qp65MGYapByuOX7ifcV4Fydxnm+mr9bT7aAn/ye1pq395ab5Eo4bikK4JfYp UNHXXYKMXUyVbmxlUbDYAx6hXinGEMPUyGYLRLryAHZo8GGlWRS/pZ6/2SP6DLP4eHI8jO Qa6tX2qGW8SylBJ+EUgdTT5qtegGK9efTDkYSCFj+TbOcxqHEfVtSM7S5JyvOA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RFc9J12clzg33; Tue, 1 Aug 2023 13:56:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 371DuqFj098221; Tue, 1 Aug 2023 13:56:52 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 371DuqWR098220; Tue, 1 Aug 2023 13:56:52 GMT (envelope-from git) Date: Tue, 1 Aug 2023 13:56:52 GMT Message-Id: <202308011356.371DuqWR098220@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: cee03fcb5e79 - stable/13 - buf: Make the number of pbufs slightly more dynamic List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: cee03fcb5e7933e9b50df498ac81e66bba24eb81 Auto-Submitted: auto-generated The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=cee03fcb5e7933e9b50df498ac81e66bba24eb81 commit cee03fcb5e7933e9b50df498ac81e66bba24eb81 Author: Mark Johnston AuthorDate: 2023-05-30 19:11:32 +0000 Commit: Mark Johnston CommitDate: 2023-08-01 13:56:16 +0000 buf: Make the number of pbufs slightly more dynamic Various subsystems pre-allocate a set of pbufs, allocated to implement I/O operations. pbuf allocations are transient, unlike most buf allocations. Most subsystems preallocate nswbuf or nswbuf/2 pbufs each. The preallocation ensures that pbuf allocation will succeed in low memory conditions, which might help avoid deadlocks. Currently we initialize nswbuf = min(nbuf / 4, 256). nbuf/4 > 256 on anything but the smallest systems. For example, nswbuf is 256 in a VM with 128MB of memory. In this configuration, a firecracker VM with one CPU preallocates over 900 pbufs. This consumes 2MB of RAM and adds several milliseconds to the kernel's (very small) boot time. Scale nswbuf by ncpu in the common case. I think this makes more sense than scaling by the amount of RAM, since pbuf allocations are transient and aren't used for caching. With the change, we get nswbuf=256 with 8 CPUs. With fewer than 8 CPUs we'll preallocate fewer pbufs than before, and with more we'll preallocate more. Event: BSDCan 2023 Reported by: cperciva Reviewed by: glebius, kib MFC after: 2 months Differential Revision: https://reviews.freebsd.org/D40216 (cherry picked from commit 4e78addbeff902aabaa87fdaafbd962f90720d69) --- sys/kern/vfs_bio.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c index bbde345a1b89..91cd42317a10 100644 --- a/sys/kern/vfs_bio.c +++ b/sys/kern/vfs_bio.c @@ -1157,7 +1157,12 @@ kern_vfs_bio_buffer_alloc(caddr_t v, long physmem_est) } if (nswbuf == 0) { - nswbuf = min(nbuf / 4, 256); + /* + * Pager buffers are allocated for short periods, so scale the + * number of reserved buffers based on the number of CPUs rather + * than amount of memory. + */ + nswbuf = min(nbuf / 4, 32 * mp_ncpus); if (nswbuf < NSWBUF_MIN) nswbuf = NSWBUF_MIN; }