Date: Tue, 1 Aug 2023 13:56:52 GMT From: Mark Johnston <markj@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Subject: git: cee03fcb5e79 - stable/13 - buf: Make the number of pbufs slightly more dynamic Message-ID: <202308011356.371DuqWR098220@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=cee03fcb5e7933e9b50df498ac81e66bba24eb81 commit cee03fcb5e7933e9b50df498ac81e66bba24eb81 Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2023-05-30 19:11:32 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2023-08-01 13:56:16 +0000 buf: Make the number of pbufs slightly more dynamic Various subsystems pre-allocate a set of pbufs, allocated to implement I/O operations. pbuf allocations are transient, unlike most buf allocations. Most subsystems preallocate nswbuf or nswbuf/2 pbufs each. The preallocation ensures that pbuf allocation will succeed in low memory conditions, which might help avoid deadlocks. Currently we initialize nswbuf = min(nbuf / 4, 256). nbuf/4 > 256 on anything but the smallest systems. For example, nswbuf is 256 in a VM with 128MB of memory. In this configuration, a firecracker VM with one CPU preallocates over 900 pbufs. This consumes 2MB of RAM and adds several milliseconds to the kernel's (very small) boot time. Scale nswbuf by ncpu in the common case. I think this makes more sense than scaling by the amount of RAM, since pbuf allocations are transient and aren't used for caching. With the change, we get nswbuf=256 with 8 CPUs. With fewer than 8 CPUs we'll preallocate fewer pbufs than before, and with more we'll preallocate more. Event: BSDCan 2023 Reported by: cperciva Reviewed by: glebius, kib MFC after: 2 months Differential Revision: https://reviews.freebsd.org/D40216 (cherry picked from commit 4e78addbeff902aabaa87fdaafbd962f90720d69) --- sys/kern/vfs_bio.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c index bbde345a1b89..91cd42317a10 100644 --- a/sys/kern/vfs_bio.c +++ b/sys/kern/vfs_bio.c @@ -1157,7 +1157,12 @@ kern_vfs_bio_buffer_alloc(caddr_t v, long physmem_est) } if (nswbuf == 0) { - nswbuf = min(nbuf / 4, 256); + /* + * Pager buffers are allocated for short periods, so scale the + * number of reserved buffers based on the number of CPUs rather + * than amount of memory. + */ + nswbuf = min(nbuf / 4, 32 * mp_ncpus); if (nswbuf < NSWBUF_MIN) nswbuf = NSWBUF_MIN; }
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202308011356.371DuqWR098220>