From nobody Fri Mar 6 15:26:21 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fS9H65p0Zz6Ty35 for ; Fri, 06 Mar 2026 15:26:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fS9H653cQz42tt for ; Fri, 06 Mar 2026 15:26:26 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772810786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PdbIvgaAtF7Usp30KptHEEw5j3Fd18wmFAYp7xmirP4=; b=fLRDQyr1x1ClJBr1+JBcJ33WhkVF28M0jU8fadrO6WSeD4vH4gxuQoOo+0mAVnDUG+SGB0 wK0qAFxwQJKasczArGLMqH9UgAqbVyG5ntvQXyADOCkdlfvoQRGLk0/6Orf/zX0ELYp9O/ UJUSDQo234NYmErZR5vFi2EfOJXQtgwWN+XqlVGTnfazRqZmLGnW/gqnTx4GKwL6BuZmA1 CnPwOnXvioyH/7tZwvJ/v5eqxZOKLTgWWV4eG0lZW7i6c1+HKFzCc0AtZkqVEMsfzcoFJX lrw+ukXwY25IsY5uPJWJRfXEMWw9vjn2WdH76tN6rj9hh31B9eCOi12f6xSFLQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1772810786; a=rsa-sha256; cv=none; b=Vel4H7SHKsBpJO/yLHLzjoaLa/KFX9OxBi6aqqF4o9nWVhC+igZ8K8N/XlFj+jh4ZEwNxt jVqCGX7OSgrTGgOzBTLKRVlSmO/K0xryRoNypp4YCQ4oClrgpGEYwlUjuQCESC/RAaw7JR pH+dnyIRx4xnTHmyS2QtgaotZUd6d9YUNgwgvfDkHutmVEZJcBP9cmXzA14aE5Xd/oFtU9 fZtfmG8hwriIMyj//AhzadQva+mSPDC1f4BmIGt3Mgx+nREFphm3b6VMCQpDQtNg7gKD3y 6ZA7pnsLX+HcPmr4pvlMaQy63QqwQ6Z7xiLcbSil7KCRrSM0xcR+dyr92aOrGw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772810786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PdbIvgaAtF7Usp30KptHEEw5j3Fd18wmFAYp7xmirP4=; b=f98gyyYXaAXOtcz11jWPV5J954FxoS7BE3rYM/m82O1vLCDaQ/eYDXGxalIhspC72Raisw inrD5LYi77pkLXsuXL1V8T77m3c0pUNP5hBV8bsaYTsFjIJDUB4C5y2b9ljrvyWmIMwpaa 4hdn8jEZpTPfPWh4W0Kdowjs+y4mL3fhFZnyu5HHRl8Ca83s5cc1OD8MBOzJ152/S2dRLW MQskajc5WfBb0vNdJqkfR+41+KZ6KaD2eznzUOPFj84ksZrZhP0t/2+Zpp2p2Mlx07H01K Ilzjn7HfZJ+UGJTjom8qkPxZWj3m03zAQ9NNgkgVsjBGiOJILY/4IQZgRrT5xg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4fS9H64fbSzCBb for ; Fri, 06 Mar 2026 15:26:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 31731 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 06 Mar 2026 15:26:21 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Gallatin Subject: git: f1e8b1aca543 - main - splice: optionally limit worker queues List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: gallatin X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f1e8b1aca54312a2c7f0d22a83aaae44950dac6b Auto-Submitted: auto-generated Date: Fri, 06 Mar 2026 15:26:21 +0000 Message-Id: <69aaf21d.31731.62be3dc2@gitrepo.freebsd.org> The branch main has been updated by gallatin: URL: https://cgit.FreeBSD.org/src/commit/?id=f1e8b1aca54312a2c7f0d22a83aaae44950dac6b commit f1e8b1aca54312a2c7f0d22a83aaae44950dac6b Author: Andrew Gallatin AuthorDate: 2026-03-06 14:59:35 +0000 Commit: Andrew Gallatin CommitDate: 2026-03-06 15:14:23 +0000 splice: optionally limit worker queues Add a new tunable/sysctl (kern.ipc.splice.num_wq) which can be used to limit the number of splice worker queues as a way to limit splice cpu use. The default (-1) keeps the current behavior of running one worker for each core in the system. An administrator can set it to 0 (either via tunable, or before the first splice call via sysctl) to effectively disable splice, or some number smaller than the number of cores to limit splice thread use. Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D55579 Sponsored by: Netflix --- sys/kern/uipc_socket.c | 50 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/sys/kern/uipc_socket.c b/sys/kern/uipc_socket.c index f7c339784693..45290c29f629 100644 --- a/sys/kern/uipc_socket.c +++ b/sys/kern/uipc_socket.c @@ -336,6 +336,33 @@ SYSCTL_BOOL(_kern_ipc_splice, OID_AUTO, receive_stream, CTLFLAG_RWTUN, &splice_receive_stream, 0, "Use soreceive_stream() for stream splices"); +static int splice_num_wq = -1; +static int +sysctl_splice_num_wq(SYSCTL_HANDLER_ARGS) +{ + int error, new; + + new = splice_num_wq; + error = sysctl_handle_int(oidp, &new, 0, req); + if (error == 0 && req->newptr && new != splice_num_wq) { + if (!cold) + sx_xlock(&splice_init_lock); + if (new < -1 || new > mp_ncpus || + (new <= 0 && splice_init_state != 0)) { + error = EINVAL; + } else { + splice_num_wq = new; + } + if (!cold) + sx_xunlock(&splice_init_lock); + } + return (error); +} +SYSCTL_PROC(_kern_ipc_splice, OID_AUTO, num_wq, + CTLTYPE_INT | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, + &splice_num_wq, 0, sysctl_splice_num_wq, "IU", + "Number of splice worker queues"); + static uma_zone_t splice_zone; static struct proc *splice_proc; struct splice_wq { @@ -447,24 +474,36 @@ splice_init(void) return (0); } + if (splice_num_wq == -1) { + /* if no user preference, use all cores */ + splice_num_wq = mp_ncpus; + } else if (splice_num_wq == 0) { + /* allow user to disable */ + splice_init_state = -1; + sx_xunlock(&splice_init_lock); + return (ENXIO); + } else if (splice_num_wq > mp_ncpus) { + splice_num_wq = mp_ncpus; + } + splice_zone = uma_zcreate("splice", sizeof(struct so_splice), NULL, NULL, splice_zinit, splice_zfini, UMA_ALIGN_CACHE, 0); - splice_wq = mallocarray(mp_maxid + 1, sizeof(*splice_wq), M_TEMP, + splice_wq = mallocarray(mp_ncpus, sizeof(*splice_wq), M_TEMP, M_WAITOK | M_ZERO); /* * Initialize the workqueues to run the splice work. We create a * work queue for each CPU. */ - CPU_FOREACH(i) { + for (i = 0; i < mp_ncpus; i++) { STAILQ_INIT(&splice_wq[i].head); mtx_init(&splice_wq[i].mtx, "splice work queue", NULL, MTX_DEF); } /* Start kthreads for each workqueue. */ error = 0; - CPU_FOREACH(i) { + for (i = 0; i < mp_ncpus; i++) { error = kproc_kthread_add(splice_work_thread, &splice_wq[i], &splice_proc, &td, 0, 0, "so_splice", "thr_%d", i); if (error) { @@ -1631,10 +1670,7 @@ so_splice_alloc(off_t max) sp->src = NULL; sp->dst = NULL; sp->max = max > 0 ? max : -1; - do { - sp->wq_index = atomic_fetchadd_32(&splice_index, 1) % - (mp_maxid + 1); - } while (CPU_ABSENT(sp->wq_index)); + sp->wq_index = atomic_fetchadd_32(&splice_index, 1) % splice_num_wq; sp->state = SPLICE_INIT; TIMEOUT_TASK_INIT(taskqueue_thread, &sp->timeout, 0, so_splice_timeout, sp);