From nobody Fri Mar 6 15:26:21 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fS9H14p8gz6TxcB for ; Fri, 06 Mar 2026 15:26:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fS9H148J2z42T1 for ; Fri, 06 Mar 2026 15:26:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772810781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PdbIvgaAtF7Usp30KptHEEw5j3Fd18wmFAYp7xmirP4=; b=qe49409pzEE155tZvaET3Dov+d7E4o5pwWCQQmcPcGSi83Gs28Nn7e4SgZkDY337tvxHyn NFd4xxa4d2lxBJkon2McT1QQibAMfVjRSQXLDwFitnKv3wx5UUUPu1U7d/W0SVDzzEwEzM JHS5xiZMVRz3nd1+u50KDgN2bjEsjs9rpzlhclKDiMnnWFqxSLXpmhRf8QXpA3CTpkRzhj RbN0mOWIx78EE7qY63vyw0lgbpSwzaXzluakfsTI0DxPAeuGaMgU9KqxiSXiTH7wyM5eh4 10GW7qGwpj5A38NvMEYtcHCDLZqYG+MW42Jn0w8B00D7rFG+LmR/AwgYtr8F2Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1772810781; a=rsa-sha256; cv=none; b=O6W5M3QdLbz0tTZQ0Z7vBQFeWpF53tk5Gwq+0ELsNSysiJkLXctaWC6w8OUB/dRMG1HA5m ANmvRT/kmbotVZvCkik0UBwmJpmfjP0dlKlLt/+49ENnCrURJclWh1CtwRrxEu3RvZos8u pi2d0Jwm1Qt6eYgFYnBECKPL8OpCrB10SIv2I9wWT3gfUq077r1Plfi79fhP7B2EpbRO/l UirogxT0E3HKDV6NErdDFx+Qdhc6dBmMgOerW4u7tv27CTHYy9wTWdkFeV+27yoipRhbPm drMVwNmTXa1vD5Zra/sgI/70KPbwXzI9+Zc78g6hVVryonD0m83W7FZUX2Dg0A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772810781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PdbIvgaAtF7Usp30KptHEEw5j3Fd18wmFAYp7xmirP4=; b=IznEFC34j9aJS7fMZf7vlRk0lfvRl6G7PTbl7wmZm5QAIpvPxGX2+ODLranoBzpY27tNu6 CLagQ34ll9uCWBQBK3ptN46fb16uYDuCx1bPZ3OHIt84hkkXqfD6JY2vlhUj1QsbyX7AaM mxFnZ993mZqMD3E9HCErS4GoHA9UizB8xgeknui0XFrBRpMLkabqC9IviELDZYE48v/xnh UVnKTna+FvxR8uJbeDCRFGf0lkqS9uztv+d3FSNcnCc0oV+L+sWrzmCGnzMlQp0Qc27VKt 1yQjdFc0DqXpR7Aa2+8wiBhTe+iNb8XkoPJ6qRi8Db6HhnAPuZop65mT/bK5SA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4fS9H13lBNzCdv for ; Fri, 06 Mar 2026 15:26:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 31731 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 06 Mar 2026 15:26:21 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Gallatin Subject: git: f1e8b1aca543 - main - splice: optionally limit worker queues List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: gallatin X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f1e8b1aca54312a2c7f0d22a83aaae44950dac6b Auto-Submitted: auto-generated Date: Fri, 06 Mar 2026 15:26:21 +0000 Message-Id: <69aaf21d.31731.62be3dc2@gitrepo.freebsd.org> The branch main has been updated by gallatin: URL: https://cgit.FreeBSD.org/src/commit/?id=f1e8b1aca54312a2c7f0d22a83aaae44950dac6b commit f1e8b1aca54312a2c7f0d22a83aaae44950dac6b Author: Andrew Gallatin AuthorDate: 2026-03-06 14:59:35 +0000 Commit: Andrew Gallatin CommitDate: 2026-03-06 15:14:23 +0000 splice: optionally limit worker queues Add a new tunable/sysctl (kern.ipc.splice.num_wq) which can be used to limit the number of splice worker queues as a way to limit splice cpu use. The default (-1) keeps the current behavior of running one worker for each core in the system. An administrator can set it to 0 (either via tunable, or before the first splice call via sysctl) to effectively disable splice, or some number smaller than the number of cores to limit splice thread use. Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D55579 Sponsored by: Netflix --- sys/kern/uipc_socket.c | 50 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/sys/kern/uipc_socket.c b/sys/kern/uipc_socket.c index f7c339784693..45290c29f629 100644 --- a/sys/kern/uipc_socket.c +++ b/sys/kern/uipc_socket.c @@ -336,6 +336,33 @@ SYSCTL_BOOL(_kern_ipc_splice, OID_AUTO, receive_stream, CTLFLAG_RWTUN, &splice_receive_stream, 0, "Use soreceive_stream() for stream splices"); +static int splice_num_wq = -1; +static int +sysctl_splice_num_wq(SYSCTL_HANDLER_ARGS) +{ + int error, new; + + new = splice_num_wq; + error = sysctl_handle_int(oidp, &new, 0, req); + if (error == 0 && req->newptr && new != splice_num_wq) { + if (!cold) + sx_xlock(&splice_init_lock); + if (new < -1 || new > mp_ncpus || + (new <= 0 && splice_init_state != 0)) { + error = EINVAL; + } else { + splice_num_wq = new; + } + if (!cold) + sx_xunlock(&splice_init_lock); + } + return (error); +} +SYSCTL_PROC(_kern_ipc_splice, OID_AUTO, num_wq, + CTLTYPE_INT | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, + &splice_num_wq, 0, sysctl_splice_num_wq, "IU", + "Number of splice worker queues"); + static uma_zone_t splice_zone; static struct proc *splice_proc; struct splice_wq { @@ -447,24 +474,36 @@ splice_init(void) return (0); } + if (splice_num_wq == -1) { + /* if no user preference, use all cores */ + splice_num_wq = mp_ncpus; + } else if (splice_num_wq == 0) { + /* allow user to disable */ + splice_init_state = -1; + sx_xunlock(&splice_init_lock); + return (ENXIO); + } else if (splice_num_wq > mp_ncpus) { + splice_num_wq = mp_ncpus; + } + splice_zone = uma_zcreate("splice", sizeof(struct so_splice), NULL, NULL, splice_zinit, splice_zfini, UMA_ALIGN_CACHE, 0); - splice_wq = mallocarray(mp_maxid + 1, sizeof(*splice_wq), M_TEMP, + splice_wq = mallocarray(mp_ncpus, sizeof(*splice_wq), M_TEMP, M_WAITOK | M_ZERO); /* * Initialize the workqueues to run the splice work. We create a * work queue for each CPU. */ - CPU_FOREACH(i) { + for (i = 0; i < mp_ncpus; i++) { STAILQ_INIT(&splice_wq[i].head); mtx_init(&splice_wq[i].mtx, "splice work queue", NULL, MTX_DEF); } /* Start kthreads for each workqueue. */ error = 0; - CPU_FOREACH(i) { + for (i = 0; i < mp_ncpus; i++) { error = kproc_kthread_add(splice_work_thread, &splice_wq[i], &splice_proc, &td, 0, 0, "so_splice", "thr_%d", i); if (error) { @@ -1631,10 +1670,7 @@ so_splice_alloc(off_t max) sp->src = NULL; sp->dst = NULL; sp->max = max > 0 ? max : -1; - do { - sp->wq_index = atomic_fetchadd_32(&splice_index, 1) % - (mp_maxid + 1); - } while (CPU_ABSENT(sp->wq_index)); + sp->wq_index = atomic_fetchadd_32(&splice_index, 1) % splice_num_wq; sp->state = SPLICE_INIT; TIMEOUT_TASK_INIT(taskqueue_thread, &sp->timeout, 0, so_splice_timeout, sp);