From nobody Thu Jul 9 13:41:45 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gwx2f0Szmz6lDVk for ; Thu, 09 Jul 2026 13:41:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gwx2d3Xjpz46y9 for ; Thu, 09 Jul 2026 13:41:45 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1783604505; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=r882wySeNafvsUiVZ2S+qOqE8jB2U66EkAblhVbnLic=; b=ANLaffIzv/xN2i6X5bl8XEjZYGY1FbKQkY9mCacEUIhF7cdWux/YJLjJerV35aQgHs892b 5m8rpr5Tcx5fEOhACOWWy2v3TmMAs78Sw2H6xOj/2wx5NupcNuvqxWOXEISlybsiiWjBDg VW7MASS1gifGXpHWE/ZmOaDmR0w/PSJL+28mvbrZSqFl97+Kamm+3vx6IuJEAqgzaNm/XP GNJ65MZyM3nzWhqx95I9GMSJM/Xnq2uS9glLkGk2gUvrBxNvLvK/qn8PsGiZMh8gJng7/S GX9oIzWv12Ha+VMPDAw6nXzv7d7pqvXY8MPXieOs3s6CYaicVmrSO7tllHxZDA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1783604505; a=rsa-sha256; cv=none; b=QHTY2OtQQ8vVBbaU36sJh+Off0EDlo79mtcdxx9wXugy3jfDO3hvlwG/+bvamX79UzvPi0 wXSDendtq7AJqpNd/WpBJKUG2leEcQSdwjfGfxZ2YuH9rdnK4A7IdlBgp7G9iFEWQXi1Nu dqP8d/7cuADwOsXYvUzapiBo7mTvz1zmZfzJRPphBdnfSv36aHD33BAzmfvpRJkDT4nak3 mJHLt/M8nnQrFQm/yLh23KX7HG1/UWG3kXT+NGyGpyi0ht+kyTLdtQM4DyJEeKhjiCgxH7 CgCnlM+tnP4zDHzLHREXOcywUBaPnfSke1YE65ELQWidqYkJfcX18h78gIDfUA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1783604505; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=r882wySeNafvsUiVZ2S+qOqE8jB2U66EkAblhVbnLic=; b=bijxHEt4EHK1YDCaNAEpnZLP33ru4m+lbCm29bffLUouVC6EmTRbqgYyJqxQN6H/RrJdim uCP2eSGyBg+CMbx8tXKoB2Vd3QDrTus3tvmO+7bTpzotnACDcOQs3qJJvQvbkyqKDN9rcl qm0KIsLpMddjKW+iyWaND3GzVlyZ9WGAxErPAJi2Itxt0YJrWBq/VicXXJfWrl3+1FycmT eAl3Pr4xTYkhfVt4qhYGjagAk+zmFD/qnnGNT51YHADdy7xeeYNjeZOoq2QhQ2FXAex3uk xgRGtlnXZEVKRL8Qi9JcEPjb/B64v7sT8+nKhS1lmXqu759CC1UMKmpXHPrHlQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4gwx2d2G0Zz18jT for ; Thu, 09 Jul 2026 13:41:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1f2ac by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 09 Jul 2026 13:41:45 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: a58590631ccc - main - taskqueue: Avoid unbounded epoch read sections List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a58590631ccc0fa5bdbbdf88021c6878d644d128 Auto-Submitted: auto-generated Date: Thu, 09 Jul 2026 13:41:45 +0000 Message-Id: <6a4fa519.1f2ac.1b16199f@gitrepo.freebsd.org> The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=a58590631ccc0fa5bdbbdf88021c6878d644d128 commit a58590631ccc0fa5bdbbdf88021c6878d644d128 Author: Mark Johnston AuthorDate: 2026-07-08 17:12:54 +0000 Commit: Mark Johnston CommitDate: 2026-07-09 13:41:27 +0000 taskqueue: Avoid unbounded epoch read sections The taskqueue thread loop tries to avoid entering and exiting net epoch read sections for every task. This reduces the overhead of net epoch integration, but the implementation wasn't bounding the length of the read section, so a busy taskqueue thread could hold an epoch open for an unbounded period. This is easy to achieve with the epair task, for instance. Bound the number of tasks that we'll execute without observing the global epoch, and provide a sysctl to control it. Let the default bound be eight. Reviewed by: glebius MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D58031 --- sys/kern/subr_gtaskqueue.c | 20 ++++++++++++-------- sys/kern/subr_taskqueue.c | 34 ++++++++++++++++++++++++++-------- sys/sys/taskqueue.h | 2 ++ 3 files changed, 40 insertions(+), 16 deletions(-) diff --git a/sys/kern/subr_gtaskqueue.c b/sys/kern/subr_gtaskqueue.c index e13721ffe1f2..4b0600101b64 100644 --- a/sys/kern/subr_gtaskqueue.c +++ b/sys/kern/subr_gtaskqueue.c @@ -342,14 +342,14 @@ gtaskqueue_run_locked(struct gtaskqueue *queue) struct epoch_tracker et; struct gtaskqueue_busy tb; struct gtask *gtask; - bool in_net_epoch; + unsigned int epochtasks; KASSERT(queue != NULL, ("tq is NULL")); TQ_ASSERT_LOCKED(queue); tb.tb_running = NULL; LIST_INSERT_HEAD(&queue->tq_active, &tb, tb_link); - in_net_epoch = false; + epochtasks = 0; while ((gtask = STAILQ_FIRST(&queue->tq_queue)) != NULL) { STAILQ_REMOVE_HEAD(&queue->tq_queue, ta_link); gtask->ta_flags &= ~TASK_ENQUEUED; @@ -358,19 +358,23 @@ gtaskqueue_run_locked(struct gtaskqueue *queue) TQ_UNLOCK(queue); KASSERT(gtask->ta_func != NULL, ("task->ta_func is NULL")); - if (!in_net_epoch && TASK_IS_NET(gtask)) { - in_net_epoch = true; - NET_EPOCH_ENTER(et); - } else if (in_net_epoch && !TASK_IS_NET(gtask)) { + if (TASK_IS_NET(gtask)) { + if (epochtasks++ == 0) + NET_EPOCH_ENTER(et); + } else if (epochtasks > 0) { NET_EPOCH_EXIT(et); - in_net_epoch = false; + epochtasks = 0; } gtask->ta_func(gtask->ta_context); + if (epochtasks > net_epoch_task_limit) { + NET_EPOCH_EXIT(et); + epochtasks = 0; + } TQ_LOCK(queue); wakeup(gtask); } - if (in_net_epoch) + if (epochtasks > 0) NET_EPOCH_EXIT(et); LIST_REMOVE(&tb, tb_link); } diff --git a/sys/kern/subr_taskqueue.c b/sys/kern/subr_taskqueue.c index f17e59a441a6..b337aa83d69d 100644 --- a/sys/kern/subr_taskqueue.c +++ b/sys/kern/subr_taskqueue.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include @@ -78,6 +79,19 @@ struct taskqueue { void *tq_cb_contexts[TASKQUEUE_NUM_CALLBACKS]; }; +static SYSCTL_NODE(_kern, OID_AUTO, taskqueue, CTLFLAG_RW | CTLFLAG_MPSAFE, 0, + "taskqueue information"); + +/* + * Limit on the number of tasks that may be run in a single epoch section. + * It's profitable to batch tasks together, but there must be a bound in order + * to maintain system liveness. + */ +unsigned int net_epoch_task_limit = 8; +SYSCTL_UINT(_kern_taskqueue, OID_AUTO, net_epoch_task_limit, CTLFLAG_RWTUN, + &net_epoch_task_limit, 0, + "Maximum number of tasks to run in an epoch section"); + #define TQ_FLAGS_ACTIVE (1 << 0) #define TQ_FLAGS_BLOCKED (1 << 1) #define TQ_FLAGS_UNLOCKED_ENQUEUE (1 << 2) @@ -486,15 +500,15 @@ taskqueue_run_locked(struct taskqueue *queue) struct epoch_tracker et; struct taskqueue_busy tb; struct task *task; - bool in_net_epoch; + unsigned int epochtasks; int pending; KASSERT(queue != NULL, ("tq is NULL")); TQ_ASSERT_LOCKED(queue); tb.tb_running = NULL; LIST_INSERT_HEAD(&queue->tq_active, &tb, tb_link); - in_net_epoch = false; + epochtasks = 0; while ((task = STAILQ_FIRST(&queue->tq_queue)) != NULL) { STAILQ_REMOVE_HEAD(&queue->tq_queue, ta_link); if (queue->tq_hint == task) @@ -507,19 +521,23 @@ taskqueue_run_locked(struct taskqueue *queue) TQ_UNLOCK(queue); KASSERT(task->ta_func != NULL, ("task->ta_func is NULL")); - if (!in_net_epoch && TASK_IS_NET(task)) { - in_net_epoch = true; - NET_EPOCH_ENTER(et); - } else if (in_net_epoch && !TASK_IS_NET(task)) { + if (TASK_IS_NET(task)) { + if (epochtasks++ == 0) + NET_EPOCH_ENTER(et); + } else if (epochtasks > 0) { NET_EPOCH_EXIT(et); - in_net_epoch = false; + epochtasks = 0; } task->ta_func(task->ta_context, pending); + if (epochtasks > net_epoch_task_limit) { + NET_EPOCH_EXIT(et); + epochtasks = 0; + } TQ_LOCK(queue); wakeup(task); } - if (in_net_epoch) + if (epochtasks > 0) NET_EPOCH_EXIT(et); LIST_REMOVE(&tb, tb_link); } diff --git a/sys/sys/taskqueue.h b/sys/sys/taskqueue.h index 9da364a1a716..b9d4ebe98be4 100644 --- a/sys/sys/taskqueue.h +++ b/sys/sys/taskqueue.h @@ -221,4 +221,6 @@ struct taskqueue *taskqueue_create_fast(const char *name, int mflags, */ TASKQUEUE_DECLARE(bus); +extern unsigned int net_epoch_task_limit; + #endif /* !_SYS_TASKQUEUE_H_ */