From nobody Mon Jul 27 22:32:42 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h8Cyy6bqzz6ly10 for ; Mon, 27 Jul 2026 22:32:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h8Cyy4p8Gz3CSB for ; Mon, 27 Jul 2026 22:32:42 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785191562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VHEm+3SWY0aMIaRAZ1m9QetuTPZHR/4W8HxecOboSSM=; b=enIIDeAvCzBZPaHL80Vw3lscSPXR0Fx1PNL+FfEO/X7T/VpV4juvN4hWp+ywACNs+qVYC0 4np09eZi9KgOOl8OunJDJwQMIMHFI5lKBZo1PrWTuc2wJb1Rabuq6VEhk0Ycru0A1m2UE5 mfXgxUqz+USpQXGJ8GH8KxeIuZlbPzULCDP0zaO4eII/log3x3Pe0WbX6cyLT95dZkntWb nMrdXYHXUXqWPwiUN+csvO+P1tU6IKg399jbtU411SxNPpSYVwBHmTUt6cQ4BKNMPGwnzY UyjX0uiAMpL2GFoH9967JQ3HMcx55KMaCHgmE1SCwZF98Dv78+KGBDPAYrO1Ww== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1785191562; a=rsa-sha256; cv=none; b=MSI7SjBLF1/hdDA7N2cYQ/rWoNhZDbXfXH0Nkj/fYrM/Dh4SSIiQ8j0akVrKxDyNadRP+Y ONzVuU5ZKWceejxlGidyY/pkWGiVMNr8ZwrLg/Z4E+3+32l9cattXJpKoDyhS7MTHsUEFn fkpB5lj/nciKrfCtmjE7AtL8mjerl4+kqSmgyYN0lhIboV64nPvB2R/2BMA3eCj29j5hzw vlwtIt5r8bxZX0Gz9oiVPCeMo3TTCF6OK5d3fCGoTU3k8QdX7+l7iMgLQ78QGR2v8064PA i7Ffhsl9w3eanwE24JuHbY74jKrzRa+e5c5I3aljk5HaOYRMkso9w3rD2JdB3g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785191562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VHEm+3SWY0aMIaRAZ1m9QetuTPZHR/4W8HxecOboSSM=; b=B5Briyx1eYRD024bcpHT3yR/nKkeux4jeSMYZO5TvOJPc/JEVCnHOAXAoJRt8suxaYJh7y vXPjtQYLehvEUf4uQ7VELAghvVUrafmpk8L1Z+cIKiaeHIemLxSP/1leb9HHISmZBuSJPt jDCSUbL1Px1cS1zQ/FRL43RzplbiG4GCG1iVfMwnD1gTmvM6yUsnM/6NxWbqDshACCTAe7 BDv/4swoCM0AtrKwfekCMHFSoh141jpmDFVJWXfD4P98m2QgG19v6DJnZJcNycq6Dkp5rP AdxpTEMQrcMdf0ZMEVilv9SXrVsDGDDvpxEdJhb8V0C6hDDiZGACEyAZB6yp8Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h8Cyy3dnwz1RKv for ; Mon, 27 Jul 2026 22:32:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 25ef7 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 27 Jul 2026 22:32:42 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 376b875cf590 - stable/14 - taskqueue: Avoid unbounded epoch read sections List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 376b875cf590bcbc0ed34b2d1b5ba9623557db39 Auto-Submitted: auto-generated Date: Mon, 27 Jul 2026 22:32:42 +0000 Message-Id: <6a67dc8a.25ef7.40ddc68b@gitrepo.freebsd.org> The branch stable/14 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=376b875cf590bcbc0ed34b2d1b5ba9623557db39 commit 376b875cf590bcbc0ed34b2d1b5ba9623557db39 Author: Mark Johnston AuthorDate: 2026-07-08 17:12:54 +0000 Commit: Mark Johnston CommitDate: 2026-07-27 19:35:19 +0000 taskqueue: Avoid unbounded epoch read sections The taskqueue thread loop tries to avoid entering and exiting net epoch read sections for every task. This reduces the overhead of net epoch integration, but the implementation wasn't bounding the length of the read section, so a busy taskqueue thread could hold an epoch open for an unbounded period. This is easy to achieve with the epair task, for instance. Bound the number of tasks that we'll execute without observing the global epoch, and provide a sysctl to control it. Let the default bound be eight. Reviewed by: glebius MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D58031 (cherry picked from commit a58590631ccc0fa5bdbbdf88021c6878d644d128) --- sys/kern/subr_gtaskqueue.c | 20 ++++++++++++-------- sys/kern/subr_taskqueue.c | 34 ++++++++++++++++++++++++++-------- sys/sys/taskqueue.h | 2 ++ 3 files changed, 40 insertions(+), 16 deletions(-) diff --git a/sys/kern/subr_gtaskqueue.c b/sys/kern/subr_gtaskqueue.c index 590dd53d0ea3..78772e000ae4 100644 --- a/sys/kern/subr_gtaskqueue.c +++ b/sys/kern/subr_gtaskqueue.c @@ -343,14 +343,14 @@ gtaskqueue_run_locked(struct gtaskqueue *queue) struct epoch_tracker et; struct gtaskqueue_busy tb; struct gtask *gtask; - bool in_net_epoch; + unsigned int epochtasks; KASSERT(queue != NULL, ("tq is NULL")); TQ_ASSERT_LOCKED(queue); tb.tb_running = NULL; LIST_INSERT_HEAD(&queue->tq_active, &tb, tb_link); - in_net_epoch = false; + epochtasks = 0; while ((gtask = STAILQ_FIRST(&queue->tq_queue)) != NULL) { STAILQ_REMOVE_HEAD(&queue->tq_queue, ta_link); gtask->ta_flags &= ~TASK_ENQUEUED; @@ -359,19 +359,23 @@ gtaskqueue_run_locked(struct gtaskqueue *queue) TQ_UNLOCK(queue); KASSERT(gtask->ta_func != NULL, ("task->ta_func is NULL")); - if (!in_net_epoch && TASK_IS_NET(gtask)) { - in_net_epoch = true; - NET_EPOCH_ENTER(et); - } else if (in_net_epoch && !TASK_IS_NET(gtask)) { + if (TASK_IS_NET(gtask)) { + if (epochtasks++ == 0) + NET_EPOCH_ENTER(et); + } else if (epochtasks > 0) { NET_EPOCH_EXIT(et); - in_net_epoch = false; + epochtasks = 0; } gtask->ta_func(gtask->ta_context); + if (epochtasks > net_epoch_task_limit) { + NET_EPOCH_EXIT(et); + epochtasks = 0; + } TQ_LOCK(queue); wakeup(gtask); } - if (in_net_epoch) + if (epochtasks > 0) NET_EPOCH_EXIT(et); LIST_REMOVE(&tb, tb_link); } diff --git a/sys/kern/subr_taskqueue.c b/sys/kern/subr_taskqueue.c index 832f120e5d82..6ee5a3f71a83 100644 --- a/sys/kern/subr_taskqueue.c +++ b/sys/kern/subr_taskqueue.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -79,6 +80,19 @@ struct taskqueue { void *tq_cb_contexts[TASKQUEUE_NUM_CALLBACKS]; }; +static SYSCTL_NODE(_kern, OID_AUTO, taskqueue, CTLFLAG_RW | CTLFLAG_MPSAFE, 0, + "taskqueue information"); + +/* + * Limit on the number of tasks that may be run in a single epoch section. + * It's profitable to batch tasks together, but there must be a bound in order + * to maintain system liveness. + */ +unsigned int net_epoch_task_limit = 8; +SYSCTL_UINT(_kern_taskqueue, OID_AUTO, net_epoch_task_limit, CTLFLAG_RWTUN, + &net_epoch_task_limit, 0, + "Maximum number of tasks to run in an epoch section"); + #define TQ_FLAGS_ACTIVE (1 << 0) #define TQ_FLAGS_BLOCKED (1 << 1) #define TQ_FLAGS_UNLOCKED_ENQUEUE (1 << 2) @@ -487,15 +501,15 @@ taskqueue_run_locked(struct taskqueue *queue) struct epoch_tracker et; struct taskqueue_busy tb; struct task *task; - bool in_net_epoch; + unsigned int epochtasks; int pending; KASSERT(queue != NULL, ("tq is NULL")); TQ_ASSERT_LOCKED(queue); tb.tb_running = NULL; LIST_INSERT_HEAD(&queue->tq_active, &tb, tb_link); - in_net_epoch = false; + epochtasks = 0; while ((task = STAILQ_FIRST(&queue->tq_queue)) != NULL) { STAILQ_REMOVE_HEAD(&queue->tq_queue, ta_link); if (queue->tq_hint == task) @@ -508,19 +522,23 @@ taskqueue_run_locked(struct taskqueue *queue) TQ_UNLOCK(queue); KASSERT(task->ta_func != NULL, ("task->ta_func is NULL")); - if (!in_net_epoch && TASK_IS_NET(task)) { - in_net_epoch = true; - NET_EPOCH_ENTER(et); - } else if (in_net_epoch && !TASK_IS_NET(task)) { + if (TASK_IS_NET(task)) { + if (epochtasks++ == 0) + NET_EPOCH_ENTER(et); + } else if (epochtasks > 0) { NET_EPOCH_EXIT(et); - in_net_epoch = false; + epochtasks = 0; } task->ta_func(task->ta_context, pending); + if (epochtasks > net_epoch_task_limit) { + NET_EPOCH_EXIT(et); + epochtasks = 0; + } TQ_LOCK(queue); wakeup(task); } - if (in_net_epoch) + if (epochtasks > 0) NET_EPOCH_EXIT(et); LIST_REMOVE(&tb, tb_link); } diff --git a/sys/sys/taskqueue.h b/sys/sys/taskqueue.h index 9da364a1a716..b9d4ebe98be4 100644 --- a/sys/sys/taskqueue.h +++ b/sys/sys/taskqueue.h @@ -221,4 +221,6 @@ struct taskqueue *taskqueue_create_fast(const char *name, int mflags, */ TASKQUEUE_DECLARE(bus); +extern unsigned int net_epoch_task_limit; + #endif /* !_SYS_TASKQUEUE_H_ */