From nobody Mon Jul 20 02:19:05 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h3PMz3mTXz6m2dD for ; Mon, 20 Jul 2026 02:19:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h3PMz33xHz3MN5 for ; Mon, 20 Jul 2026 02:19:11 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784513951; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=eACWbXqNrh2DnvmkkCYFNvBhAwqksdSln/bKZ21TkCs=; b=bUkTIJi8L7iFJBcFLKuo+el887TLi2bDSbGkB3kkOu4VEf2qYWwZK6302l2jtwL6SORXkg eGBdw6xA/SAJ9BDP6/hQ0PG70RNGZw+h+ZL5C/XP8uMp0HR58HuZtn8jRl/6Eln/OU3xWm W+fKVILLeTtIPAizwzDe+omsCoXT4m5RgYOhyNVnTBi6yys9nxWLOHz0k11/970QfHI5hL GWAIphaAWBKTt77LIiHF0XmmELJ73U5Wp3LmPuHLm6IklwFHB1rjey2FHiyOWj3218Szm+ GncKBlqjMJevtca1S+VCMLMZdEilB4Vz7vy3X1Sb5gHUXlKsz25ZfgYWbw32rg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1784513951; a=rsa-sha256; cv=none; b=JdqtbRP9ThPFGAzxGXddDaB7o4U935M8OI25G1eGF2d7yEX5A+jWZtHxhsMuUPGo9CBIgw zIPbUKj9d6U8mNVMzcKfkEFzwTHoFbm1dxrV5WyPOCnEBby1Dn7S2L5pnAOdnbHDn9a2vQ FGDqivKoODfvv1oeG62P2cD9Chd8+hWalIT4e0uwTYrwYo6LDonw07D0+tHb2MNWMem7fd 5gtpmED6xbA5KZSKG4SlyURLZHPvlGNYoD7YMpHdhV0LAgCsnJAMBOMESSlVSOl0DNLSOV bE/yP/XrjKrGFtlBU2BDzEppcWCBwaS6oPauMe6OBsQas625liy+/f47LBjs1w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784513951; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=eACWbXqNrh2DnvmkkCYFNvBhAwqksdSln/bKZ21TkCs=; b=e1/Esokkmf4efOQJ0WUjzuJ0oNatQzg1NwIda4rWFOMeGpk8hr6Sb5b60XiaayfI8GMnjg jhGEvIwsOOLhE8uNpibsHtysn5Fz+9zyxLcpI6xzQSF71kp6kG576LTJfAeEIkRY3xQ3cf LmFCCkDclbuRJ2SDMXjpP5p574tYxvSmUVb1OdsS6xZvubWH5Rm+qk/+ocbkPznUNddbnT o8dfOnx5lwUWCW0yGmf0AzHbtHjBjTo/GMVUYSDdLCx+6vRO7TdZ0tsLEqKBejtmgkHZx8 qW9W34GxSPlbjAt52gb5h1AO8ffeh6rXgtfEcof+BKQDj/CcYYAq3tBiFN0NmA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h3PMz1k0tzj4k for ; Mon, 20 Jul 2026 02:19:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 446bd by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 20 Jul 2026 02:19:05 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Olivier Certner Subject: git: a6587ae0f46d - main - sched_ule: Fix selecting lowest priority thread early in corner case List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: olce X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a6587ae0f46dba4e47fa92c831f55288ec181734 Auto-Submitted: auto-generated Date: Mon, 20 Jul 2026 02:19:05 +0000 Message-Id: <6a5d8599.446bd.7312ac01@gitrepo.freebsd.org> The branch main has been updated by olce: URL: https://cgit.FreeBSD.org/src/commit/?id=a6587ae0f46dba4e47fa92c831f55288ec181734 commit a6587ae0f46dba4e47fa92c831f55288ec181734 Author: Olivier Certner AuthorDate: 2026-06-16 21:06:36 +0000 Commit: Olivier Certner CommitDate: 2026-07-20 02:18:45 +0000 sched_ule: Fix selecting lowest priority thread early in corner case When transferring a thread with near 100% CPU statistics (but not 100%; up to 57.5/59≈97.46%) to a CPU where the enqueue offset is ahead of at least 2 from the dequeue one, which requires peculiar conditions to happen (transfer triggered by a bind request or cpuset change, or during balancing if a thread or more existed from a brief amount of time on the origin CPU), the transferred thread can get placed after the dequeue offset, effectively making it appear as a high priority one unduly, causing latency increase for other threads. The change here was missed when changing the enqueue and dequeue offsets update mechanism to recover pre-256-queue-runqueue ULE anti-starvation and fairness behavior. That change opened up the possibility that these two offsets are apart by more than one. Reviewed by: markj Discussed with: Minsoo Choo Fixes: 6792f3411f6d ("sched_ule: Recover previous nice and anti-starvation behaviors") MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D57829 --- sys/kern/sched_ule.c | 45 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 36 insertions(+), 9 deletions(-) diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c index cf0baabfda86..a9fe49b11ff7 100644 --- a/sys/kern/sched_ule.c +++ b/sys/kern/sched_ule.c @@ -484,6 +484,17 @@ sched_shouldpreempt(int pri, int cpri, int remote) return (0); } +static inline int +normalize_ts_off(int offset) +{ + /* + * Adding RQ_TS_POL_MODULO before taking the modulo is to ensure the + * dividend is positive (we want a positive result). + */ + MPASS(offset >= -RQ_TS_POL_MODULO); + return ((offset + RQ_TS_POL_MODULO) % RQ_TS_POL_MODULO); +} + /* * Add a thread to the actual run-queue. Keeps transferable counts up to * date with what is actually on the run-queue. Selects the correct @@ -517,18 +528,34 @@ tdq_runq_add(struct tdq *tdq, struct thread *td, int flags) /* Current queue from which processes are being run. */ idx = tdq->tdq_ts_deq_off; else { - idx = (RQ_PRI_TO_QUEUE_IDX(pri) - RQ_TS_POL_MIN + - tdq->tdq_ts_off) % RQ_TS_POL_MODULO; + idx = normalize_ts_off( + /* Offset corresponding to priority. */ + RQ_PRI_TO_QUEUE_IDX(pri) - RQ_TS_POL_MIN + + /* Insertion offset. */ + tdq->tdq_ts_off); /* - * We avoid enqueuing low priority threads in the queue - * that we are still draining, effectively shortening - * the runqueue by one queue. + * We avoid enqueuing low priority threads in the queues + * we still have to drain. This effectively shortens + * the runqueue by a few queues (see update of + * 'tdq_ts_deq_off' in sched_clock()). + * + * The expressions that include differences below are + * measuring the "distance" from the dequeue offset to + * either 'idx' or the insertion offset modulo + * RQ_TS_POL_MODULO. Thanks to the arithmetic operators + * always performing the usual arithmetic conversions, + * all operands are promoted to integers, which is + * necessary to accomodate corner cases (else + * we would have to be conditional on whether the first + * term is greater or lower than the second, in the + * second case correcting the result with UCHAR_MAX % + * RQ_TS_POL_MODULO). */ if (tdq->tdq_ts_deq_off != tdq->tdq_ts_off && - idx == tdq->tdq_ts_deq_off) - /* Ensure the dividend is positive. */ - idx = (idx - 1 + RQ_TS_POL_MODULO) % - RQ_TS_POL_MODULO; + normalize_ts_off(idx - tdq->tdq_ts_deq_off) < + normalize_ts_off(tdq->tdq_ts_off - + tdq->tdq_ts_deq_off)) + idx = normalize_ts_off(tdq->tdq_ts_deq_off - 1); } /* Absolute queue index. */ idx += RQ_TS_POL_MIN;