From nobody Wed Jun 18 02:13:14 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bMS2M1srbz5q6Sv; Wed, 18 Jun 2025 02:13:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bMS2L506Mz41BQ; Wed, 18 Jun 2025 02:13:14 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1750212794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FVNL6wV9md1ZUgnPxbIGaCJwXcVVS5IPqt3ZyGdyUX4=; b=Yk+frghHJHqdMRXpsKKUsvyl4nVxyoJ+jthEz55C/JVy0YkUJmfzKg8Y8dg7U0RqT8ZjAT 8CU4EDCkZxcBI413U8OarKTsRXt8qlwxSy2Kz7goICZL0ffC/qsJzthO4vvzWxLvWYHOzi y9VvolU0YtL+ZHNFTtb6AVC5ozSn/1kzYqSBu1EaPpCALhi/9DSHy4UA/DQFY1mjml66vD nk3Qym6iZKUFsr6v/l56Yfe1N28Qyzyyrhe3JZkS4yOT+QJhN3gXy1PIhcfIXL0TRiVAVF d+bsSWJwkxwSqQf7UijlkG8T3L0eiY+YHCXnk36VCW6RMNHxM2NFnG9KR5dS4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1750212794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FVNL6wV9md1ZUgnPxbIGaCJwXcVVS5IPqt3ZyGdyUX4=; b=HAkfA+GHgYpIFjPdOiNTgxwIJiBuvoFEAkn5uaMR8H4685n9McoissvvYqtJJk+S914lva XakmTbceYG3zveaP4HWauKDgXvhjbyRuJZuR/mrGmGElvzoyvFPJxKlQleAlbfxZq1sf62 JKHqyJIex2fOMTTZ30yMWBfPAxMISrTvgBq46EoFhhvt6ExlxnCmlBRFkdc40asc+85rf2 q14tIaZ9uARmVIYIVde9seG6Bkldk+JgKg6bNf4W/yqzf4wzRr9D93454qewYNEU8aEhHL ssq2A2udcAC0u8TXPZmMAeGLtnG8zlx3Ql0woTChVjXoOTf63S1h0rHcEP3xZQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1750212794; a=rsa-sha256; cv=none; b=nexFvCX05bG4b4RxiDCU3218jp7AVDtMgf10o+TKqg+FmlSB41BpccOtqR/M76zp9Ubo/G Pt4d2Y7fve0gqvP266kU+YvdH+1LYwrz/H6SKbWV8bTZWBc6C+LRoWCj1xPDaUAVsUW5Wc P5A/bAjBGyO3/QjkOKYjXtBQ89AH3QVUQsSwJHzclqcXLHH4KTjk4tSgozKXMWDEP+Oljg nh1sLBX/6jnNgJCOSjobEc7KHZ/52bOLok2FodnTJbd0Iyb9xSdik0lJBPTH2upSx/PLCx P/juumOfGlTDBhQ7dwMK2TuwJhPChNuD/fFrzBcbp5j5Pi3qMgb22UfaLW90JQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4bMS2L4QfDz18fV; Wed, 18 Jun 2025 02:13:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 55I2DEWr024137; Wed, 18 Jun 2025 02:13:14 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 55I2DETm024134; Wed, 18 Jun 2025 02:13:14 GMT (envelope-from git) Date: Wed, 18 Jun 2025 02:13:14 GMT Message-Id: <202506180213.55I2DETm024134@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Olivier Certner Subject: git: 200fc93dace7 - main - runq: Re-order functions more logically List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: olce X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 200fc93dace76cf251460adc58ca809c36031f3e Auto-Submitted: auto-generated The branch main has been updated by olce: URL: https://cgit.FreeBSD.org/src/commit/?id=200fc93dace76cf251460adc58ca809c36031f3e commit 200fc93dace76cf251460adc58ca809c36031f3e Author: Olivier Certner AuthorDate: 2024-03-01 13:26:39 +0000 Commit: Olivier Certner CommitDate: 2025-06-18 02:07:59 +0000 runq: Re-order functions more logically No code change in moved functions. Reviewed by: kib MFC after: 1 month Event: Kitchener-Waterloo Hackathon 202506 Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D45387 --- sys/kern/kern_switch.c | 201 +++++++++++++++++++++++++------------------------ sys/sys/runq.h | 7 +- 2 files changed, 105 insertions(+), 103 deletions(-) diff --git a/sys/kern/kern_switch.c b/sys/kern/kern_switch.c index 9e63709d0acc..69e6f4818e40 100644 --- a/sys/kern/kern_switch.c +++ b/sys/kern/kern_switch.c @@ -274,6 +274,24 @@ runq_init(struct runq *rq) TAILQ_INIT(&rq->rq_queues[i]); } +/* + * Set the status bit of the queue at index 'idx', indicating that it is + * non-empty. + */ +static __inline void +runq_setbit(struct runq *rq, int idx) +{ + struct rq_status *rqs; + + CHECK_IDX(idx); + rqs = &rq->rq_status; + CTR4(KTR_RUNQ, "runq_setbit: bits=%#x %#x bit=%#x word=%d", + rqs->rq_sw[RQSW_IDX(idx)], + rqs->rq_sw[RQSW_IDX(idx)] | RQSW_BIT(idx), + RQSW_BIT(idx), RQSW_IDX(idx)); + rqs->rq_sw[RQSW_IDX(idx)] |= RQSW_BIT(idx); +} + /* * Clear the status bit of the queue at index 'idx', indicating that it is * empty. @@ -292,6 +310,65 @@ runq_clrbit(struct runq *rq, int idx) rqs->rq_sw[RQSW_IDX(idx)] &= ~RQSW_BIT(idx); } +/* + * Add the thread to the queue specified by its priority, and set the + * corresponding status bit. + */ +void +runq_add(struct runq *rq, struct thread *td, int flags) +{ + + runq_add_idx(rq, td, RQ_PRI_TO_QUEUE_IDX(td->td_priority), flags); +} + +void +runq_add_idx(struct runq *rq, struct thread *td, int idx, int flags) +{ + struct rq_queue *rqq; + + /* + * runq_setbit() asserts 'idx' is non-negative and below 'RQ_NQS', and + * a static assert earlier in this file ensures that 'RQ_NQS' is no more + * than 256. + */ + td->td_rqindex = idx; + runq_setbit(rq, idx); + rqq = &rq->rq_queues[idx]; + CTR4(KTR_RUNQ, "runq_add_idx: td=%p pri=%d idx=%d rqq=%p", + td, td->td_priority, idx, rqq); + if (flags & SRQ_PREEMPTED) + TAILQ_INSERT_HEAD(rqq, td, td_runq); + else + TAILQ_INSERT_TAIL(rqq, td, td_runq); +} + +/* + * Remove the thread from the queue specified by its priority, and clear the + * corresponding status bit if the queue becomes empty. + * + * Returns whether the corresponding queue is empty after removal. + */ +bool +runq_remove(struct runq *rq, struct thread *td) +{ + struct rq_queue *rqq; + int idx; + + KASSERT(td->td_flags & TDF_INMEM, ("runq_remove: Thread swapped out")); + idx = td->td_rqindex; + CHECK_IDX(idx); + rqq = &rq->rq_queues[idx]; + CTR4(KTR_RUNQ, "runq_remove: td=%p pri=%d idx=%d rqq=%p", + td, td->td_priority, idx, rqq); + TAILQ_REMOVE(rqq, td, td_runq); + if (TAILQ_EMPTY(rqq)) { + runq_clrbit(rq, idx); + CTR1(KTR_RUNQ, "runq_remove: queue at idx=%d now empty", idx); + return (true); + } + return (false); +} + /* * Find the index of the first non-empty run queue. This is done by * scanning the status bits, a set bit indicating a non-empty queue. @@ -346,56 +423,6 @@ again: goto again; } -/* - * Set the status bit of the queue at index 'idx', indicating that it is - * non-empty. - */ -static __inline void -runq_setbit(struct runq *rq, int idx) -{ - struct rq_status *rqs; - - CHECK_IDX(idx); - rqs = &rq->rq_status; - CTR4(KTR_RUNQ, "runq_setbit: bits=%#x %#x bit=%#x word=%d", - rqs->rq_sw[RQSW_IDX(idx)], - rqs->rq_sw[RQSW_IDX(idx)] | RQSW_BIT(idx), - RQSW_BIT(idx), RQSW_IDX(idx)); - rqs->rq_sw[RQSW_IDX(idx)] |= RQSW_BIT(idx); -} - -/* - * Add the thread to the queue specified by its priority, and set the - * corresponding status bit. - */ -void -runq_add(struct runq *rq, struct thread *td, int flags) -{ - - runq_add_idx(rq, td, RQ_PRI_TO_QUEUE_IDX(td->td_priority), flags); -} - -void -runq_add_idx(struct runq *rq, struct thread *td, int idx, int flags) -{ - struct rq_queue *rqq; - - /* - * runq_setbit() asserts 'idx' is non-negative and below 'RQ_NQS', and - * a static assert earlier in this file ensures that 'RQ_NQS' is no more - * than 256. - */ - td->td_rqindex = idx; - runq_setbit(rq, idx); - rqq = &rq->rq_queues[idx]; - CTR4(KTR_RUNQ, "runq_add_idx: td=%p pri=%d idx=%d rqq=%p", - td, td->td_priority, idx, rqq); - if (flags & SRQ_PREEMPTED) - TAILQ_INSERT_HEAD(rqq, td, td_runq); - else - TAILQ_INSERT_TAIL(rqq, td, td_runq); -} - /* * Return true if there are some processes of any priority on the run queue, * false otherwise. Has no side effects. @@ -417,6 +444,30 @@ runq_not_empty(struct runq *rq) return (false); } +/* + * Find the highest priority process on the run queue. + */ +struct thread * +runq_choose(struct runq *rq) +{ + struct rq_queue *rqq; + struct thread *td; + int idx; + + idx = runq_findbit(rq); + if (idx != -1) { + rqq = &rq->rq_queues[idx]; + td = TAILQ_FIRST(rqq); + KASSERT(td != NULL, ("runq_choose: no thread on busy queue")); + CTR3(KTR_RUNQ, + "runq_choose: idx=%d thread=%p rqq=%p", idx, td, rqq); + return (td); + } + CTR1(KTR_RUNQ, "runq_choose: idlethread idx=%d", idx); + + return (NULL); +} + /* * Find the highest priority process on the run queue. */ @@ -460,30 +511,6 @@ runq_choose_fuzz(struct runq *rq, int fuzz) return (NULL); } -/* - * Find the highest priority process on the run queue. - */ -struct thread * -runq_choose(struct runq *rq) -{ - struct rq_queue *rqq; - struct thread *td; - int idx; - - idx = runq_findbit(rq); - if (idx != -1) { - rqq = &rq->rq_queues[idx]; - td = TAILQ_FIRST(rqq); - KASSERT(td != NULL, ("runq_choose: no thread on busy queue")); - CTR3(KTR_RUNQ, - "runq_choose: idx=%d thread=%p rqq=%p", idx, td, rqq); - return (td); - } - CTR1(KTR_RUNQ, "runq_choose: idlethread idx=%d", idx); - - return (NULL); -} - struct thread * runq_choose_from(struct runq *rq, int from_idx) { @@ -504,29 +531,3 @@ runq_choose_from(struct runq *rq, int from_idx) return (NULL); } -/* - * Remove the thread from the queue specified by its priority, and clear the - * corresponding status bit if the queue becomes empty. - * - * Returns whether the corresponding queue is empty after removal. - */ -bool -runq_remove(struct runq *rq, struct thread *td) -{ - struct rq_queue *rqq; - int idx; - - KASSERT(td->td_flags & TDF_INMEM, ("runq_remove: Thread swapped out")); - idx = td->td_rqindex; - CHECK_IDX(idx); - rqq = &rq->rq_queues[idx]; - CTR4(KTR_RUNQ, "runq_remove: td=%p pri=%d idx=%d rqq=%p", - td, td->td_priority, idx, rqq); - TAILQ_REMOVE(rqq, td, td_runq); - if (TAILQ_EMPTY(rqq)) { - runq_clrbit(rq, idx); - CTR1(KTR_RUNQ, "runq_remove: queue at idx=%d now empty", idx); - return (true); - } - return (false); -} diff --git a/sys/sys/runq.h b/sys/sys/runq.h index 80a1fad0a089..c570dd25503b 100644 --- a/sys/sys/runq.h +++ b/sys/sys/runq.h @@ -97,14 +97,15 @@ struct runq { struct rq_queue rq_queues[RQ_NQS]; }; +void runq_init(struct runq *); void runq_add(struct runq *, struct thread *, int _flags); void runq_add_idx(struct runq *, struct thread *, int _idx, int _flags); +bool runq_remove(struct runq *, struct thread *); + bool runq_not_empty(struct runq *); struct thread *runq_choose(struct runq *); -struct thread *runq_choose_from(struct runq *, int _idx); struct thread *runq_choose_fuzz(struct runq *, int _fuzz); -void runq_init(struct runq *); -bool runq_remove(struct runq *, struct thread *); +struct thread *runq_choose_from(struct runq *, int _idx); #endif /* _KERNEL */ #endif