Date: Fri, 19 Jan 2024 16:35:24 GMT From: Alexander Motin <mav@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Subject: git: c849f305614c - stable/13 - Schedule fast taskqueue callouts on right CPU. Message-ID: <202401191635.40JGZORS026311@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=c849f305614c51b9c71196012596266fb661c54c commit c849f305614c51b9c71196012596266fb661c54c Author: Alexander Motin <mav@FreeBSD.org> AuthorDate: 2023-12-27 03:30:56 +0000 Commit: Alexander Motin <mav@FreeBSD.org> CommitDate: 2024-01-19 16:25:17 +0000 Schedule fast taskqueue callouts on right CPU. With fast taskqueues using direct callouts we can reduce number of CPU wakeups by scheduling callout on current CPU if taskqueue calls taskqueue_enqueue_timeout() on itself. The trick won't work for regular taskqueues, since the callout thread will occupy the CPU. It also may not work in case of multiple threads since we do not know which thread will pick the task, and we do not want excessive callout migrations. So we optimize only the other cases we can. In practice this allows iichid(4) taskqueue to stay on CPU where underlying ig4(4) interrupts are routed and to not kick CPU 0 with timer interrupts on each sampling period (every 2nd/3rd sleep). MFC after: 1 month (cherry picked from commit 7bbac6419d174c98cc6ea969b68fcfe0f9a9bab8) --- sys/kern/subr_taskqueue.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/sys/kern/subr_taskqueue.c b/sys/kern/subr_taskqueue.c index 07323763204d..832f120e5d82 100644 --- a/sys/kern/subr_taskqueue.c +++ b/sys/kern/subr_taskqueue.c @@ -364,8 +364,14 @@ taskqueue_enqueue_timeout_sbt(struct taskqueue *queue, if (sbt > 0) { if (queue->tq_spin) flags |= C_DIRECT_EXEC; - callout_reset_sbt(&timeout_task->c, sbt, pr, - taskqueue_timeout_func, timeout_task, flags); + if (queue->tq_spin && queue->tq_tcount == 1 && + queue->tq_threads[0] == curthread) { + callout_reset_sbt_curcpu(&timeout_task->c, sbt, pr, + taskqueue_timeout_func, timeout_task, flags); + } else { + callout_reset_sbt(&timeout_task->c, sbt, pr, + taskqueue_timeout_func, timeout_task, flags); + } } TQ_UNLOCK(queue); }
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202401191635.40JGZORS026311>