From nobody Sat Nov 12 12:41:42 2022
X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N8ZvV73B9z4fff0;
Sat, 12 Nov 2022 12:41:42 +0000 (UTC)
(envelope-from git@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4N8ZvV6FlBz3sjh;
Sat, 12 Nov 2022 12:41:42 +0000 (UTC)
(envelope-from git@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
t=1668256902;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding;
bh=TOttvEgCERAGqobBG/cmQ/sow1rvh2D/MLtQsaXVhFE=;
b=EhaxhfgP7jSJfNHOHhmetcP7E31/E/FVzgA2AChxNS3OQmbBWVOB7C4APoQjRGada/udzQ
bnbDWx6X00cKGuovm9X+E3hpSFD/Dhhdd0PRXS/eVjixkefglHJtYTu8f68HWEAjh3Ct5j
ssqTbEfaz8h/nqO7jorAV+I6AsprngKgBSMRnRfwbNewplGLTNqb2+rGrijQNrFr7Xikz8
w7ACV130ysYSjmgJJ+gAfaxoh+lNJ+V0y4ziaxJqoGUM5gSoscmxVdsMffllqVXr2v14Ot
jDdzfs+Qkz3UeyjPtBj5i6R7lkjQ/c2YNBBpdF5DdaSCNNEQlzAOWG/2sNuLeg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
s=dkim; t=1668256902;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding;
bh=TOttvEgCERAGqobBG/cmQ/sow1rvh2D/MLtQsaXVhFE=;
b=dA4MzkAyjpDczD5LvoIUauZ7ZX8DKL7PPbnja/gOQ6JxW4IPg9iIk4FdWj+vX+8y6wDm8z
n7NmPFmuWCSUBRKsmKyLGxR5U5c4mOVBs20g/y4SX2X6H7XWGNbAZ9apnfrIZPVv2EibWl
EPfpff5iKZd0Ea77VphXhxjrWsd5rxpcnwOXIaEKFuv8ZcVFTAviYZfnymr4teLb4t6HqX
hC3UB2b3fLdIrx1UwP4HSEqpZqQIip3PBY0rq6E4Y5cv0Xl8bofP9NmZ48erDygZx/+yvH
Trv97EowyVfsjuSSIBYEUn0qRE1ydNG7s8lkpVqc4KceRMHDtPoC0o5W7PUz/g==
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1668256902; a=rsa-sha256; cv=none;
b=EIJ1Te0H5y5ZznnSXgup3sRCQVdsAAj/12+KRPvLK0VjeDeuH9poq1phpOMWrPZhfGiDTv
tpuqWD4Ifz9e9FYSAx8N64eyOPOrF9W/tI88+iHiJzDDbAwcmNCytM21podm7PxnAnEwMK
hMLE3Nar6GE85E7H7zzujsqtImsQ39YpR+Kieo3LbB3nKBuv9yHyOApaEJHnhhE+km/rG8
OvPewTw9CtAnq7nk3uSYXeUvksnWqndMo4WJ4L8pEACBZKOcoVKD8rj8msYrU5gkKGoiBO
dT2UYUDRb2wws9kkwsCzjET6LAixyZhB7zOVJ2WXkFIANokrFTxe9ZHd02z6rA==
ARC-Authentication-Results: i=1;
mx1.freebsd.org;
none
Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4N8ZvV5JnnzP2B;
Sat, 12 Nov 2022 12:41:42 +0000 (UTC)
(envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org ([127.0.1.44])
by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2ACCfgI9054744;
Sat, 12 Nov 2022 12:41:42 GMT
(envelope-from git@gitrepo.freebsd.org)
Received: (from git@localhost)
by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2ACCfgnx054743;
Sat, 12 Nov 2022 12:41:42 GMT
(envelope-from git)
Date: Sat, 12 Nov 2022 12:41:42 GMT
Message-Id: <202211121241.2ACCfgnx054743@gitrepo.freebsd.org>
To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org,
dev-commits-src-branches@FreeBSD.org
From: Hans Petter Selasky <hselasky@FreeBSD.org>
Subject: git: e21c77f80c3b - stable/13 - Improve timeout precision of pthread_cond_timedwait().
List-Id: Commit messages for all branches of the src repository <dev-commits-src-all.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all
List-Help: <mailto:dev-commits-src-all+help@freebsd.org>
List-Post: <mailto:dev-commits-src-all@freebsd.org>
List-Subscribe: <mailto:dev-commits-src-all+subscribe@freebsd.org>
List-Unsubscribe: <mailto:dev-commits-src-all+unsubscribe@freebsd.org>
Sender: owner-dev-commits-src-all@freebsd.org
X-BeenThere: dev-commits-src-all@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Git-Committer: hselasky
X-Git-Repository: src
X-Git-Refname: refs/heads/stable/13
X-Git-Reftype: branch
X-Git-Commit: e21c77f80c3b1cf2bd0a9f874e15e7a8e49f0dba
Auto-Submitted: auto-generated
X-ThisMailContainsUnwantedMimeParts: N
The branch stable/13 has been updated by hselasky:
URL: https://cgit.FreeBSD.org/src/commit/?id=e21c77f80c3b1cf2bd0a9f874e15e7a8e49f0dba
commit e21c77f80c3b1cf2bd0a9f874e15e7a8e49f0dba
Author: Alexander Motin <mav@FreeBSD.org>
AuthorDate: 2022-03-04 03:03:09 +0000
Commit: Hans Petter Selasky <hselasky@FreeBSD.org>
CommitDate: 2022-11-12 11:59:18 +0000
Improve timeout precision of pthread_cond_timedwait().
This code was not touched when all other user-space sleep functions were
switched to sbintime_t and decoupled from hardclock. When it is possible,
convert supplied times into sbinuptime to supply directly to msleep_sbt()
with C_ABSOLUTE. This provides the timeout resolution of few microseconds
instead of 2 milliseconds, plus avoids few clock reads and conversions.
Reviewed by: vangyzen
Differential Revision: https://reviews.freebsd.org/D34163
(cherry picked from commit 56070dd2e4df70e4de2a10c7ace684ceb8b0f0bd)
---
sys/kern/kern_umtx.c | 130 ++++++++++++++++++++++++++++++++-------------------
1 file changed, 82 insertions(+), 48 deletions(-)
diff --git a/sys/kern/kern_umtx.c b/sys/kern/kern_umtx.c
index 6aac9f3311d6..398ee23c5148 100644
--- a/sys/kern/kern_umtx.c
+++ b/sys/kern/kern_umtx.c
@@ -174,8 +174,6 @@ static SYSCTL_NODE(_debug_umtx, OID_AUTO, chains, CTLFLAG_RD | CTLFLAG_MPSAFE, 0
static inline void umtx_abs_timeout_init2(struct umtx_abs_timeout *timo,
const struct _umtx_time *umtxtime);
-static int umtx_abs_timeout_gethz(struct umtx_abs_timeout *timo);
-static inline void umtx_abs_timeout_update(struct umtx_abs_timeout *timo);
static void umtx_shm_init(void);
static void umtxq_sysinit(void *);
@@ -683,20 +681,14 @@ umtx_abs_timeout_init(struct umtx_abs_timeout *timo, int clockid,
timo->clockid = clockid;
if (!absolute) {
timo->is_abs_real = false;
- umtx_abs_timeout_update(timo);
+ kern_clock_gettime(curthread, timo->clockid, &timo->cur);
timespecadd(&timo->cur, timeout, &timo->end);
} else {
timo->end = *timeout;
timo->is_abs_real = clockid == CLOCK_REALTIME ||
clockid == CLOCK_REALTIME_FAST ||
- clockid == CLOCK_REALTIME_PRECISE;
- /*
- * If is_abs_real, umtxq_sleep will read the clock
- * after setting td_rtcgen; otherwise, read it here.
- */
- if (!timo->is_abs_real) {
- umtx_abs_timeout_update(timo);
- }
+ clockid == CLOCK_REALTIME_PRECISE ||
+ clockid == CLOCK_SECOND;
}
}
@@ -709,22 +701,71 @@ umtx_abs_timeout_init2(struct umtx_abs_timeout *timo,
(umtxtime->_flags & UMTX_ABSTIME) != 0, &umtxtime->_timeout);
}
-static void
-umtx_abs_timeout_update(struct umtx_abs_timeout *timo)
-{
-
- kern_clock_gettime(curthread, timo->clockid, &timo->cur);
-}
-
static int
-umtx_abs_timeout_gethz(struct umtx_abs_timeout *timo)
+umtx_abs_timeout_getsbt(struct umtx_abs_timeout *timo, sbintime_t *sbt,
+ int *flags)
{
+ struct bintime bt, bbt;
struct timespec tts;
- if (timespeccmp(&timo->end, &timo->cur, <=))
- return (-1);
- timespecsub(&timo->end, &timo->cur, &tts);
- return (tstohz(&tts));
+ switch (timo->clockid) {
+
+ /* Clocks that can be converted into absolute time. */
+ case CLOCK_REALTIME:
+ case CLOCK_REALTIME_PRECISE:
+ case CLOCK_REALTIME_FAST:
+ case CLOCK_MONOTONIC:
+ case CLOCK_MONOTONIC_PRECISE:
+ case CLOCK_MONOTONIC_FAST:
+ case CLOCK_UPTIME:
+ case CLOCK_UPTIME_PRECISE:
+ case CLOCK_UPTIME_FAST:
+ case CLOCK_SECOND:
+ timespec2bintime(&timo->end, &bt);
+ switch (timo->clockid) {
+ case CLOCK_REALTIME:
+ case CLOCK_REALTIME_PRECISE:
+ case CLOCK_REALTIME_FAST:
+ case CLOCK_SECOND:
+ getboottimebin(&bbt);
+ bintime_sub(&bt, &bbt);
+ break;
+ }
+ if (bt.sec < 0)
+ return (ETIMEDOUT);
+ if (bt.sec >= (SBT_MAX >> 32)) {
+ *sbt = 0;
+ *flags = 0;
+ return (0);
+ }
+ *sbt = bttosbt(bt);
+ switch (timo->clockid) {
+ case CLOCK_REALTIME_FAST:
+ case CLOCK_MONOTONIC_FAST:
+ case CLOCK_UPTIME_FAST:
+ *sbt += tc_tick_sbt;
+ break;
+ case CLOCK_SECOND:
+ *sbt += SBT_1S;
+ break;
+ }
+ *flags = C_ABSOLUTE;
+ return (0);
+
+ /* Clocks that has to be periodically polled. */
+ case CLOCK_VIRTUAL:
+ case CLOCK_PROF:
+ case CLOCK_THREAD_CPUTIME_ID:
+ case CLOCK_PROCESS_CPUTIME_ID:
+ default:
+ kern_clock_gettime(curthread, timo->clockid, &timo->cur);
+ if (timespeccmp(&timo->end, &timo->cur, <=))
+ return (ETIMEDOUT);
+ timespecsub(&timo->end, &timo->cur, &tts);
+ *sbt = tick_sbt * tstohz(&tts);
+ *flags = C_HARDCLOCK;
+ return (0);
+ }
}
static uint32_t
@@ -746,15 +787,11 @@ umtx_unlock_val(uint32_t flags, bool rb)
*/
int
umtxq_sleep(struct umtx_q *uq, const char *wmesg,
- struct umtx_abs_timeout *abstime)
+ struct umtx_abs_timeout *timo)
{
struct umtxq_chain *uc;
- int error, timo;
-
- if (abstime != NULL && abstime->is_abs_real) {
- curthread->td_rtcgen = atomic_load_acq_int(&rtc_generation);
- umtx_abs_timeout_update(abstime);
- }
+ sbintime_t sbt = 0;
+ int error, flags = 0;
uc = umtxq_getchain(&uq->uq_key);
UMTXQ_LOCKED_ASSERT(uc);
@@ -763,26 +800,22 @@ umtxq_sleep(struct umtx_q *uq, const char *wmesg,
error = 0;
break;
}
- if (abstime != NULL) {
- timo = umtx_abs_timeout_gethz(abstime);
- if (timo < 0) {
- error = ETIMEDOUT;
- break;
- }
- } else
- timo = 0;
- error = msleep(uq, &uc->uc_lock, PCATCH | PDROP, wmesg, timo);
- if (error == EINTR || error == ERESTART) {
- umtxq_lock(&uq->uq_key);
- break;
- }
- if (abstime != NULL) {
- if (abstime->is_abs_real)
+ if (timo != NULL) {
+ if (timo->is_abs_real)
curthread->td_rtcgen =
atomic_load_acq_int(&rtc_generation);
- umtx_abs_timeout_update(abstime);
+ error = umtx_abs_timeout_getsbt(timo, &sbt, &flags);
+ if (error != 0)
+ break;
+ }
+ error = msleep_sbt(uq, &uc->uc_lock, PCATCH, wmesg,
+ sbt, 0, flags);
+ if (error == EINTR || error == ERESTART)
+ break;
+ if (error == EWOULDBLOCK && (flags & C_ABSOLUTE) != 0) {
+ error = ETIMEDOUT;
+ break;
}
- umtxq_lock(&uq->uq_key);
}
curthread->td_rtcgen = 0;
@@ -3667,7 +3700,8 @@ again:
if (error == ERESTART)
error = EINTR;
if (error == EINTR) {
- umtx_abs_timeout_update(&timo);
+ kern_clock_gettime(curthread, timo.clockid,
+ &timo.cur);
timespecsub(&timo.end, &timo.cur,
&timeout->_timeout);
}