From nobody Fri Feb 14 19:25:29 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Yvhpd2Mjfz5nXYM; Fri, 14 Feb 2025 19:25:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Yvhpd1Kxdz48km; Fri, 14 Feb 2025 19:25:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1739561129; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iZr7KgNWEq1utHBm7tJ6qb0SKy7FPM61xjvX8u65QNc=; b=yLoHCHOqGqnCtGztyckQjNjXh3MexqDPvUxEwEn7awRNEg3hhA+TYhRtm+XVX9hZlCBPpf 9wlwpv8OsZ6P6KUvnHFq5kCE680rNXR4EdcnqtA1g/kX920mTMPX4w33Htlm4os9I0bQ4p 1gbmdiKxesaxneVLsMlDzjKTrIfDHyQynS2y4/lFHGarAQRWnaeVUCF+wcL4di6fFCdBFm ijkKQJyhCJycMyfGd/6xKqPNqeRBr7RyuU9dX5d+D74k/M8JqdQMv0M5ilbi6VXELUu2mV LvwP9uvde543M1B4q4yjGch73Plum2/XMFLK9v2Vw3W/EO3mRdWbWgfoQEpmYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1739561129; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iZr7KgNWEq1utHBm7tJ6qb0SKy7FPM61xjvX8u65QNc=; b=bPtMhdIJbD3zJ46L2DaRm2gX+c6EacHB3PwCkXIAqK9QGq1W+BELdGb15U7uvZ9E5ejB3j /wccmB9omNZPjNj9RPwFaNHy52U9kchW82xQwwV6AyuD7HSrcm5otYaHPHoL78gWE39VDa tm9totSPqZujSmjRtHrpUwpjCo2e6HuP6xyitCgUB+q3jpx5hBrhKrhejl2ZrfQPQ2MdGJ x1kA8fMrVyBezj4UMZWl3Z6UFFZd8yhbw9VhZ4Aj/R0nWF+qcBFGcPUq6lUsavtBk4zR8d ozvav5uTRpj1+4FJU7HF0VaytNmRX5FesIy/lmcOZm+lby/zSSOuXG8mJC2W5g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1739561129; a=rsa-sha256; cv=none; b=wOXBkIGi7Ow7ekWzefRcP69WPOoremoR91xGsqmE4hbdrn+04OhFiJKoZJucUljHMjFAEP gnIvoy4tK94u/KQN2iUGExmFX8SZx/vplDs2wfhkZS5TOy6vjr3LlEaM2XQAOlj14HFAqw uW4lFCLRnHQpLxqzWxLL3KtZ7slOoeXoejx7TSaDjq8jVWhG5NVZv4oiicNPDUUVJd4iFn mw0HDzMBuHMCTK7mUoyJ95uf4m9k6WQFyiGmmwI4tYVN8gSD2zRSZJpq3/Casr1B1KfBHZ wqTRMeVhq8M0R1n4TzbN6f2WodkipspSKwLB+5KoLW+3pC/2dKGVv9Y3crSB0g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Yvhpd0W3yzkks; Fri, 14 Feb 2025 19:25:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 51EJPTra038819; Fri, 14 Feb 2025 19:25:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 51EJPTsv038816; Fri, 14 Feb 2025 19:25:29 GMT (envelope-from git) Date: Fri, 14 Feb 2025 19:25:29 GMT Message-Id: <202502141925.51EJPTsv038816@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 467fa302c3ae - stable/14 - clock: Add a long ticks variable, ticksl List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 467fa302c3aeefd7ccd9fa6bd0ad0381d86cd75b Auto-Submitted: auto-generated The branch stable/14 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=467fa302c3aeefd7ccd9fa6bd0ad0381d86cd75b commit 467fa302c3aeefd7ccd9fa6bd0ad0381d86cd75b Author: Mark Johnston AuthorDate: 2025-01-10 15:37:07 +0000 Commit: Mark Johnston CommitDate: 2025-02-14 19:25:18 +0000 clock: Add a long ticks variable, ticksl For compatibility with Linux, it's useful to have a tick counter of width sizeof(long), but our tick counter is an int. Currently the linuxkpi tries paper over this difference, but this cannot really be done reliably, so it's desirable to have a wider tick counter. This change introduces ticksl, keeping the existing ticks variable. Follow a suggestion from kib to avoid having to maintain two separate counters and to avoid converting existing code to use ticksl: change hardclock() to update ticksl instead of ticks, and then use assembler directives to make ticks and ticksl overlap such that loading ticks gives the bottom 32 bits. This makes it possible to use ticksl in the linuxkpi without having to convert any native code, and without making hardclock() more complicated or expensive. Then, the linuxkpi can be modified to use ticksl instead of ticks. Reviewed by: olce, kib, emaste MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D48383 (cherry picked from commit 6b82130e6c9add4a8892ca897df5a0ec04663ea2) --- sys/conf/files | 1 + sys/kern/kern_clock.c | 26 ++++++++++++++------------ sys/kern/kern_tc.c | 4 ++-- sys/kern/subr_param.c | 2 +- sys/kern/subr_ticks.s | 44 ++++++++++++++++++++++++++++++++++++++++++++ sys/sys/kernel.h | 9 +++++++++ sys/sys/timetc.h | 2 +- 7 files changed, 72 insertions(+), 16 deletions(-) diff --git a/sys/conf/files b/sys/conf/files index b571db1e27e8..a1a919bbb3b3 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -3905,6 +3905,7 @@ kern/subr_stack.c optional ddb | stack | ktr kern/subr_stats.c optional stats kern/subr_taskqueue.c standard kern/subr_terminal.c optional vt +kern/subr_ticks.s standard kern/subr_trap.c standard kern/subr_turnstile.c standard kern/subr_uio.c standard diff --git a/sys/kern/kern_clock.c b/sys/kern/kern_clock.c index 08ba6c1cc838..f29dd5708230 100644 --- a/sys/kern/kern_clock.c +++ b/sys/kern/kern_clock.c @@ -325,7 +325,7 @@ read_cpu_time(long *cp_time) #include -static int watchdog_ticks; +static long watchdog_ticks; static int watchdog_enabled; static void watchdog_fire(void); static void watchdog_config(void *, u_int, int *); @@ -371,10 +371,9 @@ watchdog_attach(void) int stathz; int profhz; int profprocs; -volatile int ticks; int psratio; -DPCPU_DEFINE_STATIC(int, pcputicks); /* Per-CPU version of ticks. */ +DPCPU_DEFINE_STATIC(long, pcputicks); /* Per-CPU version of ticks. */ #ifdef DEVICE_POLLING static int devpoll_run = 0; #endif @@ -482,14 +481,14 @@ hardclock(int cnt, int usermode) struct pstats *pstats; struct thread *td = curthread; struct proc *p = td->td_proc; - int *t = DPCPU_PTR(pcputicks); - int global, i, newticks; + long global, newticks, *t; /* * Update per-CPU and possibly global ticks values. */ + t = DPCPU_PTR(pcputicks); *t += cnt; - global = ticks; + global = atomic_load_long(&ticksl); do { newticks = *t - global; if (newticks <= 0) { @@ -498,7 +497,7 @@ hardclock(int cnt, int usermode) newticks = 0; break; } - } while (!atomic_fcmpset_int(&ticks, &global, *t)); + } while (!atomic_fcmpset_long(&ticksl, &global, *t)); /* * Run current process's virtual and profile time, as needed. @@ -527,8 +526,10 @@ hardclock(int cnt, int usermode) } #endif /* DEVICE_POLLING */ if (watchdog_enabled > 0) { - i = atomic_fetchadd_int(&watchdog_ticks, -newticks); - if (i > 0 && i <= newticks) + long left; + + left = atomic_fetchadd_long(&watchdog_ticks, -newticks); + if (left > 0 && left <= newticks) watchdog_fire(); } intr_event_handle(clk_intr_event, NULL); @@ -542,11 +543,12 @@ hardclock(int cnt, int usermode) void hardclock_sync(int cpu) { - int *t; + long *t; + KASSERT(!CPU_ABSENT(cpu), ("Absent CPU %d", cpu)); - t = DPCPU_ID_PTR(cpu, pcputicks); - *t = ticks; + t = DPCPU_ID_PTR(cpu, pcputicks); + *t = ticksl; } /* diff --git a/sys/kern/kern_tc.c b/sys/kern/kern_tc.c index 26f09cb60260..a797a101bf6f 100644 --- a/sys/kern/kern_tc.c +++ b/sys/kern/kern_tc.c @@ -1916,9 +1916,9 @@ SYSCTL_INT(_kern_timecounter, OID_AUTO, tick, CTLFLAG_RD, &tc_tick, 0, "Approximate number of hardclock ticks in a millisecond"); void -tc_ticktock(int cnt) +tc_ticktock(long cnt) { - static int count; + static long count; if (mtx_trylock_spin(&tc_setclock_mtx)) { count += cnt; diff --git a/sys/kern/subr_param.c b/sys/kern/subr_param.c index cee70a842735..67dc10213a70 100644 --- a/sys/kern/subr_param.c +++ b/sys/kern/subr_param.c @@ -198,7 +198,7 @@ init_param1(void) * Arrange for ticks to wrap 10 minutes after boot to help catch * sign problems sooner. */ - ticks = INT_MAX - (hz * 10 * 60); + ticksl = INT_MAX - (hz * 10 * 60); vn_lock_pair_pause_max = hz / 100; if (vn_lock_pair_pause_max == 0) diff --git a/sys/kern/subr_ticks.s b/sys/kern/subr_ticks.s new file mode 100644 index 000000000000..6565ba424137 --- /dev/null +++ b/sys/kern/subr_ticks.s @@ -0,0 +1,44 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2025 Mark Johnston + */ + +/* + * Define the "ticks" and "ticksl" variables. The former is overlaid onto the + * low bits of the latter. + */ + +#if defined(__aarch64__) +#include +#include + +GNU_PROPERTY_AARCH64_FEATURE_1_NOTE(GNU_PROPERTY_AARCH64_FEATURE_1_VAL) +#endif + +#ifdef _ILP32 +#define SIZEOF_TICKSL 4 +#define TICKSL_INIT .long 0 +#else +#define SIZEOF_TICKSL 8 +#define TICKSL_INIT .quad 0 +#endif + +#if defined(_ILP32) || __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ +#define TICKS_OFFSET 0 +#else +#define TICKS_OFFSET 4 +#endif + + .data + + .global ticksl + .type ticksl, %object + .align SIZEOF_TICKSL +ticksl: TICKSL_INIT + .size ticksl, SIZEOF_TICKSL + + .global ticks + .type ticks, %object +ticks =ticksl + TICKS_OFFSET + .size ticks, 4 diff --git a/sys/sys/kernel.h b/sys/sys/kernel.h index e07dc7031bb8..d6b5f8ec10dc 100644 --- a/sys/sys/kernel.h +++ b/sys/sys/kernel.h @@ -67,7 +67,16 @@ extern int psratio; /* ratio: prof / stat */ extern int stathz; /* statistics clock's frequency */ extern int profhz; /* profiling clock's frequency */ extern int profprocs; /* number of process's profiling */ + +/* + * The ticks and ticksl symbols overlap, giving a 64-bit tick counter on 64-bit + * platforms while still maintaining compatibility with the legacy 32-bit + * counter. Either value can be used, but rollover must be handled; at 1000Hz, + * ticks (and ticksl on 32-bit platforms) roll over roughly every 25 days. On + * 64-bit platforms, ticksl will not roll over in the foreseeable future. + */ extern volatile int ticks; +extern volatile long ticksl; #endif /* _KERNEL */ diff --git a/sys/sys/timetc.h b/sys/sys/timetc.h index fed3284e21b5..2f72f8b2527a 100644 --- a/sys/sys/timetc.h +++ b/sys/sys/timetc.h @@ -87,7 +87,7 @@ extern int tc_min_ticktock_freq; /* u_int64_t tc_getfrequency(void); void tc_init(struct timecounter *tc); void tc_setclock(struct timespec *ts); -void tc_ticktock(int cnt); +void tc_ticktock(long cnt); void cpu_tick_calibration(void); #ifdef SYSCTL_DECL