From nobody Tue Apr 21 14:11:58 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g0PRy5JcBz6WkYq for ; Tue, 21 Apr 2026 14:11:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g0PRy3v7Pz45Hb for ; Tue, 21 Apr 2026 14:11:58 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1776780718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fz/HbAagoL9yrqjhCVbqVCzgIR3xMk2OffIg6bfMfvs=; b=aMlirE5njyEE33y8Lfay4hVQHtLQ/CkROvr6ZXhXsSSNKZvehtu0VF3vGyCorDuvdrtG/d H6NdNsFtkppyhlsGl3kM7IhCxfeCdS2tNJvP/304XVhG9r4pehRPuSUPeEW51fBgp1TRcz MxKskXYA2En04xy2O/1yYtllSNwlB+JWoPOqJQDNv902LgcCNwB0RmuF5JB4Z3W/g6ZrnR GAJOeNjJxbH2Q8DuMtSNXIVRe7tdXx24F9GNk9V//yHWxeYBRa2jGs8mV5zYv0Fa27KPY0 5LxW9mng2IURcAKkBdx5WmHLQMkVuLwrH244s2xDTwIfBZnmGDiEapVGrqHuVQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1776780718; a=rsa-sha256; cv=none; b=i/akaX7WJUGCn/QdS3GoFfOhN11awplrFBDs5Tzox3ZejPLRivh0oAHkdL7ZjICEuT4dHp HshNXlmrR71eSBxD3Twctijy+YOh6EDgAl+HbNoAeBEm8UFNyLpf7Sdqx5qDVcHsaJDpfc h+6vcyWEqWnLYNoYXBH4yMo/NHjeFkAT5MB5ULyOlMAlkYASyVZBZ+R61VCu9k5QgZK/q/ htCg72g4/ZHGCPQNzblic1ef0U5/mb8mikH6T3ROW3saofj415OQZYuVhXRZrSlUYDwuG5 e2K7v1RqqtksqNi5checdXE7UrUoQwNe/nLquuCGxbp9ZQDjkBX+FUVHz76zpg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1776780718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fz/HbAagoL9yrqjhCVbqVCzgIR3xMk2OffIg6bfMfvs=; b=v5Fb4zgATRWJprJvWUOo8MBBydlzwqHzafxiHPTpZVwINCzhZit+9U3Q7BpF7wPsgIVpvD 2pIjAOmQcmmH+Q7q2JKVGkRIzR3azb1t9/aVAvcba1ZlsrHKog/SkHSMX9SwLnf2/whrVS v/8fehpgE8Yv4vX1PvZB+zxuR9/XaW9qd15UyR4Z5ZVuT2JuqjjXrRt/csjoWknZnHDxqG 2p5kTuFvkbhtLRwlhn9eRpvG0yDYm+LZQSgzf1o6DOJbUPpgcivcSnyNxVPmYlVs+qy1Uz 1WOgW/XVKX/Ogx33yEq+dp6/SY+ZaKcBwuUhZ5w9aAZuezsBK4qHxsIBRx1VbQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4g0PRy3QwRznm5 for ; Tue, 21 Apr 2026 14:11:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 24088 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 21 Apr 2026 14:11:58 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Ali Mashtizadeh From: Mitchell Horne Subject: git: fd45987f1f66 - main - hwpmc: Implement IBS process sampling List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mhorne X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: fd45987f1f66fe01b0cecede9e90c8aea5cae87e Auto-Submitted: auto-generated Date: Tue, 21 Apr 2026 14:11:58 +0000 Message-Id: <69e785ae.24088.6767c652@gitrepo.freebsd.org> The branch main has been updated by mhorne: URL: https://cgit.FreeBSD.org/src/commit/?id=fd45987f1f66fe01b0cecede9e90c8aea5cae87e commit fd45987f1f66fe01b0cecede9e90c8aea5cae87e Author: Ali Mashtizadeh AuthorDate: 2026-04-07 23:30:51 +0000 Commit: Mitchell Horne CommitDate: 2026-04-21 14:11:41 +0000 hwpmc: Implement IBS process sampling This change enables process-wide sampling to work with IBS by ensuring that read/write only gets or sets the current counter. Reviewed by: mhorne Sponsored by: Netflix Pull Request: https://github.com/freebsd/freebsd-src/pull/2131 --- sys/dev/hwpmc/hwpmc_ibs.c | 28 +++++++++++++++++++++++++--- sys/dev/hwpmc/hwpmc_ibs.h | 8 ++++++++ 2 files changed, 33 insertions(+), 3 deletions(-) diff --git a/sys/dev/hwpmc/hwpmc_ibs.c b/sys/dev/hwpmc/hwpmc_ibs.c index a230288f157e..574df12093f8 100644 --- a/sys/dev/hwpmc/hwpmc_ibs.c +++ b/sys/dev/hwpmc/hwpmc_ibs.c @@ -82,13 +82,13 @@ ibs_read_pmc(int cpu, int ri, struct pmc *pm, pmc_value_t *v) KASSERT(ibs_pcpu[cpu], ("[ibs,%d] null per-cpu, cpu %d", __LINE__, cpu)); - /* read the IBS ctl */ + /* read the IBS count */ switch (ri) { case IBS_PMC_FETCH: - *v = rdmsr(IBS_FETCH_CTL); + *v = IBS_FETCH_CTL_TO_COUNT(rdmsr(IBS_FETCH_CTL)); break; case IBS_PMC_OP: - *v = rdmsr(IBS_OP_CTL); + *v = IBS_OP_CTL_TO_COUNT(rdmsr(IBS_OP_CTL)); break; } @@ -103,12 +103,31 @@ ibs_read_pmc(int cpu, int ri, struct pmc *pm, pmc_value_t *v) static int ibs_write_pmc(int cpu, int ri, struct pmc *pm, pmc_value_t v) { + pmc_value_t m; KASSERT(cpu >= 0 && cpu < pmc_cpu_max(), ("[ibs,%d] illegal CPU value %d", __LINE__, cpu)); KASSERT(ri >= 0 && ri < IBS_NPMCS, ("[ibs,%d] illegal row-index %d", __LINE__, ri)); + /* write the IBS count */ + switch (ri) { + case IBS_PMC_FETCH: + m = rdmsr(IBS_FETCH_CTL) & ~IBS_FETCH_CTL_CURCNTMASK; + /* Setting a count greater than interval is undefined. */ + if (IBS_FETCH_CTL_TO_INTERVAL(m) > v) + m |= IBS_FETCH_COUNT_TO_CTL(v); + wrmsr(IBS_FETCH_CTL, m); + break; + case IBS_PMC_OP: + m = rdmsr(IBS_OP_CTL) & ~IBS_OP_CTL_CURCNTMASK; + /* Setting a count greater than interval is undefined */ + if (IBS_OP_CTL_TO_INTERVAL(m) > v) + m |= IBS_OP_COUNT_TO_CTL(v); + wrmsr(IBS_OP_CTL, m); + break; + } + PMCDBG3(MDP, WRI, 1, "ibs-write cpu=%d ri=%d v=%jx", cpu, ri, v); return (0); @@ -177,6 +196,9 @@ ibs_allocate_pmc(int cpu __unused, int ri, struct pmc *pm, if ((caps & PMC_CAP_SYSTEM) == 0) return (EINVAL); + if (!PMC_IS_SAMPLING_MODE(a->pm_mode)) + return (EINVAL); + config = a->pm_md.pm_ibs.ibs_ctl; pm->pm_md.pm_ibs.ibs_ctl = config; diff --git a/sys/dev/hwpmc/hwpmc_ibs.h b/sys/dev/hwpmc/hwpmc_ibs.h index 1616a746ffef..4cb69503bd8b 100644 --- a/sys/dev/hwpmc/hwpmc_ibs.h +++ b/sys/dev/hwpmc/hwpmc_ibs.h @@ -92,9 +92,13 @@ #define IBS_FETCH_CTL_VALID (1ULL << 49) /* Valid */ #define IBS_FETCH_CTL_ENABLE (1ULL << 48) /* Enable */ #define IBS_FETCH_CTL_MAXCNTMASK 0x0000FFFFULL +#define IBS_FETCH_CTL_CURCNTMASK 0xFFFF0000ULL #define IBS_FETCH_INTERVAL_TO_CTL(_c) (((_c) >> 4) & 0x0000FFFF) +#define IBS_FETCH_CTL_TO_INTERVAL(_c) (((_c) & IBS_FETCH_CTL_MAXCNTMASK) << 4) #define IBS_FETCH_CTL_TO_LAT(_c) (((_c) >> 32) & 0x0000FFFF) +#define IBS_FETCH_COUNT_TO_CTL(_c) (((_c) << 12) & IBS_FETCH_CTL_CURCNTMASK) +#define IBS_FETCH_CTL_TO_COUNT(_c) (((_c) & IBS_FETCH_CTL_CURCNTMASK) >> 12) #define IBS_FETCH_LINADDR 0xC0011031 /* Fetch Linear Address */ #define IBS_FETCH_PHYSADDR 0xC0011032 /* Fetch Physical Address */ @@ -113,9 +117,13 @@ #define IBS_OP_CTL_ENABLE (1ULL << 17) /* Enable */ #define IBS_OP_CTL_L3MISSONLY (1ULL << 16) /* L3 Miss Filtering */ #define IBS_OP_CTL_MAXCNTMASK 0x07F0FFFFULL +#define IBS_OP_CTL_CURCNTMASK 0x07FFFFFF00000000ULL #define IBS_OP_CTL_LDLAT_TO_CTL(_c) ((((ldlat) >> 7) - 1) << 59) #define IBS_OP_INTERVAL_TO_CTL(_c) ((((_c) >> 4) & 0x0000FFFFULL) | ((_c) & 0x07F00000)) +#define IBS_OP_CTL_TO_INTERVAL(_c) ((((_c) & 0x0000FFFFULL) << 4) | ((_c) & 0x07F00000)) +#define IBS_OP_COUNT_TO_CTL(_c) (((_c) << 32) & IBS_OP_CTL_CURCNTMASK) +#define IBS_OP_CTL_TO_COUNT(_c) (((_c) & IBS_OP_CTL_CURCNTMASK) >> 32) #define IBS_OP_RIP 0xC0011034 /* IBS Op RIP */ #define IBS_OP_DATA 0xC0011035 /* IBS Op Data */