From nobody Wed Sep 11 08:44:21 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X3Yys5wYZz5W7bM; Wed, 11 Sep 2024 08:44:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X3Yys4FGbz41hq; Wed, 11 Sep 2024 08:44:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726044261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=56dAryMk2lww0e/OtRizUD3ascAUkCOp9Oj2nYnh8Rc=; b=gs2qt873a3lXXzU+E3droAbLx0btDJIObgsQ1OsYuYq0zrxC4gESSRQwLSajRbIXcl83ic 2sSI8agO/6UA9ZpKaAXBG5ts36Z60a8n/KiNxtBWuG0RLuUYqQcMMM5Ll/ZHvVTvMYzo0n qLNs/UlXsoLR+ySsDgwUSHcHEAZ/pO7bDWBd5u0+a2+35gxioA84WZab3sXvOYArY7M/qK 15FK63x9CE1eDL3MD2bgwlfsNtKsEBK6AiywelHxjka4Z06TSaqAw+WNe5rlOi2Pdlu94H DfyDUaPyGU8cbHGSmuj9R+orrHuO7cyMG23Ofzl5HaMJl5aWbMUppUxUh74eOQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1726044261; a=rsa-sha256; cv=none; b=pAwBkjy9jzlwrvU0PsjzEIuWeNIqRrsMqLk5MsZGsvuT7p1cWBU+ByvCneD7Jx8uSjfnRO cwa7BuqaVD5mTLkE1TFhvqfiMi6RrIy2JSupvcaHiKHRk8BGIcQxf2WgTeHmBQkpJlPvE0 PLKn1Q+4cwicLEwG5LM/HSoFLU9bKu0H7vocpMOMCQT3Aa7uQB6MsGpdLB0oo+gIX4P1I1 MVF338WPvh2dEWQ+NzrrCvriY8hzIFamQvIBeUtETDOdzL3Gs4oPpKhg0SDauvD9WUFx3X ss20hhBYY6EKoe1mpcy0hq4KAG8vUvzfuIPeRFFyC4LcLEGKAnSRw/jwnEzZ1Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726044261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=56dAryMk2lww0e/OtRizUD3ascAUkCOp9Oj2nYnh8Rc=; b=CL7lrCGgrsXBDgCnR/EvegamSZIwBnlE+TCOkE/wVgSgd26puEmGq3TFyW5WqyZiuAtSNf q4oECiEmpjH0Wv6d5H42pbXpgvPMaagy5VOdNDBAl4asG3kI9J7m5kh41DZjWgMi9LR7r7 BAhUwCf9jzwacUFkkYBxPJpJTXfjMwbq9dAK3pDH19MRowRPckJ9j88ESZclWhsCBFdjJA 1dgo6LO3v7jH0xry3eJD7Iu1rxHCkW+Q9KdKKRSP27JrUPL90O1j3mIhD4SaWk9xrkERDN 0AWNViAxnoB158mQeyOYyHVl7xDc0WM+m2ypKvHcq54TV8bjL3NNsDyoxYFUOQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4X3Yys3YwVzP9h; Wed, 11 Sep 2024 08:44:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 48B8iLKm007349; Wed, 11 Sep 2024 08:44:21 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 48B8iL2P007346; Wed, 11 Sep 2024 08:44:21 GMT (envelope-from git) Date: Wed, 11 Sep 2024 08:44:21 GMT Message-Id: <202409110844.48B8iL2P007346@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 3f5d7c42ce8b - stable/14 - x86: allow to have more than one kind of IOMMU List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 3f5d7c42ce8b42d454d708518da670ddb53f9abd Auto-Submitted: auto-generated The branch stable/14 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=3f5d7c42ce8b42d454d708518da670ddb53f9abd commit 3f5d7c42ce8b42d454d708518da670ddb53f9abd Author: Konstantin Belousov AuthorDate: 2024-05-29 14:38:14 +0000 Commit: Konstantin Belousov CommitDate: 2024-09-11 08:43:59 +0000 x86: allow to have more than one kind of IOMMU (cherry picked from commit 65b133e5d292686fe20f11dd39b53812226a8684) --- sys/x86/iommu/intel_ctx.c | 22 +++++------ sys/x86/iommu/intel_dmar.h | 18 ++++++++- sys/x86/iommu/intel_drv.c | 30 ++++++++++++-- sys/x86/iommu/intel_intrmap.c | 10 ++--- sys/x86/iommu/iommu_utils.c | 91 ++++++++++++++++++++++++++++++++++++++++++- sys/x86/iommu/x86_iommu.h | 25 +++++++++++- 6 files changed, 173 insertions(+), 23 deletions(-) diff --git a/sys/x86/iommu/intel_ctx.c b/sys/x86/iommu/intel_ctx.c index 444640570df7..9fd632d782f8 100644 --- a/sys/x86/iommu/intel_ctx.c +++ b/sys/x86/iommu/intel_ctx.c @@ -76,6 +76,9 @@ static void dmar_unref_domain_locked(struct dmar_unit *dmar, struct dmar_domain *domain); static void dmar_domain_destroy(struct dmar_domain *domain); +static void dmar_free_ctx_locked(struct dmar_unit *dmar, struct dmar_ctx *ctx); +static void dmar_free_ctx(struct dmar_ctx *ctx); + static void dmar_ensure_ctx_page(struct dmar_unit *dmar, int bus) { @@ -747,7 +750,7 @@ dmar_unref_domain_locked(struct dmar_unit *dmar, struct dmar_domain *domain) dmar_domain_destroy(domain); } -void +static void dmar_free_ctx_locked(struct dmar_unit *dmar, struct dmar_ctx *ctx) { struct sf_buf *sf; @@ -821,7 +824,7 @@ dmar_free_ctx_locked(struct dmar_unit *dmar, struct dmar_ctx *ctx) TD_PINNED_ASSERT; } -void +static void dmar_free_ctx(struct dmar_ctx *ctx) { struct dmar_unit *dmar; @@ -869,7 +872,7 @@ dmar_domain_free_entry(struct iommu_map_entry *entry, bool free) * the entry's dmamap_link field. */ void -iommu_domain_unload_entry(struct iommu_map_entry *entry, bool free, +dmar_domain_unload_entry(struct iommu_map_entry *entry, bool free, bool cansleep) { struct dmar_domain *domain; @@ -911,7 +914,7 @@ dmar_domain_unload_emit_wait(struct dmar_domain *domain, } void -iommu_domain_unload(struct iommu_domain *iodom, +dmar_domain_unload(struct iommu_domain *iodom, struct iommu_map_entries_tailq *entries, bool cansleep) { struct dmar_domain *domain; @@ -949,37 +952,34 @@ iommu_domain_unload(struct iommu_domain *iodom, } struct iommu_ctx * -iommu_get_ctx(struct iommu_unit *iommu, device_t dev, uint16_t rid, +dmar_get_ctx(struct iommu_unit *iommu, device_t dev, uint16_t rid, bool id_mapped, bool rmrr_init) { struct dmar_unit *dmar; struct dmar_ctx *ret; dmar = IOMMU2DMAR(iommu); - ret = dmar_get_ctx_for_dev(dmar, dev, rid, id_mapped, rmrr_init); - return (CTX2IOCTX(ret)); } void -iommu_free_ctx_locked(struct iommu_unit *iommu, struct iommu_ctx *context) +dmar_free_ctx_locked_method(struct iommu_unit *iommu, + struct iommu_ctx *context) { struct dmar_unit *dmar; struct dmar_ctx *ctx; dmar = IOMMU2DMAR(iommu); ctx = IOCTX2CTX(context); - dmar_free_ctx_locked(dmar, ctx); } void -iommu_free_ctx(struct iommu_ctx *context) +dmar_free_ctx_method(struct iommu_ctx *context) { struct dmar_ctx *ctx; ctx = IOCTX2CTX(context); - dmar_free_ctx(ctx); } diff --git a/sys/x86/iommu/intel_dmar.h b/sys/x86/iommu/intel_dmar.h index 4ae005238074..0ede955e12b9 100644 --- a/sys/x86/iommu/intel_dmar.h +++ b/sys/x86/iommu/intel_dmar.h @@ -293,10 +293,17 @@ struct dmar_ctx *dmar_get_ctx_for_devpath(struct dmar_unit *dmar, uint16_t rid, int dev_domain, int dev_busno, const void *dev_path, int dev_path_len, bool id_mapped, bool rmrr_init); int dmar_move_ctx_to_domain(struct dmar_domain *domain, struct dmar_ctx *ctx); -void dmar_free_ctx_locked(struct dmar_unit *dmar, struct dmar_ctx *ctx); -void dmar_free_ctx(struct dmar_ctx *ctx); +void dmar_free_ctx_locked_method(struct iommu_unit *dmar, + struct iommu_ctx *ctx); +void dmar_free_ctx_method(struct iommu_ctx *ctx); struct dmar_ctx *dmar_find_ctx_locked(struct dmar_unit *dmar, uint16_t rid); +struct iommu_ctx *dmar_get_ctx(struct iommu_unit *iommu, device_t dev, + uint16_t rid, bool id_mapped, bool rmrr_init); void dmar_domain_free_entry(struct iommu_map_entry *entry, bool free); +void dmar_domain_unload_entry(struct iommu_map_entry *entry, bool free, + bool cansleep); +void dmar_domain_unload(struct iommu_domain *iodom, + struct iommu_map_entries_tailq *entries, bool cansleep); void dmar_dev_parse_rmrr(struct dmar_domain *domain, int dev_domain, int dev_busno, const void *dev_path, int dev_path_len, @@ -308,6 +315,13 @@ void dmar_quirks_pre_use(struct iommu_unit *dmar); int dmar_init_irt(struct dmar_unit *unit); void dmar_fini_irt(struct dmar_unit *unit); +int dmar_alloc_msi_intr(device_t src, u_int *cookies, u_int count); +int dmar_map_msi_intr(device_t src, u_int cpu, u_int vector, u_int cookie, + uint64_t *addr, uint32_t *data); +int dmar_unmap_msi_intr(device_t src, u_int cookie); +int dmar_map_ioapic_intr(u_int ioapic_id, u_int cpu, u_int vector, bool edge, + bool activehi, int irq, u_int *cookie, uint32_t *hi, uint32_t *lo); +int dmar_unmap_ioapic_intr(u_int ioapic_id, u_int *cookie); extern int haw; extern int dmar_batch_coalesce; diff --git a/sys/x86/iommu/intel_drv.c b/sys/x86/iommu/intel_drv.c index 636534173715..79350358cced 100644 --- a/sys/x86/iommu/intel_drv.c +++ b/sys/x86/iommu/intel_drv.c @@ -64,6 +64,8 @@ #include #include #include +#include +#include #include #include #include @@ -1357,12 +1359,34 @@ DB_SHOW_ALL_COMMAND(dmars, db_show_all_dmars) } #endif -struct iommu_unit * -iommu_find(device_t dev, bool verbose) +static struct iommu_unit * +dmar_find_method(device_t dev, bool verbose) { struct dmar_unit *dmar; dmar = dmar_find(dev, verbose); - return (&dmar->iommu); } + +static struct x86_iommu dmar_x86_iommu = { + .domain_unload_entry = dmar_domain_unload_entry, + .domain_unload = dmar_domain_unload, + .get_ctx = dmar_get_ctx, + .free_ctx_locked = dmar_free_ctx_locked_method, + .free_ctx = dmar_free_ctx_method, + .find = dmar_find_method, + .alloc_msi_intr = dmar_alloc_msi_intr, + .map_msi_intr = dmar_map_msi_intr, + .unmap_msi_intr = dmar_unmap_msi_intr, + .map_ioapic_intr = dmar_map_ioapic_intr, + .unmap_ioapic_intr = dmar_unmap_ioapic_intr, +}; + +static void +x86_iommu_set_intel(void *arg __unused) +{ + if (cpu_vendor_id == CPU_VENDOR_INTEL) + set_x86_iommu(&dmar_x86_iommu); +} + +SYSINIT(x86_iommu, SI_SUB_TUNABLES, SI_ORDER_ANY, x86_iommu_set_intel, NULL); diff --git a/sys/x86/iommu/intel_intrmap.c b/sys/x86/iommu/intel_intrmap.c index 137801125c38..1ac51a7aaea0 100644 --- a/sys/x86/iommu/intel_intrmap.c +++ b/sys/x86/iommu/intel_intrmap.c @@ -66,7 +66,7 @@ static void dmar_ir_program_irte(struct dmar_unit *unit, u_int idx, static int dmar_ir_free_irte(struct dmar_unit *unit, u_int cookie); int -iommu_alloc_msi_intr(device_t src, u_int *cookies, u_int count) +dmar_alloc_msi_intr(device_t src, u_int *cookies, u_int count) { struct dmar_unit *unit; vmem_addr_t vmem_res; @@ -94,7 +94,7 @@ iommu_alloc_msi_intr(device_t src, u_int *cookies, u_int count) } int -iommu_map_msi_intr(device_t src, u_int cpu, u_int vector, u_int cookie, +dmar_map_msi_intr(device_t src, u_int cpu, u_int vector, u_int cookie, uint64_t *addr, uint32_t *data) { struct dmar_unit *unit; @@ -140,7 +140,7 @@ iommu_map_msi_intr(device_t src, u_int cpu, u_int vector, u_int cookie, } int -iommu_unmap_msi_intr(device_t src, u_int cookie) +dmar_unmap_msi_intr(device_t src, u_int cookie) { struct dmar_unit *unit; @@ -151,7 +151,7 @@ iommu_unmap_msi_intr(device_t src, u_int cookie) } int -iommu_map_ioapic_intr(u_int ioapic_id, u_int cpu, u_int vector, bool edge, +dmar_map_ioapic_intr(u_int ioapic_id, u_int cpu, u_int vector, bool edge, bool activehi, int irq, u_int *cookie, uint32_t *hi, uint32_t *lo) { struct dmar_unit *unit; @@ -214,7 +214,7 @@ iommu_map_ioapic_intr(u_int ioapic_id, u_int cpu, u_int vector, bool edge, } int -iommu_unmap_ioapic_intr(u_int ioapic_id, u_int *cookie) +dmar_unmap_ioapic_intr(u_int ioapic_id, u_int *cookie) { struct dmar_unit *unit; u_int idx; diff --git a/sys/x86/iommu/iommu_utils.c b/sys/x86/iommu/iommu_utils.c index ffea1cc1a190..ea2c0358e072 100644 --- a/sys/x86/iommu/iommu_utils.c +++ b/sys/x86/iommu/iommu_utils.c @@ -1,7 +1,7 @@ /*- * SPDX-License-Identifier: BSD-2-Clause * - * Copyright (c) 2013, 2014 The FreeBSD Foundation + * Copyright (c) 2013, 2014, 2024 The FreeBSD Foundation * * This software was developed by Konstantin Belousov * under sponsorship from the FreeBSD Foundation. @@ -50,6 +50,7 @@ #include #include #include +#include vm_page_t iommu_pgalloc(vm_object_t obj, vm_pindex_t idx, int flags) @@ -162,3 +163,91 @@ SYSCTL_NODE(_hw_iommu, OID_AUTO, dmar, CTLFLAG_RD | CTLFLAG_MPSAFE, SYSCTL_INT(_hw_iommu_dmar, OID_AUTO, tbl_pagecnt, CTLFLAG_RD, &iommu_tbl_pagecnt, 0, "Count of pages used for DMAR pagetables"); + +static struct x86_iommu *x86_iommu; + +void +set_x86_iommu(struct x86_iommu *x) +{ + MPASS(x86_iommu == NULL); + x86_iommu = x; +} + +struct x86_iommu * +get_x86_iommu(void) +{ + return (x86_iommu); +} + +void +iommu_domain_unload_entry(struct iommu_map_entry *entry, bool free, + bool cansleep) +{ + x86_iommu->domain_unload_entry(entry, free, cansleep); +} + +void +iommu_domain_unload(struct iommu_domain *iodom, + struct iommu_map_entries_tailq *entries, bool cansleep) +{ + x86_iommu->domain_unload(iodom, entries, cansleep); +} + +struct iommu_ctx * +iommu_get_ctx(struct iommu_unit *iommu, device_t dev, uint16_t rid, + bool id_mapped, bool rmrr_init) +{ + return (x86_iommu->get_ctx(iommu, dev, rid, id_mapped, rmrr_init)); +} + +void +iommu_free_ctx_locked(struct iommu_unit *iommu, struct iommu_ctx *context) +{ + x86_iommu->free_ctx_locked(iommu, context); +} + +void +iommu_free_ctx(struct iommu_ctx *context) +{ + x86_iommu->free_ctx(context); +} + +struct iommu_unit * +iommu_find(device_t dev, bool verbose) +{ + return (x86_iommu->find(dev, verbose)); +} + +int +iommu_alloc_msi_intr(device_t src, u_int *cookies, u_int count) +{ + return (x86_iommu->alloc_msi_intr(src, cookies, count)); +} + +int +iommu_map_msi_intr(device_t src, u_int cpu, u_int vector, u_int cookie, + uint64_t *addr, uint32_t *data) +{ + return (x86_iommu->map_msi_intr(src, cpu, vector, cookie, + addr, data)); +} + +int +iommu_unmap_msi_intr(device_t src, u_int cookie) +{ + return (x86_iommu->unmap_msi_intr(src, cookie)); +} + +int +iommu_map_ioapic_intr(u_int ioapic_id, u_int cpu, u_int vector, bool edge, + bool activehi, int irq, u_int *cookie, uint32_t *hi, uint32_t *lo) +{ + return (x86_iommu->map_ioapic_intr(ioapic_id, cpu, vector, edge, + activehi, irq, cookie, hi, lo)); +} + +int +iommu_unmap_ioapic_intr(u_int ioapic_id, u_int *cookie) +{ + return (x86_iommu->unmap_ioapic_intr(ioapic_id, cookie)); +} diff --git a/sys/x86/iommu/x86_iommu.h b/sys/x86/iommu/x86_iommu.h index 3789586f1eaf..8c908964acd0 100644 --- a/sys/x86/iommu/x86_iommu.h +++ b/sys/x86/iommu/x86_iommu.h @@ -1,7 +1,7 @@ /*- * SPDX-License-Identifier: BSD-2-Clause * - * Copyright (c) 2013-2015 The FreeBSD Foundation + * Copyright (c) 2013-2015, 2024 The FreeBSD Foundation * * This software was developed by Konstantin Belousov * under sponsorship from the FreeBSD Foundation. @@ -59,4 +59,27 @@ extern int iommu_tbl_pagecnt; SYSCTL_DECL(_hw_iommu); SYSCTL_DECL(_hw_iommu_dmar); +struct x86_iommu { + void (*domain_unload_entry)(struct iommu_map_entry *entry, bool free, + bool cansleep); + void (*domain_unload)(struct iommu_domain *iodom, + struct iommu_map_entries_tailq *entries, bool cansleep); + struct iommu_ctx *(*get_ctx)(struct iommu_unit *iommu, + device_t dev, uint16_t rid, bool id_mapped, bool rmrr_init); + void (*free_ctx_locked)(struct iommu_unit *iommu, + struct iommu_ctx *context); + void (*free_ctx)(struct iommu_ctx *context); + struct iommu_unit *(*find)(device_t dev, bool verbose); + int (*alloc_msi_intr)(device_t src, u_int *cookies, u_int count); + int (*map_msi_intr)(device_t src, u_int cpu, u_int vector, + u_int cookie, uint64_t *addr, uint32_t *data); + int (*unmap_msi_intr)(device_t src, u_int cookie); + int (*map_ioapic_intr)(u_int ioapic_id, u_int cpu, u_int vector, + bool edge, bool activehi, int irq, u_int *cookie, uint32_t *hi, + uint32_t *lo); + int (*unmap_ioapic_intr)(u_int ioapic_id, u_int *cookie); +}; +void set_x86_iommu(struct x86_iommu *); +struct x86_iommu *get_x86_iommu(void); + #endif