From nobody Mon Aug 22 13:47:50 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MBDFf60ZTz4Zgxw; Mon, 22 Aug 2022 13:47:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MBDFf5THwz444V; Mon, 22 Aug 2022 13:47:50 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1661176070; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4fBbmPIqLelb1+hgFQFbgEA+h6YauvJBRIDfye69580=; b=avrER6KNy7PrP5Vtr/hNncfDDb1vg9SEw7nL+H6Y2WfXS8vKEV5yLV4PQE0pa+xBQ1HpW6 cZbYNxryQLurpJu9xmKPmWHrBJeYS7IpLjs0aSPBvnECPxPzE/pFG7U66JZBz9Icqf/8sS Msqhg6B36E91VcccsB5i1uZ04m3GVtwGIq86TpzDL8oqb4msXjEaeKeJJ5kz6ElaZ7ZnyD hKi9UA+9hv4QIWMoV0XRKj2koOAt0Ru3SJofrqv42x7ittWi0zJApwuo8FqbRJTcIkansc 5IS2f0tQhRs5gAkj0tWcRALnjgPC+r9GdVwiRPaFnNTmvw/D0aUNLr/PpJWpPw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MBDFf4QqQzsCq; Mon, 22 Aug 2022 13:47:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 27MDloap028452; Mon, 22 Aug 2022 13:47:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 27MDloeX028451; Mon, 22 Aug 2022 13:47:50 GMT (envelope-from git) Date: Mon, 22 Aug 2022 13:47:50 GMT Message-Id: <202208221347.27MDloeX028451@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: d4f7acfad732 - stable/13 - iommu_gas: add iommu_gas_remove() List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d4f7acfad7322852a85cded19af7ebb737119993 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1661176070; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4fBbmPIqLelb1+hgFQFbgEA+h6YauvJBRIDfye69580=; b=RyH7LZ40vx6mdarDV007JHeWx3537DxwdWsqCaSMd0qPFMH4Pr4war5zn7Vn4mCSscCO55 SHKBs1+zO+ZG8UUQKsi5nxaBj1sSpgufH8qlFP8o/AGUatQKKipHDrSvz+eM3J4EUorviu HJHCvbHg/wH/7eGrZRDLlwjHPTuIVhYJ3r6nS+HKr8FCFob4HGSozCAOetRaZusa64BqWy cwovq47OmeRSumzMOI++enzZ0O4XiOE/Hd6KSktmT1D2BZ9TrJeG8x6T+9PT0l+Dtc0V56 9znMXBQYmbak0CpyWYc7DJSVodiCXLnCXC4iekwi6HkgkTt3//gUFrMX2XwvvA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1661176070; a=rsa-sha256; cv=none; b=qPBnwBVqlwuTenufyBt9pVr37B7niefXTRpoBxZDhqzz3Fpx5HAC733x4IGBDaeEIz3DV1 PkwYoaTGHKkaJ1eP77udptG13V3Xx579DlxXuiJaMCTS3SEulam9tTqkpaYtIhSxmK6l5l yzAywO2w885oDCrZVZejsbdtaHZtvwptKzunTL7SJqUkUwaYKGEHirnyiQJso5Hgo6+A/Y XN38mXStrV/lH2EvQCIC/Bt862RqUYaZXCp4Og4qkrrW16cC6XVEE0S302oQWEDAW7Tp0e XN27Uo2mP86icgMC4gEFQQmF2ON8rguyQIuhbI/lIlNrHfAjwRxUXVI0lyDAvA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=d4f7acfad7322852a85cded19af7ebb737119993 commit d4f7acfad7322852a85cded19af7ebb737119993 Author: Konstantin Belousov AuthorDate: 2022-07-31 08:43:43 +0000 Commit: Konstantin Belousov CommitDate: 2022-08-22 13:37:09 +0000 iommu_gas: add iommu_gas_remove() (cherry picked from commit c9e4d25052065d1e0cbdb3a0eea3f21c9225d1d1) --- sys/dev/iommu/iommu.h | 2 + sys/dev/iommu/iommu_gas.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++ sys/dev/iommu/iommu_gas.h | 2 + 3 files changed, 122 insertions(+) diff --git a/sys/dev/iommu/iommu.h b/sys/dev/iommu/iommu.h index ae4022c5c4f7..2531ef09d9cd 100644 --- a/sys/dev/iommu/iommu.h +++ b/sys/dev/iommu/iommu.h @@ -171,6 +171,8 @@ struct iommu_map_entry *iommu_gas_alloc_entry(struct iommu_domain *domain, u_int flags); void iommu_gas_free_entry(struct iommu_map_entry *entry); void iommu_gas_free_space(struct iommu_map_entry *entry); +void iommu_gas_remove(struct iommu_domain *domain, iommu_gaddr_t start, + iommu_gaddr_t size); int iommu_gas_map(struct iommu_domain *domain, const struct bus_dma_tag_common *common, iommu_gaddr_t size, int offset, u_int eflags, u_int flags, vm_page_t *ma, struct iommu_map_entry **res); diff --git a/sys/dev/iommu/iommu_gas.c b/sys/dev/iommu/iommu_gas.c index bad56ab9140e..ca5a614060fe 100644 --- a/sys/dev/iommu/iommu_gas.c +++ b/sys/dev/iommu/iommu_gas.c @@ -600,6 +600,124 @@ iommu_gas_free_region(struct iommu_map_entry *entry) IOMMU_DOMAIN_UNLOCK(domain); } +static struct iommu_map_entry * +iommu_gas_remove_clip_left(struct iommu_domain *domain, iommu_gaddr_t start, + iommu_gaddr_t end, struct iommu_map_entry **r) +{ + struct iommu_map_entry *entry, *res, fentry; + + IOMMU_DOMAIN_ASSERT_LOCKED(domain); + MPASS(start <= end); + MPASS(end <= domain->last_place->end); + + /* + * Find an entry which contains the supplied guest's address + * start, or the first entry after the start. Since we + * asserted that start is below domain end, entry should + * exist. Then clip it if needed. + */ + fentry.start = start + 1; + fentry.end = start + 1; + entry = RB_NFIND(iommu_gas_entries_tree, &domain->rb_root, &fentry); + + if (entry->start >= start || + (entry->flags & IOMMU_MAP_ENTRY_RMRR) != 0) + return (entry); + + res = *r; + *r = NULL; + *res = *entry; + res->start = entry->end = start; + RB_UPDATE_AUGMENT(entry, rb_entry); + iommu_gas_rb_insert(domain, res); + return (res); +} + +static bool +iommu_gas_remove_clip_right(struct iommu_domain *domain, + iommu_gaddr_t end, struct iommu_map_entry *entry, + struct iommu_map_entry *r) +{ + if (entry->start >= end || (entry->flags & IOMMU_MAP_ENTRY_RMRR) != 0) + return (false); + + *r = *entry; + r->end = entry->start = end; + RB_UPDATE_AUGMENT(entry, rb_entry); + iommu_gas_rb_insert(domain, r); + return (true); +} + +static void +iommu_gas_remove_unmap(struct iommu_domain *domain, + struct iommu_map_entry *entry, struct iommu_map_entries_tailq *gcp) +{ + IOMMU_DOMAIN_ASSERT_LOCKED(domain); + + if ((entry->flags & (IOMMU_MAP_ENTRY_UNMAPPED | + IOMMU_MAP_ENTRY_REMOVING)) != 0) + return; + MPASS((entry->flags & IOMMU_MAP_ENTRY_PLACE) == 0); + entry->flags |= IOMMU_MAP_ENTRY_REMOVING; + TAILQ_INSERT_TAIL(gcp, entry, dmamap_link); +} + +/* + * Remove specified range from the GAS of the domain. Note that the + * removal is not guaranteed to occur upon the function return, it + * might be finalized some time after, when hardware reports that + * (queued) IOTLB invalidation was performed. + */ +void +iommu_gas_remove(struct iommu_domain *domain, iommu_gaddr_t start, + iommu_gaddr_t size) +{ + struct iommu_map_entry *entry, *nentry, *r1, *r2; + struct iommu_map_entries_tailq gc; + iommu_gaddr_t end; + + end = start + size; + r1 = iommu_gas_alloc_entry(domain, IOMMU_PGF_WAITOK); + r2 = iommu_gas_alloc_entry(domain, IOMMU_PGF_WAITOK); + TAILQ_INIT(&gc); + + IOMMU_DOMAIN_LOCK(domain); + + nentry = iommu_gas_remove_clip_left(domain, start, end, &r1); + RB_FOREACH_FROM(entry, iommu_gas_entries_tree, nentry) { + if (entry->start >= end) + break; + KASSERT(start <= entry->start, + ("iommu_gas_remove entry (%#jx, %#jx) start %#jx", + entry->start, entry->end, start)); + if ((entry->flags & IOMMU_MAP_ENTRY_RMRR) != 0) + continue; + iommu_gas_remove_unmap(domain, entry, &gc); + } + if (iommu_gas_remove_clip_right(domain, end, entry, r2)) { + iommu_gas_remove_unmap(domain, r2, &gc); + r2 = NULL; + } + +#ifdef INVARIANTS + RB_FOREACH(entry, iommu_gas_entries_tree, &domain->rb_root) { + if ((entry->flags & IOMMU_MAP_ENTRY_RMRR) != 0) + continue; + KASSERT(entry->end <= start || entry->start >= end, + ("iommu_gas_remove leftover entry (%#jx, %#jx) range " + "(%#jx, %#jx)", + entry->start, entry->end, start, end)); + } +#endif + + IOMMU_DOMAIN_UNLOCK(domain); + if (r1 != NULL) + iommu_gas_free_entry(r1); + if (r2 != NULL) + iommu_gas_free_entry(r2); + iommu_domain_unload(domain, &gc, true); +} + int iommu_gas_map(struct iommu_domain *domain, const struct bus_dma_tag_common *common, iommu_gaddr_t size, int offset, diff --git a/sys/dev/iommu/iommu_gas.h b/sys/dev/iommu/iommu_gas.h index a9d0df5f272f..75a57ec0478f 100644 --- a/sys/dev/iommu/iommu_gas.h +++ b/sys/dev/iommu/iommu_gas.h @@ -50,6 +50,8 @@ #define IOMMU_MAP_ENTRY_MAP 0x0004 /* Busdma created, linked by dmamap_link */ #define IOMMU_MAP_ENTRY_UNMAPPED 0x0010 /* No backing pages */ +#define IOMMU_MAP_ENTRY_REMOVING 0x0020 /* In process of removal by + iommu_gas_remove() */ #define IOMMU_MAP_ENTRY_READ 0x1000 /* Read permitted */ #define IOMMU_MAP_ENTRY_WRITE 0x2000 /* Write permitted */ #define IOMMU_MAP_ENTRY_SNOOP 0x4000 /* Snoop */