From owner-svn-src-all@freebsd.org Wed Dec 9 18:43:59 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 578B44AC429; Wed, 9 Dec 2020 18:43:59 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CrmCz21Y4z3kQX; Wed, 9 Dec 2020 18:43:59 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 37E612D61; Wed, 9 Dec 2020 18:43:59 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0B9IhxVE025561; Wed, 9 Dec 2020 18:43:59 GMT (envelope-from rlibby@FreeBSD.org) Received: (from rlibby@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0B9IhwSJ025558; Wed, 9 Dec 2020 18:43:58 GMT (envelope-from rlibby@FreeBSD.org) Message-Id: <202012091843.0B9IhwSJ025558@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rlibby set sender to rlibby@FreeBSD.org using -f From: Ryan Libby Date: Wed, 9 Dec 2020 18:43:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r368490 - in head/sys: dev/iommu x86/iommu X-SVN-Group: head X-SVN-Commit-Author: rlibby X-SVN-Commit-Paths: in head/sys: dev/iommu x86/iommu X-SVN-Commit-Revision: 368490 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2020 18:43:59 -0000 Author: rlibby Date: Wed Dec 9 18:43:58 2020 New Revision: 368490 URL: https://svnweb.freebsd.org/changeset/base/368490 Log: dmar: reserve memory windows of PCIe root port PCI memory address space is shared between memory-mapped devices (MMIO) and host memory (which may be remapped by an IOMMU). Device accesses to an address within a memory aperture in a PCIe root port will be treated as peer-to-peer and not forwarded to an IOMMU. To avoid this, reserve the address space of the root port's memory apertures in the address space used by the IOMMU for remapping. Reviewed by: kib, tychon Discussed with: Anton Rang Tested by: tychon Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D27503 Modified: head/sys/dev/iommu/iommu.h head/sys/dev/iommu/iommu_gas.c head/sys/x86/iommu/intel_ctx.c Modified: head/sys/dev/iommu/iommu.h ============================================================================== --- head/sys/dev/iommu/iommu.h Wed Dec 9 18:37:43 2020 (r368489) +++ head/sys/dev/iommu/iommu.h Wed Dec 9 18:43:58 2020 (r368490) @@ -199,6 +199,8 @@ int iommu_gas_map_region(struct iommu_domain *domain, struct iommu_map_entry *entry, u_int eflags, u_int flags, vm_page_t *ma); int iommu_gas_reserve_region(struct iommu_domain *domain, iommu_gaddr_t start, iommu_gaddr_t end, struct iommu_map_entry **entry0); +int iommu_gas_reserve_region_extend(struct iommu_domain *domain, + iommu_gaddr_t start, iommu_gaddr_t end); void iommu_set_buswide_ctx(struct iommu_unit *unit, u_int busno); bool iommu_is_buswide_ctx(struct iommu_unit *unit, u_int busno); Modified: head/sys/dev/iommu/iommu_gas.c ============================================================================== --- head/sys/dev/iommu/iommu_gas.c Wed Dec 9 18:37:43 2020 (r368489) +++ head/sys/dev/iommu/iommu_gas.c Wed Dec 9 18:43:58 2020 (r368490) @@ -677,6 +677,22 @@ iommu_gas_map_region(struct iommu_domain *domain, stru return (0); } +static int +iommu_gas_reserve_region_locked(struct iommu_domain *domain, + iommu_gaddr_t start, iommu_gaddr_t end, struct iommu_map_entry *entry) +{ + int error; + + IOMMU_DOMAIN_ASSERT_LOCKED(domain); + + entry->start = start; + entry->end = end; + error = iommu_gas_alloc_region(domain, entry, IOMMU_MF_CANWAIT); + if (error == 0) + entry->flags |= IOMMU_MAP_ENTRY_UNMAPPED; + return (error); +} + int iommu_gas_reserve_region(struct iommu_domain *domain, iommu_gaddr_t start, iommu_gaddr_t end, struct iommu_map_entry **entry0) @@ -685,17 +701,63 @@ iommu_gas_reserve_region(struct iommu_domain *domain, int error; entry = iommu_gas_alloc_entry(domain, IOMMU_PGF_WAITOK); - entry->start = start; - entry->end = end; IOMMU_DOMAIN_LOCK(domain); - error = iommu_gas_alloc_region(domain, entry, IOMMU_MF_CANWAIT); - if (error == 0) - entry->flags |= IOMMU_MAP_ENTRY_UNMAPPED; + error = iommu_gas_reserve_region_locked(domain, start, end, entry); IOMMU_DOMAIN_UNLOCK(domain); if (error != 0) iommu_gas_free_entry(domain, entry); else if (entry0 != NULL) *entry0 = entry; + return (error); +} + +/* + * As in iommu_gas_reserve_region, reserve [start, end), but allow for existing + * entries. + */ +int +iommu_gas_reserve_region_extend(struct iommu_domain *domain, + iommu_gaddr_t start, iommu_gaddr_t end) +{ + struct iommu_map_entry *entry, *next, *prev, key = {}; + iommu_gaddr_t entry_start, entry_end; + int error; + + error = 0; + entry = NULL; + end = ummin(end, domain->end); + while (start < end) { + /* Preallocate an entry. */ + if (entry == NULL) + entry = iommu_gas_alloc_entry(domain, + IOMMU_PGF_WAITOK); + /* Calculate the free region from here to the next entry. */ + key.start = key.end = start; + IOMMU_DOMAIN_LOCK(domain); + next = RB_NFIND(iommu_gas_entries_tree, &domain->rb_root, &key); + KASSERT(next != NULL, ("domain %p with end %#jx has no entry " + "after %#jx", domain, (uintmax_t)domain->end, + (uintmax_t)start)); + entry_end = ummin(end, next->start); + prev = RB_PREV(iommu_gas_entries_tree, &domain->rb_root, next); + if (prev != NULL) + entry_start = ummax(start, prev->end); + else + entry_start = start; + start = next->end; + /* Reserve the region if non-empty. */ + if (entry_start != entry_end) { + error = iommu_gas_reserve_region_locked(domain, + entry_start, entry_end, entry); + if (error != 0) + break; + entry = NULL; + } + IOMMU_DOMAIN_UNLOCK(domain); + } + /* Release a preallocated entry if it was not used. */ + if (entry != NULL) + iommu_gas_free_entry(domain, entry); return (error); } Modified: head/sys/x86/iommu/intel_ctx.c ============================================================================== --- head/sys/x86/iommu/intel_ctx.c Wed Dec 9 18:37:43 2020 (r368489) +++ head/sys/x86/iommu/intel_ctx.c Wed Dec 9 18:43:58 2020 (r368490) @@ -317,6 +317,66 @@ domain_init_rmrr(struct dmar_domain *domain, device_t return (error); } +/* + * PCI memory address space is shared between memory-mapped devices (MMIO) and + * host memory (which may be remapped by an IOMMU). Device accesses to an + * address within a memory aperture in a PCIe root port will be treated as + * peer-to-peer and not forwarded to an IOMMU. To avoid this, reserve the + * address space of the root port's memory apertures in the address space used + * by the IOMMU for remapping. + */ +static int +dmar_reserve_pci_regions(struct dmar_domain *domain, device_t dev) +{ + struct iommu_domain *iodom; + device_t root; + uint32_t val; + uint64_t base, limit; + int error; + + iodom = DOM2IODOM(domain); + + root = pci_find_pcie_root_port(dev); + if (root == NULL) + return (0); + + /* Disable downstream memory */ + base = PCI_PPBMEMBASE(0, pci_read_config(root, PCIR_MEMBASE_1, 2)); + limit = PCI_PPBMEMLIMIT(0, pci_read_config(root, PCIR_MEMLIMIT_1, 2)); + error = iommu_gas_reserve_region_extend(iodom, base, limit + 1); + if (bootverbose || error != 0) + device_printf(dev, "DMAR reserve [%#jx-%#jx] (error %d)\n", + base, limit + 1, error); + if (error != 0) + return (error); + + /* Disable downstream prefetchable memory */ + val = pci_read_config(root, PCIR_PMBASEL_1, 2); + if (val != 0 || pci_read_config(root, PCIR_PMLIMITL_1, 2) != 0) { + if ((val & PCIM_BRPM_MASK) == PCIM_BRPM_64) { + base = PCI_PPBMEMBASE( + pci_read_config(root, PCIR_PMBASEH_1, 4), + val); + limit = PCI_PPBMEMLIMIT( + pci_read_config(root, PCIR_PMLIMITH_1, 4), + pci_read_config(root, PCIR_PMLIMITL_1, 2)); + } else { + base = PCI_PPBMEMBASE(0, val); + limit = PCI_PPBMEMLIMIT(0, + pci_read_config(root, PCIR_PMLIMITL_1, 2)); + } + error = iommu_gas_reserve_region_extend(iodom, base, + limit + 1); + if (bootverbose || error != 0) + device_printf(dev, "DMAR reserve [%#jx-%#jx] " + "(error %d)\n", base, limit + 1, error); + if (error != 0) + return (error); + } + + return (error); +} + static struct dmar_domain * dmar_domain_alloc(struct dmar_unit *dmar, bool id_mapped) { @@ -502,6 +562,8 @@ dmar_get_ctx_for_dev1(struct dmar_unit *dmar, device_t error = domain_init_rmrr(domain1, dev, bus, slot, func, dev_domain, dev_busno, dev_path, dev_path_len); + if (error == 0) + error = dmar_reserve_pci_regions(domain1, dev); if (error != 0) { dmar_domain_destroy(domain1); TD_PINNED_ASSERT;