From owner-svn-src-head@freebsd.org Mon Jul 22 16:01:22 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAEF5B4D26; Mon, 22 Jul 2019 16:01:22 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B093675658; Mon, 22 Jul 2019 16:01:22 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8A7C1F97E; Mon, 22 Jul 2019 16:01:22 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x6MG1Mto067720; Mon, 22 Jul 2019 16:01:22 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x6MG1KJb067708; Mon, 22 Jul 2019 16:01:20 GMT (envelope-from br@FreeBSD.org) Message-Id: <201907221601.x6MG1KJb067708@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Mon, 22 Jul 2019 16:01:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r350212 - in head/sys: conf dev/altera/msgdma dev/xdma mips/beri X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: in head/sys: conf dev/altera/msgdma dev/xdma mips/beri X-SVN-Commit-Revision: 350212 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: B093675658 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.93 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.93)[-0.934,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2019 16:01:22 -0000 Author: br Date: Mon Jul 22 16:01:20 2019 New Revision: 350212 URL: https://svnweb.freebsd.org/changeset/base/350212 Log: o Add support for BERI IOMMU device o Add an experimental IOMMU support to xDMA framework The BERI IOMMU device is the part of CHERI device-model project [1]. It translates memory addresses for various BERI peripherals modelled in software. It accepts FreeBSD/mips64 page directories format and manages BERI TLB. 1. https://github.com/CTSRD-CHERI/device-model Sponsored by: DARPA, AFRL Added: head/sys/dev/xdma/xdma_iommu.c (contents, props changed) head/sys/mips/beri/beri_iommu.c (contents, props changed) Modified: head/sys/conf/files head/sys/dev/altera/msgdma/msgdma.c head/sys/dev/xdma/xdma.c head/sys/dev/xdma/xdma.h head/sys/dev/xdma/xdma_if.m head/sys/dev/xdma/xdma_sg.c head/sys/mips/beri/files.beri Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/conf/files Mon Jul 22 16:01:20 2019 (r350212) @@ -3431,6 +3431,7 @@ dev/xdma/xdma_bank.c optional xdma dev/xdma/xdma_bio.c optional xdma dev/xdma/xdma_fdt_test.c optional xdma xdma_test fdt dev/xdma/xdma_if.m optional xdma +dev/xdma/xdma_iommu.c optional xdma dev/xdma/xdma_mbuf.c optional xdma dev/xdma/xdma_queue.c optional xdma dev/xdma/xdma_sg.c optional xdma Modified: head/sys/dev/altera/msgdma/msgdma.c ============================================================================== --- head/sys/dev/altera/msgdma/msgdma.c Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/dev/altera/msgdma/msgdma.c Mon Jul 22 16:01:20 2019 (r350212) @@ -414,7 +414,8 @@ msgdma_channel_alloc(device_t dev, struct xdma_channel if (chan->used == 0) { chan->xchan = xchan; xchan->chan = (void *)chan; - xchan->caps |= XCHAN_CAP_BUSDMA; + if ((xchan->caps & XCHAN_CAP_IOMMU) == 0) + xchan->caps |= XCHAN_CAP_BUSDMA; chan->index = i; chan->sc = sc; chan->used = 1; Modified: head/sys/dev/xdma/xdma.c ============================================================================== --- head/sys/dev/xdma/xdma.c Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/dev/xdma/xdma.c Mon Jul 22 16:01:20 2019 (r350212) @@ -70,6 +70,39 @@ static struct mtx xdma_mtx; #define FDT_REG_CELLS 4 +#ifdef FDT +static int +xdma_get_iommu_fdt(xdma_controller_t *xdma, xdma_channel_t *xchan) +{ + struct xdma_iommu *xio; + phandle_t node; + pcell_t prop; + size_t len; + + node = ofw_bus_get_node(xdma->dma_dev); + if (OF_getproplen(node, "xdma,iommu") <= 0) + return (0); + + len = OF_getencprop(node, "xdma,iommu", &prop, sizeof(prop)); + if (len != sizeof(prop)) { + device_printf(xdma->dev, + "%s: Can't get iommu device node\n", __func__); + return (0); + } + + xio = &xchan->xio; + xio->dev = OF_device_from_xref(prop); + if (xio->dev == NULL) { + device_printf(xdma->dev, + "%s: Can't get iommu device\n", __func__); + return (0); + } + + /* Found */ + return (1); +} +#endif + /* * Allocate virtual xDMA channel. */ @@ -81,6 +114,13 @@ xdma_channel_alloc(xdma_controller_t *xdma, uint32_t c xchan = malloc(sizeof(xdma_channel_t), M_XDMA, M_WAITOK | M_ZERO); xchan->xdma = xdma; + +#ifdef FDT + /* Check if this DMA controller supports IOMMU. */ + if (xdma_get_iommu_fdt(xdma, xchan)) + caps |= XCHAN_CAP_IOMMU | XCHAN_CAP_NOSEG; +#endif + xchan->caps = caps; XDMA_LOCK(); @@ -109,6 +149,9 @@ xdma_channel_alloc(xdma_controller_t *xdma, uint32_t c TAILQ_INIT(&xchan->queue_out); TAILQ_INIT(&xchan->processing); + if (xchan->caps & XCHAN_CAP_IOMMU) + xdma_iommu_init(&xchan->xio); + TAILQ_INSERT_TAIL(&xdma->channels, xchan, xchan_next); XDMA_UNLOCK(); @@ -139,6 +182,9 @@ xdma_channel_free(xdma_channel_t *xchan) if (xchan->flags & XCHAN_TYPE_SG) xdma_channel_free_sg(xchan); + if (xchan->caps & XCHAN_CAP_IOMMU) + xdma_iommu_release(&xchan->xio); + xdma_teardown_all_intr(xchan); mtx_destroy(&xchan->mtx_lock); @@ -306,7 +352,7 @@ xdma_ofw_md_data(xdma_controller_t *xdma, pcell_t *cel return (ret); } -static int +int xdma_handle_mem_node(vmem_t *vmem, phandle_t memory) { pcell_t reg[FDT_REG_CELLS * FDT_MEM_REGIONS]; Modified: head/sys/dev/xdma/xdma.h ============================================================================== --- head/sys/dev/xdma/xdma.h Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/dev/xdma/xdma.h Mon Jul 22 16:01:20 2019 (r350212) @@ -37,6 +37,14 @@ #include #include +#ifdef FDT +#include +#include +#endif + +#include +#include + enum xdma_direction { XDMA_MEM_TO_MEM, XDMA_MEM_TO_DEV, @@ -121,6 +129,12 @@ struct xdma_sglist { bool last; }; +struct xdma_iommu { + struct pmap p; + vmem_t *vmem; /* VA space */ + device_t dev; /* IOMMU device */ +}; + struct xdma_channel { xdma_controller_t *xdma; vmem_t *vmem; @@ -138,6 +152,7 @@ struct xdma_channel { #define XCHAN_CAP_BUSDMA (1 << 0) #define XCHAN_CAP_NOSEG (1 << 1) #define XCHAN_CAP_BOUNCE (1 << 2) +#define XCHAN_CAP_IOMMU (1 << 3) /* A real hardware driver channel. */ void *chan; @@ -171,6 +186,9 @@ struct xdma_channel { TAILQ_HEAD(, xdma_request) queue_in; TAILQ_HEAD(, xdma_request) queue_out; TAILQ_HEAD(, xdma_request) processing; + + /* iommu */ + struct xdma_iommu xio; }; typedef struct xdma_channel xdma_channel_t; @@ -216,6 +234,9 @@ xdma_controller_t *xdma_ofw_get(device_t dev, const ch int xdma_put(xdma_controller_t *xdma); vmem_t * xdma_get_memory(device_t dev); void xdma_put_memory(vmem_t *vmem); +#ifdef FDT +int xdma_handle_mem_node(vmem_t *vmem, phandle_t memory); +#endif /* xDMA channel ops */ xdma_channel_t * xdma_channel_alloc(xdma_controller_t *, uint32_t caps); @@ -270,5 +291,12 @@ void xchan_bank_init(xdma_channel_t *xchan); int xchan_bank_free(xdma_channel_t *xchan); struct xdma_request * xchan_bank_get(xdma_channel_t *xchan); int xchan_bank_put(xdma_channel_t *xchan, struct xdma_request *xr); + +/* IOMMU */ +void xdma_iommu_add_entry(xdma_channel_t *xchan, vm_offset_t *va, + vm_paddr_t pa, vm_size_t size, vm_prot_t prot); +void xdma_iommu_remove_entry(xdma_channel_t *xchan, vm_offset_t va); +int xdma_iommu_init(struct xdma_iommu *xio); +int xdma_iommu_release(struct xdma_iommu *xio); #endif /* !_DEV_XDMA_XDMA_H_ */ Modified: head/sys/dev/xdma/xdma_if.m ============================================================================== --- head/sys/dev/xdma/xdma_if.m Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/dev/xdma/xdma_if.m Mon Jul 22 16:01:20 2019 (r350212) @@ -1,5 +1,5 @@ #- -# Copyright (c) 2016-2018 Ruslan Bukin +# Copyright (c) 2016-2019 Ruslan Bukin # All rights reserved. # # This software was developed by SRI International and the University of @@ -112,4 +112,41 @@ METHOD int channel_control { device_t dev; struct xdma_channel *xchan; int cmd; +}; + +# IOMMU interface + +# +# pmap is initialized +# +METHOD int iommu_init { + device_t dev; + struct xdma_iommu *xio; +}; + +# +# pmap is released +# +METHOD int iommu_release { + device_t dev; + struct xdma_iommu *xio; +}; + +# +# Mapping entered +# +METHOD int iommu_enter { + device_t dev; + struct xdma_iommu *xio; + vm_offset_t va; + vm_offset_t pa; +}; + +# +# Mapping removed +# +METHOD int iommu_remove { + device_t dev; + struct xdma_iommu *xio; + vm_offset_t va; }; Added: head/sys/dev/xdma/xdma_iommu.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/xdma/xdma_iommu.c Mon Jul 22 16:01:20 2019 (r350212) @@ -0,0 +1,174 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2019 Ruslan Bukin + * + * This software was developed by SRI International and the University of + * Cambridge Computer Laboratory (Department of Computer Science and + * Technology) under DARPA contract HR0011-18-C-0016 ("ECATS"), as part of the + * DARPA SSITH research programme. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "opt_platform.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include + +#ifdef FDT +#include +#include +#include +#endif + +#include +#include "xdma_if.h" + +void +xdma_iommu_remove_entry(xdma_channel_t *xchan, vm_offset_t va) +{ + struct xdma_iommu *xio; + + xio = &xchan->xio; + + va &= ~(PAGE_SIZE - 1); + pmap_remove(&xio->p, va, va + PAGE_SIZE); + + XDMA_IOMMU_REMOVE(xio->dev, xio, va); + + vmem_free(xio->vmem, va, PAGE_SIZE); +} + +static void +xdma_iommu_enter(struct xdma_iommu *xio, vm_offset_t va, + vm_paddr_t pa, vm_size_t size, vm_prot_t prot) +{ + vm_page_t m; + pmap_t p; + + p = &xio->p; + + KASSERT((size & PAGE_MASK) == 0, + ("%s: device mapping not page-sized", __func__)); + + for (; size > 0; size -= PAGE_SIZE) { + m = PHYS_TO_VM_PAGE(pa); + pmap_enter(p, va, m, prot, prot | PMAP_ENTER_WIRED, 0); + + XDMA_IOMMU_ENTER(xio->dev, xio, va, pa); + + va += PAGE_SIZE; + pa += PAGE_SIZE; + } +} + +void +xdma_iommu_add_entry(xdma_channel_t *xchan, vm_offset_t *va, + vm_paddr_t pa, vm_size_t size, vm_prot_t prot) +{ + struct xdma_iommu *xio; + vm_offset_t addr; + + size = roundup2(size, PAGE_SIZE); + xio = &xchan->xio; + + if (vmem_alloc(xio->vmem, size, + M_FIRSTFIT | M_NOWAIT, &addr)) { + panic("Could not allocate virtual address.\n"); + } + + addr |= pa & (PAGE_SIZE - 1); + + if (va) + *va = addr; + + xdma_iommu_enter(xio, addr, pa, size, prot); +} + +int +xdma_iommu_init(struct xdma_iommu *xio) +{ +#ifdef FDT + phandle_t mem_node, node; + pcell_t mem_handle; +#endif + + pmap_pinit(&xio->p); + +#ifdef FDT + node = ofw_bus_get_node(xio->dev); + if (!OF_hasprop(node, "va-region")) + return (ENXIO); + + if (OF_getencprop(node, "va-region", (void *)&mem_handle, + sizeof(mem_handle)) <= 0) + return (ENXIO); +#endif + + xio->vmem = vmem_create("xDMA vmem", 0, 0, PAGE_SIZE, + PAGE_SIZE, M_FIRSTFIT | M_WAITOK); + if (xio->vmem == NULL) + return (ENXIO); + +#ifdef FDT + mem_node = OF_node_from_xref(mem_handle); + if (xdma_handle_mem_node(xio->vmem, mem_node) != 0) { + vmem_destroy(xio->vmem); + return (ENXIO); + } +#endif + + XDMA_IOMMU_INIT(xio->dev, xio); + + return (0); +} + +int +xdma_iommu_release(struct xdma_iommu *xio) +{ + + pmap_release(&xio->p); + + vmem_destroy(xio->vmem); + + XDMA_IOMMU_RELEASE(xio->dev, xio); + + return (0); +} Modified: head/sys/dev/xdma/xdma_sg.c ============================================================================== --- head/sys/dev/xdma/xdma_sg.c Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/dev/xdma/xdma_sg.c Mon Jul 22 16:01:20 2019 (r350212) @@ -327,6 +327,7 @@ xchan_seg_done(xdma_channel_t *xchan, struct xdma_request *xr; xdma_controller_t *xdma; struct xchan_buf *b; + bus_addr_t addr; xdma = xchan->xdma; @@ -352,6 +353,12 @@ xchan_seg_done(xdma_channel_t *xchan, xr->direction == XDMA_DEV_TO_MEM) m_copyback(xr->m, 0, st->transferred, (void *)xr->buf.vaddr); + } else if (xchan->caps & XCHAN_CAP_IOMMU) { + if (xr->direction == XDMA_MEM_TO_DEV) + addr = xr->src_addr; + else + addr = xr->dst_addr; + xdma_iommu_remove_entry(xchan, addr); } xr->status.error = st->error; xr->status.transferred = st->transferred; @@ -484,11 +491,17 @@ _xdma_load_data(xdma_channel_t *xchan, struct xdma_req xdma_controller_t *xdma; struct mbuf *m; uint32_t nsegs; + vm_offset_t va, addr; + bus_addr_t pa; + vm_prot_t prot; xdma = xchan->xdma; m = xr->m; + KASSERT(xchan->caps & XCHAN_CAP_NOSEG, + ("Handling segmented data is not implemented here.")); + nsegs = 1; switch (xr->req_type) { @@ -498,6 +511,27 @@ _xdma_load_data(xdma_channel_t *xchan, struct xdma_req m_copydata(m, 0, m->m_pkthdr.len, (void *)xr->buf.vaddr); seg[0].ds_addr = (bus_addr_t)xr->buf.paddr; + } else if (xchan->caps & XCHAN_CAP_IOMMU) { + addr = mtod(m, bus_addr_t); + pa = vtophys(addr); + + if (xr->direction == XDMA_MEM_TO_DEV) + prot = VM_PROT_READ; + else + prot = VM_PROT_WRITE; + + xdma_iommu_add_entry(xchan, &va, + pa, m->m_pkthdr.len, prot); + + /* + * Save VA so we can unload data later + * after completion of this transfer. + */ + if (xr->direction == XDMA_MEM_TO_DEV) + xr->src_addr = va; + else + xr->dst_addr = va; + seg[0].ds_addr = va; } else seg[0].ds_addr = mtod(m, bus_addr_t); seg[0].ds_len = m->m_pkthdr.len; Added: head/sys/mips/beri/beri_iommu.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/mips/beri/beri_iommu.c Mon Jul 22 16:01:20 2019 (r350212) @@ -0,0 +1,237 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2019 Ruslan Bukin + * + * This software was developed by SRI International and the University of + * Cambridge Computer Laboratory (Department of Computer Science and + * Technology) under DARPA contract HR0011-18-C-0016 ("ECATS"), as part of the + * DARPA SSITH research programme. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include + +#include + +#include "xdma_if.h" + +#define IOMMU_INVALIDATE 0x00 +#define IOMMU_SET_BASE 0x08 + +struct beri_iommu_softc { + struct resource *res[1]; + device_t dev; + bus_space_tag_t bst_data; + bus_space_handle_t bsh_data; + uint32_t offs; +}; + +static struct resource_spec beri_iommu_spec[] = { + { SYS_RES_MEMORY, 0, RF_ACTIVE }, + { -1, 0 } +}; + +static void +beri_iommu_invalidate(struct beri_iommu_softc *sc, vm_offset_t addr) +{ + + bus_write_8(sc->res[0], IOMMU_INVALIDATE, htole64(addr)); +} + +static void +beri_iommu_set_base(struct beri_iommu_softc *sc, vm_offset_t addr) +{ + + bus_write_8(sc->res[0], IOMMU_SET_BASE, htole64(addr)); +} + +static int +beri_iommu_release(device_t dev, struct xdma_iommu *xio) +{ + struct beri_iommu_softc *sc; + + sc = device_get_softc(dev); + + beri_iommu_set_base(sc, 0); + + return (0); +} + +static int +beri_iommu_init(device_t dev, struct xdma_iommu *xio) +{ + struct beri_iommu_softc *sc; + + sc = device_get_softc(dev); + + beri_iommu_set_base(sc, (uintptr_t)xio->p.pm_segtab); + + return (0); +} + +static int +beri_iommu_remove(device_t dev, struct xdma_iommu *xio, vm_offset_t va) +{ + struct beri_iommu_softc *sc; + + sc = device_get_softc(dev); + + beri_iommu_invalidate(sc, va); + + return (0); +} + +static int +beri_iommu_enter(device_t dev, struct xdma_iommu *xio, vm_offset_t va, + vm_paddr_t pa) +{ + struct beri_iommu_softc *sc; + pt_entry_t opte, npte; + pt_entry_t *pte; + pmap_t p; + + sc = device_get_softc(dev); + p = &xio->p; + + pte = pmap_pte(p, va); + if (pte == NULL) + panic("pte is NULL\n"); + + /* Make pte uncacheable. */ + opte = *pte; + npte = opte & ~PTE_C_MASK; + npte |= PTE_C(VM_MEMATTR_UNCACHEABLE); + *pte = npte; + + /* Write back, invalidate pte. */ + mips_dcache_wbinv_range((vm_offset_t)pte, sizeof(vm_offset_t)); + + /* Invalidate the entry. */ + if (pte_test(&opte, PTE_V) && opte != npte) + beri_iommu_invalidate(sc, va); + + return (0); +} + +static int +beri_iommu_probe(device_t dev) +{ + + if (!ofw_bus_status_okay(dev)) + return (ENXIO); + + if (!ofw_bus_is_compatible(dev, "beri,iommu")) + return (ENXIO); + + device_set_desc(dev, "BERI IOMMU"); + + return (BUS_PROBE_DEFAULT); +} + +static int +beri_iommu_attach(device_t dev) +{ + struct beri_iommu_softc *sc; + phandle_t xref, node; + + sc = device_get_softc(dev); + sc->dev = dev; + + if (bus_alloc_resources(dev, beri_iommu_spec, sc->res)) { + device_printf(dev, "could not allocate resources\n"); + return (ENXIO); + } + + /* Memory interface */ + sc->bst_data = rman_get_bustag(sc->res[0]); + sc->bsh_data = rman_get_bushandle(sc->res[0]); + + node = ofw_bus_get_node(dev); + xref = OF_xref_from_node(node); + OF_device_register_xref(xref, dev); + + return (0); +} + +static int +beri_iommu_detach(device_t dev) +{ + struct beri_iommu_softc *sc; + + sc = device_get_softc(dev); + + bus_release_resources(dev, beri_iommu_spec, sc->res); + + return (0); +} + +static device_method_t beri_iommu_methods[] = { + + /* xDMA IOMMU interface */ + DEVMETHOD(xdma_iommu_init, beri_iommu_init), + DEVMETHOD(xdma_iommu_release, beri_iommu_release), + DEVMETHOD(xdma_iommu_enter, beri_iommu_enter), + DEVMETHOD(xdma_iommu_remove, beri_iommu_remove), + + /* Device interface */ + DEVMETHOD(device_probe, beri_iommu_probe), + DEVMETHOD(device_attach, beri_iommu_attach), + DEVMETHOD(device_detach, beri_iommu_detach), + + { 0, 0 } +}; + +static driver_t beri_iommu_driver = { + "beri_iommu", + beri_iommu_methods, + sizeof(struct beri_iommu_softc), +}; + +static devclass_t beri_iommu_devclass; + +DRIVER_MODULE(beri_iommu, simplebus, beri_iommu_driver, + beri_iommu_devclass, 0, 0); Modified: head/sys/mips/beri/files.beri ============================================================================== --- head/sys/mips/beri/files.beri Mon Jul 22 10:37:56 2019 (r350211) +++ head/sys/mips/beri/files.beri Mon Jul 22 16:01:20 2019 (r350212) @@ -20,6 +20,7 @@ dev/terasic/mtl/terasic_mtl_reg.c optional terasic_mtl dev/terasic/mtl/terasic_mtl_syscons.c optional terasic_mtl sc dev/terasic/mtl/terasic_mtl_text.c optional terasic_mtl dev/terasic/mtl/terasic_mtl_vt.c optional terasic_mtl vt +mips/beri/beri_iommu.c optional xdma mips/beri/beri_machdep.c standard mips/beri/beri_mp.c optional smp mips/beri/beri_pic.c optional fdt