From owner-svn-src-all@freebsd.org Tue Aug 27 04:13:25 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42B56C8B93; Tue, 27 Aug 2019 04:13:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Hb8P1f7qz4LCf; Tue, 27 Aug 2019 04:13:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 00B82191EC; Tue, 27 Aug 2019 04:13:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x7R4DOsN006897; Tue, 27 Aug 2019 04:13:24 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x7R4DMwX006886; Tue, 27 Aug 2019 04:13:22 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201908270413.x7R4DMwX006886@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 27 Aug 2019 04:13:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r351536 - in stable/12/sys: amd64/conf conf dev/ntb dev/ntb/ntb_hw i386/conf modules/ntb modules/ntb/ntb_hw_amd X-SVN-Group: stable-12 X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in stable/12/sys: amd64/conf conf dev/ntb dev/ntb/ntb_hw i386/conf modules/ntb modules/ntb/ntb_hw_amd X-SVN-Commit-Revision: 351536 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Aug 2019 04:13:25 -0000 Author: mav Date: Tue Aug 27 04:13:22 2019 New Revision: 351536 URL: https://svnweb.freebsd.org/changeset/base/351536 Log: MFC r349594, r349595, r349601: Add driver for NTB in AMD SoC. This patch is the driver for NTB hardware in AMD SoCs (ported from Linux) and enables the NTB infrastructure like Doorbells, Scratchpads and Memory window in AMD SoC. This driver has been validated using ntb_transport and if_ntb driver already available in FreeBSD. Submitted by: Rajesh Kumar Relnotes: yes Added: stable/12/sys/dev/ntb/ntb_hw/ntb_hw_amd.c - copied, changed from r349594, head/sys/dev/ntb/ntb_hw/ntb_hw_amd.c stable/12/sys/dev/ntb/ntb_hw/ntb_hw_amd.h - copied unchanged from r349594, head/sys/dev/ntb/ntb_hw/ntb_hw_amd.h stable/12/sys/modules/ntb/ntb_hw_amd/ - copied from r349594, head/sys/modules/ntb/ntb_hw_amd/ Modified: stable/12/sys/amd64/conf/NOTES stable/12/sys/conf/files.amd64 stable/12/sys/conf/files.i386 stable/12/sys/dev/ntb/ntb.c stable/12/sys/dev/ntb/ntb.h stable/12/sys/dev/ntb/ntb_if.m stable/12/sys/dev/ntb/ntb_transport.c stable/12/sys/i386/conf/NOTES stable/12/sys/modules/ntb/Makefile Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/amd64/conf/NOTES ============================================================================== --- stable/12/sys/amd64/conf/NOTES Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/amd64/conf/NOTES Tue Aug 27 04:13:22 2019 (r351536) @@ -398,6 +398,7 @@ device wpifw device if_ntb # Virtual NTB network interface device ntb_transport # NTB packet transport driver device ntb # NTB hardware interface +device ntb_hw_amd # AMD NTB hardware driver device ntb_hw_intel # Intel NTB hardware driver device ntb_hw_plx # PLX NTB hardware driver Modified: stable/12/sys/conf/files.amd64 ============================================================================== --- stable/12/sys/conf/files.amd64 Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/conf/files.amd64 Tue Aug 27 04:13:22 2019 (r351536) @@ -346,8 +346,9 @@ dev/nctgpio/nctgpio.c optional nctgpio dev/nfe/if_nfe.c optional nfe pci dev/ntb/if_ntb/if_ntb.c optional if_ntb dev/ntb/ntb_transport.c optional ntb_transport | if_ntb -dev/ntb/ntb.c optional ntb | ntb_transport | if_ntb | ntb_hw_intel | ntb_hw_plx | ntb_hw -dev/ntb/ntb_if.m optional ntb | ntb_transport | if_ntb | ntb_hw_intel | ntb_hw_plx | ntb_hw +dev/ntb/ntb.c optional ntb | ntb_transport | if_ntb | ntb_hw_amd | ntb_hw_intel | ntb_hw_plx | ntb_hw +dev/ntb/ntb_if.m optional ntb | ntb_transport | if_ntb | ntb_hw_amd | ntb_hw_intel | ntb_hw_plx | ntb_hw +dev/ntb/ntb_hw/ntb_hw_amd.c optional ntb_hw_amd | ntb_hw dev/ntb/ntb_hw/ntb_hw_intel.c optional ntb_hw_intel | ntb_hw dev/ntb/ntb_hw/ntb_hw_plx.c optional ntb_hw_plx | ntb_hw dev/nvd/nvd.c optional nvd nvme Modified: stable/12/sys/conf/files.i386 ============================================================================== --- stable/12/sys/conf/files.i386 Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/conf/files.i386 Tue Aug 27 04:13:22 2019 (r351536) @@ -284,8 +284,9 @@ dev/nctgpio/nctgpio.c optional nctgpio dev/nfe/if_nfe.c optional nfe pci dev/ntb/if_ntb/if_ntb.c optional if_ntb dev/ntb/ntb_transport.c optional ntb_transport | if_ntb -dev/ntb/ntb.c optional ntb | ntb_transport | if_ntb | ntb_hw_intel | ntb_hw_plx | ntb_hw -dev/ntb/ntb_if.m optional ntb | ntb_transport | if_ntb | ntb_hw_intel | ntb_hw_plx | ntb_hw +dev/ntb/ntb.c optional ntb | ntb_transport | if_ntb | ntb_hw_amd | ntb_hw_intel | ntb_hw_plx | ntb_hw +dev/ntb/ntb_if.m optional ntb | ntb_transport | if_ntb | ntb_hw_amd | ntb_hw_intel | ntb_hw_plx | ntb_hw +dev/ntb/ntb_hw/ntb_hw_amd.c optional ntb_hw_amd | ntb_hw dev/ntb/ntb_hw/ntb_hw_intel.c optional ntb_hw_intel | ntb_hw dev/ntb/ntb_hw/ntb_hw_plx.c optional ntb_hw_plx | ntb_hw dev/nvd/nvd.c optional nvd nvme Modified: stable/12/sys/dev/ntb/ntb.c ============================================================================== --- stable/12/sys/dev/ntb/ntb.c Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/dev/ntb/ntb.c Tue Aug 27 04:13:22 2019 (r351536) @@ -243,6 +243,30 @@ ntb_db_event(device_t dev, uint32_t vec) } } +int +ntb_port_number(device_t ntb) +{ + return (NTB_PORT_NUMBER(device_get_parent(ntb))); +} + +int +ntb_peer_port_count(device_t ntb) +{ + return (NTB_PEER_PORT_COUNT(device_get_parent(ntb))); +} + +int +ntb_peer_port_number(device_t ntb, int pidx) +{ + return (NTB_PEER_PORT_NUMBER(device_get_parent(ntb), pidx)); +} + +int +ntb_peer_port_idx(device_t ntb, int port) +{ + return (NTB_PEER_PORT_IDX(device_get_parent(ntb), port)); +} + bool ntb_link_is_up(device_t ntb, enum ntb_speed *speed, enum ntb_width *width) { Modified: stable/12/sys/dev/ntb/ntb.h ============================================================================== --- stable/12/sys/dev/ntb/ntb.h Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/dev/ntb/ntb.h Tue Aug 27 04:13:22 2019 (r351536) @@ -64,6 +64,51 @@ void ntb_link_event(device_t ntb); */ void ntb_db_event(device_t ntb, uint32_t vec); +/** + * ntb_port_number() - get the local port number + * @ntb: NTB device context. + * + * Hardware driver returns local port number in compliance with topology. + * + * Return: the local port number + */ +int ntb_port_number(device_t ntb); + +/** + * ntb_port_count() - get the number of peer device ports + * @ntb: NTB device context. + * + * By default hardware driver supports just one peer device. + * + * Return: the number of peer ports + */ +int ntb_peer_port_count(device_t ntb); + +/** + * ntb_peer_port_number() - get the peer port by given index + * @ntb: NTB device context. + * @idx: Peer port index (should be zero for now). + * + * By default hardware driver supports just one peer device, so this method + * shall return the corresponding value. + * + * Return: the peer device port or an error number + */ +int ntb_peer_port_number(device_t ntb, int pidx); + +/* + * ntb_peer_port_idx() - get the peer device port index by given port + * number + * @ntb: NTB device context. + * @port: Peer port number + * + * By default hardware driver supports just one peer device, so given a + * valid peer port number, the return value shall be zero. + * + * Return: the peer port index or an error number + */ +int ntb_peer_port_idx(device_t ntb, int port); + /* * ntb_link_is_up() - get the current ntb link state * @ntb: NTB device context Copied and modified: stable/12/sys/dev/ntb/ntb_hw/ntb_hw_amd.c (from r349594, head/sys/dev/ntb/ntb_hw/ntb_hw_amd.c) ============================================================================== --- head/sys/dev/ntb/ntb_hw/ntb_hw_amd.c Tue Jul 2 05:25:18 2019 (r349594, copy source) +++ stable/12/sys/dev/ntb/ntb_hw/ntb_hw_amd.c Tue Aug 27 04:13:22 2019 (r351536) @@ -59,6 +59,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -93,6 +94,26 @@ SYSCTL_UINT(_hw_ntb, OID_AUTO, debug_level, CTLFLAG_RW device_printf(ntb->device, __VA_ARGS__); \ } while (0) +#ifdef __i386__ +static __inline uint64_t +bus_space_read_8(bus_space_tag_t tag, bus_space_handle_t handle, + bus_size_t offset) +{ + + return (bus_space_read_4(tag, handle, offset) | + ((uint64_t)bus_space_read_4(tag, handle, offset + 4)) << 32); +} + +static __inline void +bus_space_write_8(bus_space_tag_t tag, bus_space_handle_t handle, + bus_size_t offset, uint64_t val) +{ + + bus_space_write_4(tag, handle, offset, val); + bus_space_write_4(tag, handle, offset + 4, val >> 32); +} +#endif + /* * AMD NTB INTERFACE ROUTINES */ @@ -316,11 +337,6 @@ amd_ntb_mw_get_range(device_t dev, unsigned mw_idx, vm *plimit = BUS_SPACE_MAXADDR_32BIT; } - amd_ntb_printf(1, "%s: mw %d padd %p vadd %p psize 0x%lx " - "align 0x%lx asize 0x%lx alimit %p\n", __func__, mw_idx, - (void *)*base, (void *)*vbase, (uint64_t)*size, (uint64_t)*align, - (uint64_t)*align_size, (void *)*plimit); - return (0); } @@ -353,14 +369,14 @@ amd_ntb_mw_set_trans(device_t dev, unsigned mw_idx, bu * set and verify setting the translation address */ amd_ntb_peer_reg_write(8, bar_info->xlat_off, (uint64_t)addr); - amd_ntb_printf(0, "%s: mw %d xlat_off 0x%x cur_val 0x%lx addr %p\n", + amd_ntb_printf(0, "%s: mw %d xlat_off 0x%x cur_val 0x%jx addr %p\n", __func__, mw_idx, bar_info->xlat_off, amd_ntb_peer_reg_read(8, bar_info->xlat_off), (void *)addr); /* set and verify setting the limit */ if (mw_idx != 0) { amd_ntb_reg_write(8, bar_info->limit_off, (uint64_t)size); - amd_ntb_printf(1, "%s: limit_off 0x%x cur_val 0x%lx limit 0x%x\n", + amd_ntb_printf(1, "%s: limit_off 0x%x cur_val 0x%jx limit 0x%x\n", __func__, bar_info->limit_off, amd_ntb_peer_reg_read(8, bar_info->limit_off), (uint32_t)size); } else { @@ -470,7 +486,7 @@ amd_ntb_db_read(device_t dev) dbstat_off = (uint64_t)amd_ntb_reg_read(2, AMD_DBSTAT_OFFSET); - amd_ntb_printf(1, "%s: dbstat_off 0x%lx\n", __func__, dbstat_off); + amd_ntb_printf(1, "%s: dbstat_off 0x%jx\n", __func__, dbstat_off); return (dbstat_off); } @@ -480,7 +496,7 @@ amd_ntb_db_clear(device_t dev, uint64_t db_bits) { struct amd_ntb_softc *ntb = device_get_softc(dev); - amd_ntb_printf(1, "%s: db_bits 0x%lx\n", __func__, db_bits); + amd_ntb_printf(1, "%s: db_bits 0x%jx\n", __func__, db_bits); amd_ntb_reg_write(2, AMD_DBSTAT_OFFSET, (uint16_t)db_bits); } @@ -490,7 +506,7 @@ amd_ntb_db_set_mask(device_t dev, uint64_t db_bits) struct amd_ntb_softc *ntb = device_get_softc(dev); DB_MASK_LOCK(ntb); - amd_ntb_printf(1, "%s: db_mask 0x%x db_bits 0x%lx\n", + amd_ntb_printf(1, "%s: db_mask 0x%x db_bits 0x%jx\n", __func__, ntb->db_mask, db_bits); ntb->db_mask |= db_bits; @@ -504,7 +520,7 @@ amd_ntb_db_clear_mask(device_t dev, uint64_t db_bits) struct amd_ntb_softc *ntb = device_get_softc(dev); DB_MASK_LOCK(ntb); - amd_ntb_printf(1, "%s: db_mask 0x%x db_bits 0x%lx\n", + amd_ntb_printf(1, "%s: db_mask 0x%x db_bits 0x%jx\n", __func__, ntb->db_mask, db_bits); ntb->db_mask &= ~db_bits; @@ -517,7 +533,7 @@ amd_ntb_peer_db_set(device_t dev, uint64_t db_bits) { struct amd_ntb_softc *ntb = device_get_softc(dev); - amd_ntb_printf(1, "%s: db_bits 0x%lx\n", __func__, db_bits); + amd_ntb_printf(1, "%s: db_bits 0x%jx\n", __func__, db_bits); amd_ntb_reg_write(2, AMD_DBREQ_OFFSET, (uint16_t)db_bits); } @@ -660,17 +676,17 @@ amd_ntb_hw_info_handler(SYSCTL_HANDLER_ARGS) sbuf_printf(sb, "AMD Doorbell: 0x%x\n", amd_ntb_reg_read(4, AMD_DBSTAT_OFFSET)); sbuf_printf(sb, "AMD NTB Incoming XLAT: \n"); - sbuf_printf(sb, "AMD XLAT1: 0x%lx\n", + sbuf_printf(sb, "AMD XLAT1: 0x%jx\n", amd_ntb_peer_reg_read(8, AMD_BAR1XLAT_OFFSET)); - sbuf_printf(sb, "AMD XLAT23: 0x%lx\n", + sbuf_printf(sb, "AMD XLAT23: 0x%jx\n", amd_ntb_peer_reg_read(8, AMD_BAR23XLAT_OFFSET)); - sbuf_printf(sb, "AMD XLAT45: 0x%lx\n", + sbuf_printf(sb, "AMD XLAT45: 0x%jx\n", amd_ntb_peer_reg_read(8, AMD_BAR45XLAT_OFFSET)); sbuf_printf(sb, "AMD LMT1: 0x%x\n", amd_ntb_reg_read(4, AMD_BAR1LMT_OFFSET)); - sbuf_printf(sb, "AMD LMT23: 0x%lx\n", + sbuf_printf(sb, "AMD LMT23: 0x%jx\n", amd_ntb_reg_read(8, AMD_BAR23LMT_OFFSET)); - sbuf_printf(sb, "AMD LMT45: 0x%lx\n", + sbuf_printf(sb, "AMD LMT45: 0x%jx\n", amd_ntb_reg_read(8, AMD_BAR45LMT_OFFSET)); rc = sbuf_finish(sb); Copied: stable/12/sys/dev/ntb/ntb_hw/ntb_hw_amd.h (from r349594, head/sys/dev/ntb/ntb_hw/ntb_hw_amd.h) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/12/sys/dev/ntb/ntb_hw/ntb_hw_amd.h Tue Aug 27 04:13:22 2019 (r351536, copy of r349594, head/sys/dev/ntb/ntb_hw/ntb_hw_amd.h) @@ -0,0 +1,259 @@ +/*- + * This file is provided under a dual BSD/GPLv2 license. When using or + * redistributing this file, you may do so under either license. + * + * GPL LICENSE SUMMARY + * + * Copyright (C) 2019 Advanced Micro Devices, Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of version 2 of the GNU General Public License as + * published by the Free Software Foundation. + * + * BSD LICENSE + * + * Copyright (C) 2019 Advanced Micro Devices, Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copy + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the distribution. + * 3. Neither the name of AMD corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * Contact Information : + * Rajesh Kumar + * + * $FreeBSD$ + */ + +#ifndef NTB_HW_AMD_H +#define NTB_HW_AMD_H + +#define NTB_HW_AMD_VENDOR_ID 0x1022 +#define NTB_HW_AMD_DEVICE_ID 0x145B + +#define NTB_DEF_PEER_CNT 1 +#define NTB_DEF_PEER_IDX 0 + +#define BIT(n) (1 << n) +#define AMD_LINK_HB_TIMEOUT (1 * hz) + +#define NTB_LIN_STA_ACTIVE_BIT 0x00000002 +#define NTB_LNK_STA_SPEED_MASK 0x000F0000 +#define NTB_LNK_STA_WIDTH_MASK 0x03F00000 +#define NTB_LNK_STA_ACTIVE(x) (!!((x) & NTB_LIN_STA_ACTIVE_BIT)) +#define NTB_LNK_STA_SPEED(x) (((x) & NTB_LNK_STA_SPEED_MASK) >> 16) +#define NTB_LNK_STA_WIDTH(x) (((x) & NTB_LNK_STA_WIDTH_MASK) >> 20) + +#define amd_ntb_bar_read(SIZE, bar, offset) \ + bus_space_read_ ## SIZE (ntb->bar_info[(bar)].pci_bus_tag, \ + ntb->bar_info[(bar)].pci_bus_handle, (offset)) +#define amd_ntb_bar_write(SIZE, bar, offset, val) \ + bus_space_write_ ## SIZE (ntb->bar_info[(bar)].pci_bus_tag, \ + ntb->bar_info[(bar)].pci_bus_handle, (offset), (val)) +#define amd_ntb_reg_read(SIZE, offset) \ + amd_ntb_bar_read(SIZE, NTB_CONFIG_BAR, offset) +#define amd_ntb_reg_write(SIZE, offset, val) \ + amd_ntb_bar_write(SIZE, NTB_CONFIG_BAR, offset, val) +#define amd_ntb_peer_reg_read(SIZE, offset) \ + amd_ntb_bar_read(SIZE, NTB_CONFIG_BAR, offset + AMD_PEER_OFFSET) +#define amd_ntb_peer_reg_write(SIZE, offset, val) \ + amd_ntb_bar_write(SIZE, NTB_CONFIG_BAR, offset + AMD_PEER_OFFSET, val) + +#define DB_MASK_LOCK(sc) mtx_lock_spin(&(sc)->db_mask_lock) +#define DB_MASK_UNLOCK(sc) mtx_unlock_spin(&(sc)->db_mask_lock) +#define DB_MASK_ASSERT(sc, f) mtx_assert(&(sc)->db_mask_lock, (f)) + +/* amd_ntb_conn_type are hardware numbers, cannot change. */ +enum amd_ntb_conn_type { + NTB_CONN_NONE = -1, + NTB_CONN_PRI, + NTB_CONN_SEC, +}; + +enum ntb_default_port { + NTB_PORT_PRI_USD, + NTB_PORT_SEC_DSD +}; + +enum amd_ntb_bar { + NTB_CONFIG_BAR = 0, + NTB_BAR_1, + NTB_BAR_2, + NTB_BAR_3, + NTB_MAX_BARS +}; + +struct amd_ntb_hw_info { + uint32_t device_id; + const char *desc; +}; + +struct amd_ntb_pci_bar_info { + bus_space_tag_t pci_bus_tag; + bus_space_handle_t pci_bus_handle; + struct resource *pci_resource; + vm_paddr_t pbase; + caddr_t vbase; + vm_size_t size; + vm_memattr_t map_mode; + int pci_resource_id; + + /* Configuration register offsets */ + uint32_t xlat_off; + uint32_t limit_off; +}; + +struct amd_ntb_int_info { + struct resource *res; + void *tag; + int rid; +}; + +struct amd_ntb_vec { + struct amd_ntb_softc *ntb; + uint32_t num; + unsigned masked; +}; + +enum { + /* AMD NTB Capability */ + AMD_MW_CNT = 3, + AMD_DB_CNT = 16, + AMD_MSIX_VECTOR_CNT = 24, + AMD_SPADS_CNT = 16, + + /* AMD NTB Link Status Offset */ + AMD_LINK_STATUS_OFFSET = 0x68, + + /* AMD NTB register offset */ + AMD_CNTL_OFFSET = 0x200, + + /* NTB control register bits */ + PMM_REG_CTL = BIT(21), + SMM_REG_CTL = BIT(20), + SMM_REG_ACC_PATH = BIT(18), + PMM_REG_ACC_PATH = BIT(17), + NTB_CLK_EN = BIT(16), + + AMD_STA_OFFSET = 0x204, + AMD_PGSLV_OFFSET = 0x208, + AMD_SPAD_MUX_OFFSET = 0x20C, + AMD_SPAD_OFFSET = 0x210, + AMD_RSMU_HCID = 0x250, + AMD_RSMU_SIID = 0x254, + AMD_PSION_OFFSET = 0x300, + AMD_SSION_OFFSET = 0x330, + AMD_MMINDEX_OFFSET = 0x400, + AMD_MMDATA_OFFSET = 0x404, + AMD_SIDEINFO_OFFSET = 0x408, + + AMD_SIDE_MASK = BIT(0), + AMD_SIDE_READY = BIT(1), + + /* limit register */ + AMD_ROMBARLMT_OFFSET = 0x410, + AMD_BAR1LMT_OFFSET = 0x414, + AMD_BAR23LMT_OFFSET = 0x418, + AMD_BAR45LMT_OFFSET = 0x420, + + /* xlat address */ + AMD_ROMBARXLAT_OFFSET = 0x428, + AMD_BAR1XLAT_OFFSET = 0x430, + AMD_BAR23XLAT_OFFSET = 0x438, + AMD_BAR45XLAT_OFFSET = 0x440, + + /* doorbell and interrupt */ + AMD_DBFM_OFFSET = 0x450, + AMD_DBREQ_OFFSET = 0x454, + AMD_MIRRDBSTAT_OFFSET = 0x458, + AMD_DBMASK_OFFSET = 0x45C, + AMD_DBSTAT_OFFSET = 0x460, + AMD_INTMASK_OFFSET = 0x470, + AMD_INTSTAT_OFFSET = 0x474, + + /* event type */ + AMD_PEER_FLUSH_EVENT = BIT(0), + AMD_PEER_RESET_EVENT = BIT(1), + AMD_PEER_D3_EVENT = BIT(2), + AMD_PEER_PMETO_EVENT = BIT(3), + AMD_PEER_D0_EVENT = BIT(4), + AMD_LINK_UP_EVENT = BIT(5), + AMD_LINK_DOWN_EVENT = BIT(6), + AMD_EVENT_INTMASK = (AMD_PEER_FLUSH_EVENT | + AMD_PEER_RESET_EVENT | AMD_PEER_D3_EVENT | + AMD_PEER_PMETO_EVENT | AMD_PEER_D0_EVENT | + AMD_LINK_UP_EVENT | AMD_LINK_DOWN_EVENT), + + AMD_PMESTAT_OFFSET = 0x480, + AMD_PMSGTRIG_OFFSET = 0x490, + AMD_LTRLATENCY_OFFSET = 0x494, + AMD_FLUSHTRIG_OFFSET = 0x498, + + /* SMU register*/ + AMD_SMUACK_OFFSET = 0x4A0, + AMD_SINRST_OFFSET = 0x4A4, + AMD_RSPNUM_OFFSET = 0x4A8, + AMD_SMU_SPADMUTEX = 0x4B0, + AMD_SMU_SPADOFFSET = 0x4B4, + + AMD_PEER_OFFSET = 0x400, +}; + +struct amd_ntb_softc { + /* ntb.c context. Do not move! Must go first! */ + void *ntb_store; + + device_t device; + enum amd_ntb_conn_type conn_type; + + struct amd_ntb_pci_bar_info bar_info[NTB_MAX_BARS]; + struct amd_ntb_int_info int_info[AMD_MSIX_VECTOR_CNT]; + struct amd_ntb_vec *msix_vec; + uint16_t allocated_interrupts; + + struct callout hb_timer; + + uint8_t mw_count; + uint8_t spad_count; + uint8_t db_count; + uint8_t msix_vec_count; + + struct mtx db_mask_lock; + + volatile uint32_t ntb_ctl; + volatile uint32_t lnk_sta; + volatile uint32_t peer_sta; + volatile uint32_t cntl_sta; + + uint16_t db_valid_mask; + uint16_t db_mask; + uint32_t int_mask; + + unsigned int self_spad; + unsigned int peer_spad; +}; + +static void amd_init_side_info(struct amd_ntb_softc *ntb); +static void amd_deinit_side_info(struct amd_ntb_softc *ntb); +static int amd_ntb_detach(device_t device); + +#endif Modified: stable/12/sys/dev/ntb/ntb_if.m ============================================================================== --- stable/12/sys/dev/ntb/ntb_if.m Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/dev/ntb/ntb_if.m Tue Aug 27 04:13:22 2019 (r351536) @@ -61,6 +61,24 @@ HEADER { }; }; +METHOD int port_number { + device_t ntb; +}; + +METHOD int peer_port_count { + device_t ntb; +}; + +METHOD int peer_port_number { + device_t ntb; + int pidx; +}; + +METHOD int peer_port_idx { + device_t ntb; + int port; +}; + METHOD bool link_is_up { device_t ntb; enum ntb_speed *speed; Modified: stable/12/sys/dev/ntb/ntb_transport.c ============================================================================== --- stable/12/sys/dev/ntb/ntb_transport.c Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/dev/ntb/ntb_transport.c Tue Aug 27 04:13:22 2019 (r351536) @@ -179,7 +179,6 @@ struct ntb_transport_mw { bus_addr_t addr_limit; /* Tx buff is off vbase / phys_addr */ caddr_t vbase; - size_t xlat_size; size_t buff_size; /* Rx buff is off virt_addr / dma_addr */ bus_dma_tag_t dma_tag; @@ -376,7 +375,6 @@ ntb_transport_attach(device_t dev) goto err; mw->buff_size = 0; - mw->xlat_size = 0; mw->virt_addr = NULL; mw->dma_addr = 0; @@ -462,6 +460,13 @@ ntb_transport_attach(device_t dev) ntb_link_enable(dev, NTB_SPEED_AUTO, NTB_WIDTH_AUTO); + for (i = 0; i < nt->mw_count; i++) { + mw = &nt->mw_vec[i]; + rc = ntb_mw_set_trans(nt->dev, i, mw->dma_addr, mw->buff_size); + if (rc != 0) + ntb_printf(0, "load time mw%d xlat fails, rc %d\n", i, rc); + } + if (enable_xeon_watchdog != 0) callout_reset(&nt->link_watchdog, 0, xeon_link_watchdog_hb, nt); @@ -1088,6 +1093,7 @@ static void ntb_transport_link_work(void *arg) { struct ntb_transport_ctx *nt = arg; + struct ntb_transport_mw *mw; device_t dev = nt->dev; struct ntb_transport_qp *qp; uint64_t val64, size; @@ -1132,9 +1138,26 @@ ntb_transport_link_work(void *arg) ntb_spad_read(dev, NTBT_MW0_SZ_LOW + (i * 2), &val); val64 |= val; - rc = ntb_set_mw(nt, i, val64); - if (rc != 0) - goto free_mws; + mw = &nt->mw_vec[i]; + val64 = roundup(val64, mw->xlat_align_size); + if (mw->buff_size != val64) { + + rc = ntb_set_mw(nt, i, val64); + if (rc != 0) { + ntb_printf(0, "link up set mw%d fails, rc %d\n", + i, rc); + goto free_mws; + } + + /* Notify HW the memory location of the receive buffer */ + rc = ntb_mw_set_trans(nt->dev, i, mw->dma_addr, + mw->buff_size); + if (rc != 0) { + ntb_printf(0, "link up mw%d xlat fails, rc %d\n", + i, rc); + goto free_mws; + } + } } nt->link_is_up = true; @@ -1179,42 +1202,37 @@ ntb_set_mw(struct ntb_transport_ctx *nt, int num_mw, s { struct ntb_transport_mw *mw = &nt->mw_vec[num_mw]; struct ntb_load_cb_args cba; - size_t xlat_size, buff_size; - int rc; + size_t buff_size; if (size == 0) return (EINVAL); - xlat_size = roundup(size, mw->xlat_align_size); - buff_size = xlat_size; + buff_size = roundup(size, mw->xlat_align_size); /* No need to re-setup */ - if (mw->xlat_size == xlat_size) + if (mw->buff_size == buff_size) return (0); if (mw->buff_size != 0) ntb_free_mw(nt, num_mw); /* Alloc memory for receiving data. Must be aligned */ - mw->xlat_size = xlat_size; mw->buff_size = buff_size; if (bus_dma_tag_create(bus_get_dma_tag(nt->dev), mw->xlat_align, 0, mw->addr_limit, BUS_SPACE_MAXADDR, NULL, NULL, mw->buff_size, 1, mw->buff_size, 0, NULL, NULL, &mw->dma_tag)) { - ntb_printf(0, "Unable to create MW tag of size %zu/%zu\n", - mw->buff_size, mw->xlat_size); - mw->xlat_size = 0; + ntb_printf(0, "Unable to create MW tag of size %zu\n", + mw->buff_size); mw->buff_size = 0; return (ENOMEM); } if (bus_dmamem_alloc(mw->dma_tag, (void **)&mw->virt_addr, BUS_DMA_WAITOK | BUS_DMA_ZERO, &mw->dma_map)) { bus_dma_tag_destroy(mw->dma_tag); - ntb_printf(0, "Unable to allocate MW buffer of size %zu/%zu\n", - mw->buff_size, mw->xlat_size); - mw->xlat_size = 0; + ntb_printf(0, "Unable to allocate MW buffer of size %zu\n", + mw->buff_size); mw->buff_size = 0; return (ENOMEM); } @@ -1222,22 +1240,13 @@ ntb_set_mw(struct ntb_transport_ctx *nt, int num_mw, s mw->buff_size, ntb_load_cb, &cba, BUS_DMA_NOWAIT) || cba.error) { bus_dmamem_free(mw->dma_tag, mw->virt_addr, mw->dma_map); bus_dma_tag_destroy(mw->dma_tag); - ntb_printf(0, "Unable to load MW buffer of size %zu/%zu\n", - mw->buff_size, mw->xlat_size); - mw->xlat_size = 0; + ntb_printf(0, "Unable to load MW buffer of size %zu\n", + mw->buff_size); mw->buff_size = 0; return (ENOMEM); } mw->dma_addr = cba.addr; - /* Notify HW the memory location of the receive buffer */ - rc = ntb_mw_set_trans(nt->dev, num_mw, mw->dma_addr, mw->xlat_size); - if (rc) { - ntb_printf(0, "Unable to set mw%d translation\n", num_mw); - ntb_free_mw(nt, num_mw); - return (rc); - } - return (0); } @@ -1253,7 +1262,6 @@ ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw) bus_dmamap_unload(mw->dma_tag, mw->dma_map); bus_dmamem_free(mw->dma_tag, mw->virt_addr, mw->dma_map); bus_dma_tag_destroy(mw->dma_tag); - mw->xlat_size = 0; mw->buff_size = 0; mw->virt_addr = NULL; } @@ -1280,7 +1288,7 @@ ntb_transport_setup_qp_mw(struct ntb_transport_ctx *nt else num_qps_mw = nt->qp_count / mw_count; - rx_size = mw->xlat_size / num_qps_mw; + rx_size = mw->buff_size / num_qps_mw; qp->rx_buff = mw->virt_addr + rx_size * (qp_num / mw_count); rx_size -= sizeof(struct ntb_rx_info); Modified: stable/12/sys/i386/conf/NOTES ============================================================================== --- stable/12/sys/i386/conf/NOTES Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/i386/conf/NOTES Tue Aug 27 04:13:22 2019 (r351536) @@ -643,6 +643,7 @@ device wpifw device if_ntb # Virtual NTB network interface device ntb_transport # NTB packet transport driver device ntb # NTB hardware interface +device ntb_hw_amd # AMD NTB hardware driver device ntb_hw_intel # Intel NTB hardware driver device ntb_hw_plx # PLX NTB hardware driver Modified: stable/12/sys/modules/ntb/Makefile ============================================================================== --- stable/12/sys/modules/ntb/Makefile Tue Aug 27 04:04:19 2019 (r351535) +++ stable/12/sys/modules/ntb/Makefile Tue Aug 27 04:13:22 2019 (r351536) @@ -1,5 +1,5 @@ # $FreeBSD$ -SUBDIR= ntb ntb_hw_intel ntb_hw_plx ntb_transport if_ntb +SUBDIR= ntb ntb_hw_amd ntb_hw_intel ntb_hw_plx ntb_transport if_ntb .include