From owner-svn-src-all@freebsd.org Thu Aug 2 08:42:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEEA6105EE6B; Thu, 2 Aug 2018 08:42:42 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70F007D0BE; Thu, 2 Aug 2018 08:42:42 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 51DC875D1; Thu, 2 Aug 2018 08:42:42 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728ggR3056083; Thu, 2 Aug 2018 08:42:42 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728geCx056075; Thu, 2 Aug 2018 08:42:40 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020842.w728geCx056075@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:42:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337103 - in stable/11/sys: conf dev/mlx5 dev/mlx5/mlx5_core modules/mlx5 X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys: conf dev/mlx5 dev/mlx5/mlx5_core modules/mlx5 X-SVN-Commit-Revision: 337103 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:42:43 -0000 Author: hselasky Date: Thu Aug 2 08:42:40 2018 New Revision: 337103 URL: https://svnweb.freebsd.org/changeset/base/337103 Log: MFC r336397: Refactor access to CR-space into using VSC APIs in mlx5core. Remove no longer used files and APIs. Sponsored by: Mellanox Technologies Deleted: stable/11/sys/dev/mlx5/mlx5_core/mlx5_crspace.c Modified: stable/11/sys/conf/files stable/11/sys/dev/mlx5/driver.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c stable/11/sys/modules/mlx5/Makefile Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/conf/files ============================================================================== --- stable/11/sys/conf/files Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/conf/files Thu Aug 2 08:42:40 2018 (r337103) @@ -4542,8 +4542,6 @@ dev/mlx5/mlx5_core/mlx5_cmd.c optional mlx5 pci \ compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_cq.c optional mlx5 pci \ compile-with "${OFED_C}" -dev/mlx5/mlx5_core/mlx5_crspace.c optional mlx5 pci \ - compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_diagnostics.c optional mlx5 pci \ compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_eq.c optional mlx5 pci \ Modified: stable/11/sys/dev/mlx5/driver.h ============================================================================== --- stable/11/sys/dev/mlx5/driver.h Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/driver.h Thu Aug 2 08:42:40 2018 (r337103) @@ -656,7 +656,6 @@ struct mlx5_core_dev { struct mlx5_flow_root_namespace *sniffer_tx_root_ns; u32 num_q_counter_allocated[MLX5_INTERFACE_NUMBER]; struct mlx5_dump_data *dump_data; - u32 vsec_addr; }; enum { @@ -1029,8 +1028,11 @@ int mlx5_vsc_find_cap(struct mlx5_core_dev *mdev); int mlx5_vsc_lock(struct mlx5_core_dev *mdev); void mlx5_vsc_unlock(struct mlx5_core_dev *mdev); int mlx5_vsc_set_space(struct mlx5_core_dev *mdev, u16 space); -int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, u32 *data); +int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, const u32 *data); int mlx5_vsc_read(struct mlx5_core_dev *mdev, u32 addr, u32 *data); +int mlx5_vsc_lock_addr_space(struct mlx5_core_dev *mdev, u32 addr); +int mlx5_vsc_unlock_addr_space(struct mlx5_core_dev *mdev, u32 addr); + static inline u32 mlx5_mkey_to_idx(u32 mkey) { return mkey >> 8; Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:42:40 2018 (r337103) @@ -70,12 +70,6 @@ enum mlx5_semaphore_space_address { MLX5_SEMAPHORE_SW_RESET = 0x20, }; -enum { - UNLOCK = 0, - LOCK = 1, - CAP_ID = 0x9, -}; - struct mlx5_core_dev; int mlx5_query_hca_caps(struct mlx5_core_dev *dev); @@ -109,8 +103,4 @@ struct mlx5_crspace_regmap { extern struct pci_driver mlx5_core_driver; -void mlx5_vsec_init(struct mlx5_core_dev *dev); -int mlx5_pciconf_cap9_sem(struct mlx5_core_dev *dev, int state); -int mlx5_pciconf_set_sem_addr_space(struct mlx5_core_dev *dev, - u32 sem_space_address, int state); #endif /* __MLX5_CORE_H__ */ Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:42:40 2018 (r337103) @@ -59,19 +59,19 @@ enum { MLX5_SENSOR_FW_SYND_RFR = 5, }; -static int lock_sem_sw_reset(struct mlx5_core_dev *dev, int state) +static int lock_sem_sw_reset(struct mlx5_core_dev *dev) { - int ret, err; + int ret; /* Lock GW access */ - ret = mlx5_pciconf_cap9_sem(dev, LOCK); + ret = -mlx5_vsc_lock(dev); if (ret) { - mlx5_core_warn(dev, "Timed out locking gateway %d, %d\n", state, ret); + mlx5_core_warn(dev, "Timed out locking gateway %d\n", ret); return ret; } - ret = mlx5_pciconf_set_sem_addr_space(dev, MLX5_SEMAPHORE_SW_RESET, state); - if (ret && state == LOCK) { + ret = -mlx5_vsc_lock_addr_space(dev, MLX5_SEMAPHORE_SW_RESET); + if (ret) { if (ret == -EBUSY) mlx5_core_dbg(dev, "SW reset FW semaphore already locked, another function will handle the reset\n"); else @@ -79,13 +79,30 @@ static int lock_sem_sw_reset(struct mlx5_core_dev *dev } /* Unlock GW access */ - err = mlx5_pciconf_cap9_sem(dev, UNLOCK); - if (err) - mlx5_core_warn(dev, "Timed out unlocking gateway: state %d, err %d\n", state, err); + mlx5_vsc_unlock(dev); return ret; } +static int unlock_sem_sw_reset(struct mlx5_core_dev *dev) +{ + int ret; + + /* Lock GW access */ + ret = -mlx5_vsc_lock(dev); + if (ret) { + mlx5_core_warn(dev, "Timed out locking gateway %d\n", ret); + return ret; + } + + ret = -mlx5_vsc_unlock_addr_space(dev, MLX5_SEMAPHORE_SW_RESET); + + /* Unlock GW access */ + mlx5_vsc_unlock(dev); + + return ret; +} + static u8 get_nic_mode(struct mlx5_core_dev *dev) { return (ioread32be(&dev->iseg->cmdq_addr_l_sz) >> 8) & 7; @@ -223,7 +240,7 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, if (fatal_error == MLX5_SENSOR_FW_SYND_RFR) { /* Get cr-dump and reset FW semaphore */ if (mlx5_core_is_pf(dev)) - lock = lock_sem_sw_reset(dev, LOCK); + lock = lock_sem_sw_reset(dev); /* Execute cr-dump and SW reset */ if (lock != -EBUSY) { @@ -249,7 +266,7 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, /* Release FW semaphore if you are the lock owner */ if (!lock) - lock_sem_sw_reset(dev, UNLOCK); + unlock_sem_sw_reset(dev); mlx5_core_err(dev, "system error event triggered\n"); Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:42:40 2018 (r337103) @@ -873,7 +873,9 @@ static int mlx5_init_once(struct mlx5_core_dev *dev, s struct pci_dev *pdev = dev->pdev; int err; - mlx5_vsec_init(dev); + err = mlx5_vsc_find_cap(dev); + if (err) + dev_err(&pdev->dev, "Unable to find vendor specific capabilities\n"); err = mlx5_query_hca_caps(dev); if (err) { Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c Thu Aug 2 08:42:40 2018 (r337103) @@ -29,6 +29,8 @@ #include #include +#define MLX5_SEMAPHORE_SPACE_DOMAIN 0xA + struct mlx5_ifc_vsc_space_bits { u8 status[0x3]; u8 reserved0[0xd]; @@ -139,7 +141,7 @@ int mlx5_vsc_set_space(struct mlx5_core_dev *mdev, u16 return 0; } -int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, u32 *data) +int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, const u32 *data) { device_t dev = mdev->pdev->dev.bsddev; int vsc_addr = mdev->vsc_addr; @@ -185,6 +187,60 @@ int mlx5_vsc_read(struct mlx5_core_dev *mdev, u32 addr } *data = pci_read_config(dev, vsc_addr + MLX5_VSC_DATA_OFFSET, 4); + + return 0; +} + +int mlx5_vsc_lock_addr_space(struct mlx5_core_dev *mdev, u32 addr) +{ + device_t dev = mdev->pdev->dev.bsddev; + int vsc_addr = mdev->vsc_addr; + u32 data; + int ret; + u32 id; + + ret = mlx5_vsc_set_space(mdev, MLX5_SEMAPHORE_SPACE_DOMAIN); + if (ret) + return ret; + + /* Get a unique ID based on the counter */ + id = pci_read_config(dev, vsc_addr + MLX5_VSC_COUNTER_OFFSET, 4); + + /* Try to modify lock */ + ret = mlx5_vsc_write(mdev, addr, &id); + if (ret) + return ret; + + /* Verify */ + ret = mlx5_vsc_read(mdev, addr, &data); + if (ret) + return ret; + if (data != id) + return EBUSY; + + return 0; +} + +int mlx5_vsc_unlock_addr_space(struct mlx5_core_dev *mdev, u32 addr) +{ + u32 data = 0; + int ret; + + ret = mlx5_vsc_set_space(mdev, MLX5_SEMAPHORE_SPACE_DOMAIN); + if (ret) + return ret; + + /* Try to modify lock */ + ret = mlx5_vsc_write(mdev, addr, &data); + if (ret) + return ret; + + /* Verify */ + ret = mlx5_vsc_read(mdev, addr, &data); + if (ret) + return ret; + if (data != 0) + return EBUSY; return 0; } Modified: stable/11/sys/modules/mlx5/Makefile ============================================================================== --- stable/11/sys/modules/mlx5/Makefile Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/modules/mlx5/Makefile Thu Aug 2 08:42:40 2018 (r337103) @@ -6,7 +6,6 @@ SRCS= \ mlx5_alloc.c \ mlx5_cmd.c \ mlx5_cq.c \ -mlx5_crspace.c \ mlx5_diagnostics.c \ mlx5_eq.c \ mlx5_fs_cmd.c \