From owner-svn-src-head@freebsd.org Fri May 19 12:22:50 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65AF3D7227F; Fri, 19 May 2017 12:22:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 238A71C01; Fri, 19 May 2017 12:22:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v4JCMnNX004848; Fri, 19 May 2017 12:22:49 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v4JCMni4004846; Fri, 19 May 2017 12:22:49 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201705191222.v4JCMni4004846@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Fri, 19 May 2017 12:22:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r318531 - head/sys/dev/mlx4/mlx4_core X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 May 2017 12:22:50 -0000 Author: hselasky Date: Fri May 19 12:22:48 2017 New Revision: 318531 URL: https://svnweb.freebsd.org/changeset/base/318531 Log: mlx4: Use the CQ quota for SRIOV when creating completion EQs When creating EQs to handle CQ completion events for the PF or for VFs, we create enough EQE entries to handle completions for the max number of CQs that can use that EQ. When SRIOV is activated, the max number of CQs a VF (or the PF) can obtain is its CQ quota (determined by the Hypervisor resource tracker). Therefore, when creating an EQ, the number of EQE entries that the VF should request for that EQ is the CQ quota value (and not the total number of CQs available in the firmware). Under SRIOV, the PF, also must use its CQ quota, because the resource tracker also controls how many CQs the PF can obtain. Using the firmware total CQs instead of the CQ quota when creating EQs resulted wasting MTT entries, due to allocating more EQEs than were needed. MFC after: 3 days Sponsored by: Mellanox Technologies Modified: head/sys/dev/mlx4/mlx4_core/mlx4_eq.c head/sys/dev/mlx4/mlx4_core/mlx4_main.c Modified: head/sys/dev/mlx4/mlx4_core/mlx4_eq.c ============================================================================== --- head/sys/dev/mlx4/mlx4_core/mlx4_eq.c Fri May 19 11:45:14 2017 (r318530) +++ head/sys/dev/mlx4/mlx4_core/mlx4_eq.c Fri May 19 12:22:48 2017 (r318531) @@ -1169,8 +1169,7 @@ int mlx4_init_eq_table(struct mlx4_dev * } for (i = 0; i < dev->caps.num_comp_vectors; ++i) { - err = mlx4_create_eq(dev, dev->caps.num_cqs - - dev->caps.reserved_cqs + + err = mlx4_create_eq(dev, dev->quotas.cq + MLX4_NUM_SPARE_EQE, (dev->flags & MLX4_FLAG_MSI_X) ? i : 0, &priv->eq_table.eq[i]); @@ -1190,8 +1189,7 @@ int mlx4_init_eq_table(struct mlx4_dev * for (i = dev->caps.num_comp_vectors + 1; i < dev->caps.num_comp_vectors + dev->caps.comp_pool + 1; ++i) { - err = mlx4_create_eq(dev, dev->caps.num_cqs - - dev->caps.reserved_cqs + + err = mlx4_create_eq(dev, dev->quotas.cq + MLX4_NUM_SPARE_EQE, (dev->flags & MLX4_FLAG_MSI_X) ? i : 0, &priv->eq_table.eq[i]); Modified: head/sys/dev/mlx4/mlx4_core/mlx4_main.c ============================================================================== --- head/sys/dev/mlx4/mlx4_core/mlx4_main.c Fri May 19 11:45:14 2017 (r318530) +++ head/sys/dev/mlx4/mlx4_core/mlx4_main.c Fri May 19 12:22:48 2017 (r318531) @@ -3446,6 +3446,8 @@ slave_start: goto err_free_eq; } + mlx4_init_quotas(dev); + err = mlx4_setup_hca(dev); if (err == -EBUSY && (dev->flags & MLX4_FLAG_MSI_X) && !mlx4_is_mfunc(dev)) { @@ -3459,7 +3461,6 @@ slave_start: if (err) goto err_steer; - mlx4_init_quotas(dev); mlx4_init_hca_info(dev); for (port = 1; port <= dev->caps.num_ports; port++) {