From owner-svn-src-all@freebsd.org Thu Sep 6 12:28:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3ED9CFF9EB7; Thu, 6 Sep 2018 12:28:07 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E973A74AF2; Thu, 6 Sep 2018 12:28:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E43C14FF4; Thu, 6 Sep 2018 12:28:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w86CS6PK047001; Thu, 6 Sep 2018 12:28:06 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w86CS6Bo046998; Thu, 6 Sep 2018 12:28:06 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201809061228.w86CS6Bo046998@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 6 Sep 2018 12:28:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r338492 - head/sys/dev/mlx5/mlx5_en X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: head/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 338492 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2018 12:28:07 -0000 Author: hselasky Date: Thu Sep 6 12:28:06 2018 New Revision: 338492 URL: https://svnweb.freebsd.org/changeset/base/338492 Log: Add support for receive side scaling stride, RSSS, in mlx5en(4). The receive side scaling stride parameter is a value which define the interval between active receive side queues. The traffic for the inactive queues is redirected to the nearest active queue by use of modulus. The default value of this parameter is one, which means all receive side queues are used. The point of this feature is to redirect more traffic to fewer receive side queues in order to take more advantage of sorted large receive offload, sorted LRO. The sorted LRO works better when more packets are accumulated per service interval. MFC after: 3 days Approved by: re (marius) Sponsored by: Mellanox Technologies Modified: head/sys/dev/mlx5/mlx5_en/en.h head/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Modified: head/sys/dev/mlx5/mlx5_en/en.h ============================================================================== --- head/sys/dev/mlx5/mlx5_en/en.h Thu Sep 6 12:26:57 2018 (r338491) +++ head/sys/dev/mlx5/mlx5_en/en.h Thu Sep 6 12:28:06 2018 (r338492) @@ -454,6 +454,7 @@ struct mlx5e_params { u32 rx_priority_flow_control __aligned(4); u16 tx_max_inline; u8 tx_min_inline_mode; + u8 channels_rsss; }; #define MLX5E_PARAMS(m) \ @@ -462,6 +463,7 @@ struct mlx5e_params { m(+1, u64 tx_queue_size, "tx_queue_size", "Default send queue size") \ m(+1, u64 rx_queue_size, "rx_queue_size", "Default receive queue size") \ m(+1, u64 channels, "channels", "Default number of channels") \ + m(+1, u64 channels_rsss, "channels_rsss", "Default channels receive side scaling stride") \ m(+1, u64 coalesce_usecs_max, "coalesce_usecs_max", "Maximum usecs for joining packets") \ m(+1, u64 coalesce_pkts_max, "coalesce_pkts_max", "Maximum packets to join") \ m(+1, u64 rx_coalesce_usecs, "rx_coalesce_usecs", "Limit in usec for joining rx packets") \ Modified: head/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c ============================================================================== --- head/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Sep 6 12:26:57 2018 (r338491) +++ head/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Sep 6 12:28:06 2018 (r338492) @@ -493,6 +493,24 @@ mlx5e_ethtool_handler(SYSCTL_HANDLER_ARGS) mlx5e_open_locked(priv->ifp); break; + case MLX5_PARAM_OFFSET(channels_rsss): + /* network interface must be down */ + if (was_opened) + mlx5e_close_locked(priv->ifp); + + /* import number of channels */ + if (priv->params_ethtool.channels_rsss < 1) + priv->params_ethtool.channels_rsss = 1; + else if (priv->params_ethtool.channels_rsss > 128) + priv->params_ethtool.channels_rsss = 128; + + priv->params.channels_rsss = priv->params_ethtool.channels_rsss; + + /* restart network interface, if any */ + if (was_opened) + mlx5e_open_locked(priv->ifp); + break; + case MLX5_PARAM_OFFSET(channels): /* network interface must be down */ if (was_opened) @@ -1041,6 +1059,7 @@ mlx5e_create_ethtool(struct mlx5e_priv *priv) priv->params_ethtool.tx_queue_size = 1 << priv->params.log_sq_size; priv->params_ethtool.rx_queue_size = 1 << priv->params.log_rq_size; priv->params_ethtool.channels = priv->params.num_channels; + priv->params_ethtool.channels_rsss = priv->params.channels_rsss; priv->params_ethtool.coalesce_pkts_max = MLX5E_FLD_MAX(cqc, cq_max_count); priv->params_ethtool.coalesce_usecs_max = MLX5E_FLD_MAX(cqc, cq_period); priv->params_ethtool.rx_coalesce_mode = priv->params.rx_cq_moderation_mode; Modified: head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Sep 6 12:26:57 2018 (r338491) +++ head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Sep 6 12:28:06 2018 (r338492) @@ -2204,14 +2204,16 @@ mlx5e_open_rqt(struct mlx5e_priv *priv) MLX5_SET(rqtc, rqtc, rqt_max_size, sz); for (i = 0; i < sz; i++) { - int ix; + int ix = i; #ifdef RSS - ix = rss_get_indirection_to_bucket(i); -#else - ix = i; + ix = rss_get_indirection_to_bucket(ix); #endif /* ensure we don't overflow */ ix %= priv->params.num_channels; + + /* apply receive side scaling stride, if any */ + ix -= ix % (int)priv->params.channels_rsss; + MLX5_SET(rqtc, rqtc, rq_num[i], priv->channel[ix]->rq.rqn); } @@ -3083,6 +3085,7 @@ mlx5e_build_ifp_priv(struct mlx5_core_dev *mdev, priv->mdev = mdev; priv->params.num_channels = num_comp_vectors; + priv->params.channels_rsss = 1; priv->order_base_2_num_channels = order_base_2(num_comp_vectors); priv->queue_mapping_channel_mask = roundup_pow_of_two(num_comp_vectors) - 1;