From nobody Tue Jul 26 19:33:36 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LsnC44lsDz4Wx99; Tue, 26 Jul 2022 19:33:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LsnC44JXXz48Tw; Tue, 26 Jul 2022 19:33:36 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658864016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=N5p7u3713Ap/XG4T1EqgbaRwfkCKiLwtaZT0S/sNPzE=; b=ZLXUht0Tt7zX7UbTplIWvrgsIfFmbfRWRE0RJ2QORSEbrg20ALBEYD1ODxGrx0PqQeVGYS tbbcUrj7cDUo2Ed1fZjgDNB74V5ErqQAxhqGWVcnTPZdTQ2pMKL5TEY14beCLNFc4IvmTd F7k7DeUygls/m6pvy4vkPcw3ntkNttwosZWlsvuxQheUq7/vP/Msv2T/hK8EHdv31NiKKo bPCnC06/3PesJisotShd8oaMJfCbXWZqJvU85YtrQ4V4h4b73wcJ3VPg3j/IRu1eDjje1d ns05Dh//nPkpxSlzw2bCym6+a0Z1m+yN3gCNYs/vLFkU8IIhi1Ce8fIAABadHg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4LsnC43Q1GzplN; Tue, 26 Jul 2022 19:33:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 26QJXa1k034082; Tue, 26 Jul 2022 19:33:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 26QJXamT034081; Tue, 26 Jul 2022 19:33:36 GMT (envelope-from git) Date: Tue, 26 Jul 2022 19:33:36 GMT Message-Id: <202207261933.26QJXamT034081@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Marcin Wojtas Subject: git: 417cd20ee3c4 - stable/12 - ena: Fix LLQ descriptor reconfiguration List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mw X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 417cd20ee3c470ae76bfc9305eea5600588c28b9 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658864016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=N5p7u3713Ap/XG4T1EqgbaRwfkCKiLwtaZT0S/sNPzE=; b=e8IxVGS2B6DbUn/4cs2FlluFhfXksAcyJdsJ8ysSNF1ELQyhtQSQJDtbCM5K1JCanU7gjr eUY+BApx9OJfVQo1RRXdAOa7ANPIpAxNO4RkNpggnsMIpUBz8eDe8jdafgHQzSExEU0mWS HUdIgaDrcoezwEuM+vNq9ZaXRahlf1colqld9qF8q+AdVKhhE60TfuTx89klMY9SUAafl+ HUvXHNuIw5WGoKt+AidZBqQAspjEj7f5XZXsjG53xOwGyAGRfue68gLMAQ4t7o9HpcTw+n yqrJjc+sHwu+LOkSEXFOpkMSLhteDue2IZsLEqyh4KIX6DJuo84sf5HMkDOfeQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1658864016; a=rsa-sha256; cv=none; b=t1thRd7qzXCkC4d8czUbjhHyk/gXD0thrRFo4fyhvm2XIp9hhIkq7wKVf6ab/jvWhBAc4o oxgREjkZoZcMruBOhdFODkLWBSC8YmutUMI/N4/0nJ9B6vKV7iisO4CSan4IzIPCr3jnM2 ZO+6oFnBkxefzqHYJ3YoY9eCwL+9kloCwW0vrXH1OpkhOn3Ey0kgyd2EeZgWyt1JXyV1O4 1n6hwqWMmFrOFbFid7L5B/CgeVSdwlxBcrEwG+MBhd6lIb8gx9VqomPBEgtNc7uU6YqIsN g2QlWw4dmnGFmvRaI5/tWG6MRJa9+dPyS2xZ3SzbeQy9jjQ0VcIEFMvU7dK36g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by mw: URL: https://cgit.FreeBSD.org/src/commit/?id=417cd20ee3c470ae76bfc9305eea5600588c28b9 commit 417cd20ee3c470ae76bfc9305eea5600588c28b9 Author: Michal Krawczyk AuthorDate: 2022-07-05 10:59:25 +0000 Commit: Marcin Wojtas CommitDate: 2022-07-26 19:33:05 +0000 ena: Fix LLQ descriptor reconfiguration After the device reset, the LLQ configuration descriptor wasn't passed to the hardware. On a 6-generation AWS instances (like C6gn), it is required to pass the LLQ descriptor after the device reset, otherwise the hardware will be missing the LLQ configuration resulting in performance degradation. This patch reconfigures the LLQ each time the ena_device_init() is called. This means that the LLQ descriptor will be passed during the initial configuration and after a reset. The ena_map_llq_mem_bar() function call was moved before the ena_device_init() call, to make sure that the mem bar is available. Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commit 3324e304c14d121dc04338a38c7f3277cbfba58b) --- sys/dev/ena/ena.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index 3efc0021a746..a93e55e36bde 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -2583,8 +2583,7 @@ ena_map_llq_mem_bar(device_t pdev, struct ena_com_dev *ena_dev) RF_ACTIVE); if (unlikely(adapter->memory == NULL)) { ena_log(pdev, WARN, - "unable to allocate LLQ bar resource. Fallback to host mode policy.\n"); - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + "Unable to allocate LLQ bar resource. LLQ mode won't be used.\n"); return (0); } @@ -2769,6 +2768,7 @@ static int ena_device_init(struct ena_adapter *adapter, device_t pdev, struct ena_com_dev_get_features_ctx *get_feat_ctx, int *wd_active) { + struct ena_llq_configurations llq_config; struct ena_com_dev *ena_dev = adapter->ena_dev; bool readless_supported; uint32_t aenq_groups; @@ -2848,6 +2848,15 @@ ena_device_init(struct ena_adapter *adapter, device_t pdev, *wd_active = !!(aenq_groups & BIT(ENA_ADMIN_KEEP_ALIVE)); + set_default_llq_configurations(&llq_config, &get_feat_ctx->llq); + + rc = ena_set_queues_placement_policy(pdev, ena_dev, &get_feat_ctx->llq, + &llq_config); + if (unlikely(rc != 0)) { + ena_log(pdev, ERR, "Failed to set placement policy\n"); + goto err_admin_init; + } + return (0); err_admin_init: @@ -3491,7 +3500,6 @@ static int ena_attach(device_t pdev) { struct ena_com_dev_get_features_ctx get_feat_ctx; - struct ena_llq_configurations llq_config; struct ena_calc_queue_size_ctx calc_queue_ctx = { 0 }; static int version_printed; struct ena_adapter *adapter; @@ -3564,7 +3572,11 @@ ena_attach(device_t pdev) goto err_bus_free; } - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + rc = ena_map_llq_mem_bar(pdev, ena_dev); + if (unlikely(rc != 0)) { + ena_log(pdev, ERR, "Failed to map ENA mem bar"); + goto err_bus_free; + } /* Initially clear all the flags */ ENA_FLAG_ZERO(adapter); @@ -3577,21 +3589,6 @@ ena_attach(device_t pdev) goto err_bus_free; } - set_default_llq_configurations(&llq_config, &get_feat_ctx.llq); - - rc = ena_map_llq_mem_bar(pdev, ena_dev); - if (unlikely(rc != 0)) { - ena_log(pdev, ERR, "failed to map ENA mem bar"); - goto err_com_free; - } - - rc = ena_set_queues_placement_policy(pdev, ena_dev, &get_feat_ctx.llq, - &llq_config); - if (unlikely(rc != 0)) { - ena_log(pdev, ERR, "failed to set placement policy\n"); - goto err_com_free; - } - if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) adapter->disable_meta_caching = !!( get_feat_ctx.llq.accel_mode.u.get.supported_flags &