From nobody Wed Feb 8 18:15:18 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PBp7q0qrcz3nkf3; Wed, 8 Feb 2023 18:15:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PBp7p4XtSz3qQw; Wed, 8 Feb 2023 18:15:18 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675880118; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gYHCsxQpLR/2kdkkwUdgopBagZ2rN45YwpFUrN6c5wQ=; b=hDut+UMZejViWJTzBFB+fcQ3gHqVL5qTo4pgRRBHkturUPPsUt/rwOEkw9NkdTjiS3UJrZ vUAe3DrJEVGzjCGwEvcv5Kgv9kgdDu0h8+bnaUTJ79VloBVG7XMuZQRpTH5+l/QSXNg2Sr BRKV7EXxTYzwGgTr3FR+vBP2Q2h7sowEuqYUBKtyS+6noxssks7IfKlQ15B59fzLDqvS6M DSn14dHoeB7uE7g3OArlQnPpM/klm9QnSgQG4tuYT2ApnIeLIgh2IAfQMikZwfIuDS02Gl VMkXmOs57jTM4ZR7ioQI/xPmslstqqQlCfqH9Ay9zs1enJkb38V85BCwKgB6jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1675880118; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gYHCsxQpLR/2kdkkwUdgopBagZ2rN45YwpFUrN6c5wQ=; b=ISfSVKuM3RYirAKLwpwP5Y9CCBRGn3QtFTwhUlpzWmGNuL3k7kFVgoqPtOLc61XWMmzTlp o6IFh+AxJpdUzTZKUVnQGQt1TbM0bFgWBh6tP6Juz4v6aDNm1RxHzps3Bmr+gKa02l4vVx jF2zlPqm+MWleFEFNVdMGclwyEwUkuGrxVCYNJOHzvjmO1mPQi9MQCP1DDQfd2WjNde0BC zhWqM+yWUttjCdIeXG4pmFhKcU5Lyy9yv+/WAaXWF/j0mf6NnovHwCfwUYQbBWWnUbASkP Qk9U9DsLj9oq9DL+o+tkgPvK8GEKQCyesUarNIpv22fcovOZluWd2IjSCx9vnA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1675880118; a=rsa-sha256; cv=none; b=JQjtxYiwOXJ57ZGxwJsJrD5DE0ymEoXB59aszPhltzQoNWKQG6JVCPkbTMu2c3BGu4LcY0 Hl34R80SpgzbqS4+FkYhnoG6QxBKiaXMGxmyVxmdYSwGQ/45sYq69O2ai3E0cz2dEmj9tC u004pI/9Tna7iEW0hooQ12wkUbtqmikcgJZRbpvBWXIiChaXBYxLzfkNanAGQAVnK/uQal fTRjsn4OvCmmbc0QfqZkjO/oxpTRY5TFDDMQb/VKbnuWNJkAm2jw6oD9tnAKLxxJrMONMe hBXdtCOL/ksn9F/YNS8WJ4qPv+LzRtDeL06tFgPGHF+cbO/CAC6Z8/uVA+Tuew== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PBp7p3fpGz167T; Wed, 8 Feb 2023 18:15:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 318IFIYZ022312; Wed, 8 Feb 2023 18:15:18 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 318IFItT022311; Wed, 8 Feb 2023 18:15:18 GMT (envelope-from git) Date: Wed, 8 Feb 2023 18:15:18 GMT Message-Id: <202302081815.318IFItT022311@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Gordon Tetlow Subject: git: b508850e150e - releng/13.1 - Fix ena driver crash after reset in 7th gen AWS instance types. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: gordon X-Git-Repository: src X-Git-Refname: refs/heads/releng/13.1 X-Git-Reftype: branch X-Git-Commit: b508850e150ebc7a8bea0cdf0a4981c2bc4d5323 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch releng/13.1 has been updated by gordon: URL: https://cgit.FreeBSD.org/src/commit/?id=b508850e150ebc7a8bea0cdf0a4981c2bc4d5323 commit b508850e150ebc7a8bea0cdf0a4981c2bc4d5323 Author: Gordon Tetlow AuthorDate: 2023-02-08 16:18:27 +0000 Commit: Gordon Tetlow CommitDate: 2023-02-08 16:18:27 +0000 Fix ena driver crash after reset in 7th gen AWS instance types. Approved by: so Security: FreeBSD-EN-23:03.ena --- sys/dev/ena/ena.c | 54 ++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 34 insertions(+), 20 deletions(-) diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index 1b26a91c5d9e..36b84dea9a76 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -155,6 +155,7 @@ static int ena_setup_ifnet(device_t, struct ena_adapter *, static int ena_enable_wc(device_t, struct resource *); static int ena_set_queues_placement_policy(device_t, struct ena_com_dev *, struct ena_admin_feature_llq_desc *, struct ena_llq_configurations *); +static int ena_map_llq_mem_bar(device_t, struct ena_com_dev *); static uint32_t ena_calc_max_io_queue_num(device_t, struct ena_com_dev *, struct ena_com_dev_get_features_ctx *); static int ena_calc_io_queue_size(struct ena_calc_queue_size_ctx *); @@ -2571,8 +2572,7 @@ ena_set_queues_placement_policy(device_t pdev, struct ena_com_dev *ena_dev, struct ena_admin_feature_llq_desc *llq, struct ena_llq_configurations *llq_default_configurations) { - struct ena_adapter *adapter = device_get_softc(pdev); - int rc, rid; + int rc; uint32_t llq_feature_mask; llq_feature_mask = 1 << ENA_ADMIN_LLQ; @@ -2583,26 +2583,36 @@ ena_set_queues_placement_policy(device_t pdev, struct ena_com_dev *ena_dev, return (0); } + if (ena_dev->mem_bar == NULL) { + ena_log(pdev, WARN, + "LLQ is advertised as supported but device doesn't expose mem bar.\n"); + ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + return (0); + } + rc = ena_com_config_dev_mode(ena_dev, llq, llq_default_configurations); if (unlikely(rc != 0)) { ena_log(pdev, WARN, "Failed to configure the device mode. " "Fallback to host mode policy.\n"); ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; - return (0); } - /* Nothing to config, exit */ - if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) - return (0); + return (0); +} + +static int +ena_map_llq_mem_bar(device_t pdev, struct ena_com_dev *ena_dev) +{ + struct ena_adapter *adapter = device_get_softc(pdev); + int rc, rid; /* Try to allocate resources for LLQ bar */ rid = PCIR_BAR(ENA_MEM_BAR); adapter->memory = bus_alloc_resource_any(pdev, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (unlikely(adapter->memory == NULL)) { - ena_log(pdev, WARN, "unable to allocate LLQ bar resource. " - "Fallback to host mode policy.\n"); - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + ena_log(pdev, WARN, + "Unable to allocate LLQ bar resource. LLQ mode won't be used.\n"); return (0); } @@ -2791,6 +2801,7 @@ static int ena_device_init(struct ena_adapter *adapter, device_t pdev, struct ena_com_dev_get_features_ctx *get_feat_ctx, int *wd_active) { + struct ena_llq_configurations llq_config; struct ena_com_dev* ena_dev = adapter->ena_dev; bool readless_supported; uint32_t aenq_groups; @@ -2870,6 +2881,15 @@ ena_device_init(struct ena_adapter *adapter, device_t pdev, *wd_active = !!(aenq_groups & BIT(ENA_ADMIN_KEEP_ALIVE)); + set_default_llq_configurations(&llq_config, &get_feat_ctx->llq); + + rc = ena_set_queues_placement_policy(pdev, ena_dev, &get_feat_ctx->llq, + &llq_config); + if (unlikely(rc != 0)) { + ena_log(pdev, ERR, "Failed to set placement policy\n"); + goto err_admin_init; + } + return (0); err_admin_init: @@ -3506,7 +3526,6 @@ static int ena_attach(device_t pdev) { struct ena_com_dev_get_features_ctx get_feat_ctx; - struct ena_llq_configurations llq_config; struct ena_calc_queue_size_ctx calc_queue_ctx = { 0 }; static int version_printed; struct ena_adapter *adapter; @@ -3579,7 +3598,11 @@ ena_attach(device_t pdev) goto err_bus_free; } - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + rc = ena_map_llq_mem_bar(pdev, ena_dev); + if (unlikely(rc != 0)) { + ena_log(pdev, ERR, "Failed to map ENA mem bar"); + goto err_bus_free; + } /* Initially clear all the flags */ ENA_FLAG_ZERO(adapter); @@ -3592,15 +3615,6 @@ ena_attach(device_t pdev) goto err_bus_free; } - set_default_llq_configurations(&llq_config, &get_feat_ctx.llq); - - rc = ena_set_queues_placement_policy(pdev, ena_dev, &get_feat_ctx.llq, - &llq_config); - if (unlikely(rc != 0)) { - ena_log(pdev, ERR, "failed to set placement policy\n"); - goto err_com_free; - } - if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) adapter->disable_meta_caching = !!(get_feat_ctx.llq.accel_mode.u.get.supported_flags &