From nobody Thu Jul 20 22:37:38 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R6SHl0F6Pz4dZ1n; Thu, 20 Jul 2023 22:37:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R6SHk6mjVz4Fml; Thu, 20 Jul 2023 22:37:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689892659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Wq4pEaSs314yab6KMoV9rbyX3xcH1nswkBaKozWKGeU=; b=eXOSr0p2X28TzdC2EHe6LAeWmd3omJPravJwgw8EvPGusnMRjeJw5U+W6EZqzagQpGOQ72 cNNnxnMRT3T8wFJnHKePfrVXYI1mT9sJOHXbxsNU5psogfBX4zPRxiAPMPRP+l39pQQdlG DqtWj3wSzBjumvO2R2dfo873zRVcNnTxQGOEv16Tw9u3uS2t70nwCLyDRQ4GCw40CSdt5p OHx4eVJEa+oqvHpzHkbukrhbfp10CIiHgK2gUu6UanYCzCfQosdVXjL51T0pt52NSf4t/P bHm1yxzgMEPI1IsJsjX81b5HNUYx57bhntPB+aDkAzholhS3AkHrgU4pajM75g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689892658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Wq4pEaSs314yab6KMoV9rbyX3xcH1nswkBaKozWKGeU=; b=J6tc/fWSPnWpx7PURbwM+LpHjwbXBwBVQ2r9rtRjHggLvTs1yyg7MS0/qIpl4pae2IsbWu DRupVaUO6y9wMSMDfN4cE/QUtYot0ySc4/OXPY44XD5RL9lcoPJ87I27B5SEXmUpBZNrLH bcZybTzGzjxGaGpIA6BvOPxMG6jgboIJAzFooRDkNi7DqMeRPeae8zk17KlYuWzgmeFgpl B7rVv3Z2Q+LadcYAVXzdVfzOe1XnRO/BM2pLeJc9wAHGDNPf87G5qd3yPPOPDlrny4hPgB Pdm/UtlRe4thJqWe3FHz6jRTW6pqmDjtFyPsYPG7MNfZ1cwPQddjqirmbTlB8A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1689892658; a=rsa-sha256; cv=none; b=NHP+6b6WHHqBPPLrs7aXB1BDBpYXptTuiU13sqhqLt3t/BiyElZ9xcNj+6nBM6gNPxBtL3 bKXCV3rr+oNik4+fVAGrwitmNtPqBeWEEoOlXCU8N4xTzgczkJCBG7NLWpO4zZpqyEUdh5 cCZKshFtQEPSC6Kf680hVD/Rh4Hl4YTSbzvGt5m3EmrRnIDeue/xnWOvTUEXxXNqsvd8cs X2AjHHFZvznkzJ8vrBUMfmFkQMJYJK/RlM+Q5TOLZQRUJVFiQxsyEdlHrl90sDtN6vw6Ya c5XRF0P3Pn3r62skaXosUgX8NBQUJOHMYR/x5D8NC2K3onZ2ogdXXl9BoZyArg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4R6SHk5m0szpBL; Thu, 20 Jul 2023 22:37:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 36KMbcNg018310; Thu, 20 Jul 2023 22:37:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 36KMbcVY018309; Thu, 20 Jul 2023 22:37:38 GMT (envelope-from git) Date: Thu, 20 Jul 2023 22:37:38 GMT Message-Id: <202307202237.36KMbcVY018309@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Eric Joyner Subject: git: ba2f531f816a - main - ixl(4): Add link state polling List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: erj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ba2f531f816a6bc1ef5f2cba4a329ff7bdbec0f3 Auto-Submitted: auto-generated The branch main has been updated by erj: URL: https://cgit.FreeBSD.org/src/commit/?id=ba2f531f816a6bc1ef5f2cba4a329ff7bdbec0f3 commit ba2f531f816a6bc1ef5f2cba4a329ff7bdbec0f3 Author: Krzysztof Galazka AuthorDate: 2023-07-20 22:33:52 +0000 Commit: Eric Joyner CommitDate: 2023-07-20 22:33:52 +0000 ixl(4): Add link state polling In some cases driver may ask FW about link state before FW finishes configuration of a (Q)SFP+ transceiver. If first attempt of using Get Link Status AQC after loading driver or handling a reset fails, then re-try periodically for 5 seconds. Signed-off-by: Krzysztof Galazka Signed-off-by: Eric Joyner Tested by: jeffrey.e.pieper@intel.com Approved by: erj@ MFC after: 2 days Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D40899 --- sys/dev/ixl/i40e_adminq_cmd.h | 2 +- sys/dev/ixl/if_ixl.c | 50 ++++++++++++++++++++++++++++++++++++------- sys/dev/ixl/ixl.h | 2 ++ sys/dev/ixl/ixl_debug.h | 2 ++ sys/dev/ixl/ixl_pf.h | 4 ++++ sys/dev/ixl/ixl_pf_iflib.c | 4 +--- sys/dev/ixl/ixl_pf_main.c | 30 ++++++++++++++++++++------ 7 files changed, 75 insertions(+), 19 deletions(-) diff --git a/sys/dev/ixl/i40e_adminq_cmd.h b/sys/dev/ixl/i40e_adminq_cmd.h index 564a076761d0..679e191412cd 100644 --- a/sys/dev/ixl/i40e_adminq_cmd.h +++ b/sys/dev/ixl/i40e_adminq_cmd.h @@ -44,7 +44,7 @@ #define I40E_FW_API_VERSION_MAJOR 0x0001 #define I40E_FW_API_VERSION_MINOR_X722 0x000C -#define I40E_FW_API_VERSION_MINOR_X710 0x000E +#define I40E_FW_API_VERSION_MINOR_X710 0x000F #define I40E_FW_MINOR_VERSION(_h) ((_h)->mac.type == I40E_MAC_XL710 ? \ I40E_FW_API_VERSION_MINOR_X710 : \ diff --git a/sys/dev/ixl/if_ixl.c b/sys/dev/ixl/if_ixl.c index 7ebbed1f81c8..6e1a3e3e2229 100644 --- a/sys/dev/ixl/if_ixl.c +++ b/sys/dev/ixl/if_ixl.c @@ -444,6 +444,29 @@ ixl_admin_timer(void *arg) { struct ixl_pf *pf = (struct ixl_pf *)arg; + if (ixl_test_state(&pf->state, IXL_STATE_LINK_POLLING)) { + struct i40e_hw *hw = &pf->hw; + sbintime_t stime; + enum i40e_status_code status; + + hw->phy.get_link_info = TRUE; + status = i40e_get_link_status(hw, &pf->link_up); + if (status == I40E_SUCCESS) { + ixl_clear_state(&pf->state, IXL_STATE_LINK_POLLING); + /* OS link info is updated in the admin task */ + } else { + device_printf(pf->dev, + "%s: i40e_get_link_status status %s, aq error %s\n", + __func__, i40e_stat_str(hw, status), + i40e_aq_str(hw, hw->aq.asq_last_status)); + stime = getsbinuptime(); + if (stime - pf->link_poll_start > IXL_PF_MAX_LINK_POLL) { + device_printf(pf->dev, "Polling link status failed\n"); + ixl_clear_state(&pf->state, IXL_STATE_LINK_POLLING); + } + } + } + /* Fire off the admin task */ iflib_admin_intr_deferred(pf->vsi.ctx); @@ -706,12 +729,6 @@ ixl_if_attach_post(if_ctx_t ctx) return (0); } - /* Determine link state */ - if (ixl_attach_get_link_status(pf)) { - error = EINVAL; - goto err; - } - error = ixl_switch_config(pf); if (error) { device_printf(dev, "Initial ixl_switch_config() failed: %d\n", @@ -740,6 +757,11 @@ ixl_if_attach_post(if_ctx_t ctx) device_printf(dev, "Allocating %d queues for PF LAN VSI; %d queues active\n", pf->qtag.num_allocated, pf->qtag.num_active); + /* Determine link state */ + error = ixl_attach_get_link_status(pf); + if (error == EINVAL) + goto err; + /* Limit PHY interrupts to link, autoneg, and modules failure */ status = i40e_aq_set_phy_int_mask(hw, IXL_DEFAULT_PHY_INT_MASK, NULL); @@ -775,8 +797,20 @@ ixl_if_attach_post(if_ctx_t ctx) ixl_set_link(pf, ixl_test_state(&pf->state, IXL_STATE_LINK_ACTIVE_ON_DOWN)); hw->phy.get_link_info = true; - i40e_get_link_status(hw, &pf->link_up); - ixl_update_link_status(pf); + status = i40e_get_link_status(hw, &pf->link_up); + if (status != I40E_SUCCESS) { + device_printf(dev, + "%s get link status, status: %s aq_err=%s\n", + __func__, i40e_stat_str(hw, status), + i40e_aq_str(hw, hw->aq.asq_last_status)); + /* + * Most probably FW has not finished configuring PHY. + * Retry periodically in a timer callback. + */ + ixl_set_state(&pf->state, IXL_STATE_LINK_POLLING); + pf->link_poll_start = getsbinuptime(); + } else + ixl_update_link_status(pf); #ifdef PCI_IOV ixl_initialize_sriov(pf); diff --git a/sys/dev/ixl/ixl.h b/sys/dev/ixl/ixl.h index 30e8ce40126b..03a1fa46fa65 100644 --- a/sys/dev/ixl/ixl.h +++ b/sys/dev/ixl/ixl.h @@ -286,6 +286,8 @@ /* For stats sysctl naming */ #define IXL_QUEUE_NAME_LEN 32 +#define IXL_PF_MAX_LINK_POLL SBT_1S * 5 + MALLOC_DECLARE(M_IXL); #define IXL_DEV_ERR(_dev, _format, ...) \ diff --git a/sys/dev/ixl/ixl_debug.h b/sys/dev/ixl/ixl_debug.h index 0d8c624d2df9..818ba8a1df83 100644 --- a/sys/dev/ixl/ixl_debug.h +++ b/sys/dev/ixl/ixl_debug.h @@ -101,6 +101,8 @@ enum ixl_dbg_mask { IXL_DBG_SWITCH_INFO = 0x00010000, IXL_DBG_I2C = 0x00020000, + IXL_DBG_LINK = 0x00100000, + IXL_DBG_ALL = 0xFFFFFFFF }; diff --git a/sys/dev/ixl/ixl_pf.h b/sys/dev/ixl/ixl_pf.h index 80b35fcb6696..9a786845be13 100644 --- a/sys/dev/ixl/ixl_pf.h +++ b/sys/dev/ixl/ixl_pf.h @@ -89,6 +89,7 @@ enum ixl_state { IXL_STATE_FW_LLDP_DISABLED = 9, IXL_STATE_EEE_ENABLED = 10, IXL_STATE_LINK_ACTIVE_ON_DOWN = 11, + IXL_STATE_LINK_POLLING = 12, }; #define IXL_PF_IN_RECOVERY_MODE(pf) \ @@ -172,6 +173,8 @@ struct ixl_pf { int num_vfs; uint16_t veb_seid; int vc_debug_lvl; + + sbintime_t link_poll_start; }; /* @@ -282,6 +285,7 @@ struct ixl_pf { #define ixl_dbg_info(pf, s, ...) ixl_debug_core((pf)->dev, (pf)->dbg_mask, IXL_DBG_INFO, s, ##__VA_ARGS__) #define ixl_dbg_filter(pf, s, ...) ixl_debug_core((pf)->dev, (pf)->dbg_mask, IXL_DBG_FILTER, s, ##__VA_ARGS__) #define ixl_dbg_iov(pf, s, ...) ixl_debug_core((pf)->dev, (pf)->dbg_mask, IXL_DBG_IOV, s, ##__VA_ARGS__) +#define ixl_dbg_link(pf, s, ...) ixl_debug_core((pf)->dev, (pf)->dbg_mask, IXL_DBG_LINK, s, ##__VA_ARGS__) /* PF-only function declarations */ void ixl_set_state(volatile u32 *s, enum ixl_state bit); diff --git a/sys/dev/ixl/ixl_pf_iflib.c b/sys/dev/ixl/ixl_pf_iflib.c index e0452c7dc01e..a0b450dc038b 100644 --- a/sys/dev/ixl/ixl_pf_iflib.c +++ b/sys/dev/ixl/ixl_pf_iflib.c @@ -1028,9 +1028,7 @@ ixl_rebuild_hw_structs_after_reset(struct ixl_pf *pf, bool is_up) i40e_aq_set_vsi_broadcast(&pf->hw, vsi->seid, TRUE, NULL); /* Determine link state */ - if (ixl_attach_get_link_status(pf)) { - error = EINVAL; - } + ixl_attach_get_link_status(pf); i40e_aq_set_dcb_parameters(hw, TRUE, NULL); diff --git a/sys/dev/ixl/ixl_pf_main.c b/sys/dev/ixl/ixl_pf_main.c index 1ba59c0592a7..13ba4c5f5d17 100644 --- a/sys/dev/ixl/ixl_pf_main.c +++ b/sys/dev/ixl/ixl_pf_main.c @@ -4705,22 +4705,38 @@ ixl_attach_get_link_status(struct ixl_pf *pf) { struct i40e_hw *hw = &pf->hw; device_t dev = pf->dev; - int error = 0; + enum i40e_status_code status; if (((hw->aq.fw_maj_ver == 4) && (hw->aq.fw_min_ver < 33)) || (hw->aq.fw_maj_ver < 4)) { i40e_msec_delay(75); - error = i40e_aq_set_link_restart_an(hw, TRUE, NULL); - if (error) { - device_printf(dev, "link restart failed, aq_err=%d\n", - pf->hw.aq.asq_last_status); - return error; + status = i40e_aq_set_link_restart_an(hw, TRUE, NULL); + if (status != I40E_SUCCESS) { + device_printf(dev, + "%s link restart failed status: %s, aq_err=%s\n", + __func__, i40e_stat_str(hw, status), + i40e_aq_str(hw, hw->aq.asq_last_status)); + return (EINVAL); } } /* Determine link state */ hw->phy.get_link_info = TRUE; - i40e_get_link_status(hw, &pf->link_up); + status = i40e_get_link_status(hw, &pf->link_up); + if (status != I40E_SUCCESS) { + device_printf(dev, + "%s get link status, status: %s aq_err=%s\n", + __func__, i40e_stat_str(hw, status), + i40e_aq_str(hw, hw->aq.asq_last_status)); + /* + * Most probably FW has not finished configuring PHY. + * Retry periodically in a timer callback. + */ + ixl_set_state(&pf->state, IXL_STATE_LINK_POLLING); + pf->link_poll_start = getsbinuptime(); + return (EAGAIN); + } + ixl_dbg_link(pf, "%s link_up: %d\n", __func__, pf->link_up); /* Flow Control mode not set by user, read current FW settings */ if (pf->fc == -1)