From owner-svn-src-all@freebsd.org Mon Oct 7 10:20:24 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1B873129862; Mon, 7 Oct 2019 10:20:24 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46mxLv74Lzz3HLZ; Mon, 7 Oct 2019 10:20:23 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D29A6D1EC; Mon, 7 Oct 2019 10:20:23 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x97AKNLJ018769; Mon, 7 Oct 2019 10:20:23 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x97AKNDa018767; Mon, 7 Oct 2019 10:20:23 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201910071020.x97AKNDa018767@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 7 Oct 2019 10:20:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r353253 - in stable/12/sys/dev/mlx5: . mlx5_core X-SVN-Group: stable-12 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/12/sys/dev/mlx5: . mlx5_core X-SVN-Commit-Revision: 353253 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Oct 2019 10:20:24 -0000 Author: hselasky Date: Mon Oct 7 10:20:23 2019 New Revision: 353253 URL: https://svnweb.freebsd.org/changeset/base/353253 Log: MFC r352991 and 353000: Wait for FW readiness before initializing command interface in mlx5core. Before attempting to initialize the command interface we must wait till the fw_initializing bit is clear. If we fail to meet this condition the hardware will drop our configuration, specifically the descriptors page address. This scenario can happen when the firmware is still executing an FLR flow and did not finish yet so the driver needs to wait for that to finish. Linux commits: 6c780a0267b8 b8a92577f4be. Sponsored by: Mellanox Technologies Modified: stable/12/sys/dev/mlx5/device.h stable/12/sys/dev/mlx5/mlx5_core/mlx5_main.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/dev/mlx5/device.h ============================================================================== --- stable/12/sys/dev/mlx5/device.h Mon Oct 7 10:16:50 2019 (r353252) +++ stable/12/sys/dev/mlx5/device.h Mon Oct 7 10:20:23 2019 (r353253) @@ -32,8 +32,10 @@ #include #include -#define FW_INIT_TIMEOUT_MILI 2000 -#define FW_INIT_WAIT_MS 2 +#define FW_INIT_TIMEOUT_MILI 2000 +#define FW_INIT_WAIT_MS 2 +#define FW_PRE_INIT_TIMEOUT_MILI 120000 +#define FW_INIT_WARN_MESSAGE_INTERVAL 20000 #if defined(__LITTLE_ENDIAN) #define MLX5_SET_HOST_ENDIANNESS 0 Modified: stable/12/sys/dev/mlx5/mlx5_core/mlx5_main.c ============================================================================== --- stable/12/sys/dev/mlx5/mlx5_core/mlx5_main.c Mon Oct 7 10:16:50 2019 (r353252) +++ stable/12/sys/dev/mlx5/mlx5_core/mlx5_main.c Mon Oct 7 10:20:23 2019 (r353253) @@ -676,19 +676,33 @@ static inline int fw_initializing(struct mlx5_core_dev return ioread32be(&dev->iseg->initializing) >> 31; } -static int wait_fw_init(struct mlx5_core_dev *dev, u32 max_wait_mili) +static int wait_fw_init(struct mlx5_core_dev *dev, u32 max_wait_mili, + u32 warn_time_mili) { - u64 end = jiffies + msecs_to_jiffies(max_wait_mili); + int warn = jiffies + msecs_to_jiffies(warn_time_mili); + int end = jiffies + msecs_to_jiffies(max_wait_mili); int err = 0; - while (fw_initializing(dev)) { + MPASS(max_wait_mili > warn_time_mili); + + while (fw_initializing(dev) == 1) { if (time_after(jiffies, end)) { err = -EBUSY; break; } + if (warn_time_mili && time_after(jiffies, warn)) { + mlx5_core_warn(dev, + "Waiting for FW initialization, timeout abort in %u s\n", + (unsigned int)(jiffies_to_msecs(end - warn) / 1000)); + warn = jiffies + msecs_to_jiffies(warn_time_mili); + } msleep(FW_INIT_WAIT_MS); } + if (err != 0) + mlx5_core_dbg(dev, "Full initializing bit dword = 0x%x\n", + ioread32be(&dev->iseg->initializing)); + return err; } @@ -994,15 +1008,29 @@ static int mlx5_load_one(struct mlx5_core_dev *dev, st */ dev->state = MLX5_DEVICE_STATE_UP; + /* wait for firmware to accept initialization segments configurations + */ + err = wait_fw_init(dev, FW_PRE_INIT_TIMEOUT_MILI, + FW_INIT_WARN_MESSAGE_INTERVAL); + if (err) { + dev_err(&dev->pdev->dev, + "Firmware over %d MS in pre-initializing state, aborting\n", + FW_PRE_INIT_TIMEOUT_MILI); + goto out_err; + } + err = mlx5_cmd_init(dev); if (err) { - mlx5_core_err(dev, "Failed initializing command interface, aborting\n"); + mlx5_core_err(dev, + "Failed initializing command interface, aborting\n"); goto out_err; } - err = wait_fw_init(dev, FW_INIT_TIMEOUT_MILI); + err = wait_fw_init(dev, FW_INIT_TIMEOUT_MILI, 0); if (err) { - mlx5_core_err(dev, "Firmware over %d MS in initializing state, aborting\n", FW_INIT_TIMEOUT_MILI); + mlx5_core_err(dev, + "Firmware over %d MS in initializing state, aborting\n", + FW_INIT_TIMEOUT_MILI); goto err_cmd_cleanup; }