From nobody Wed Apr 30 17:21:37 2025 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZnkW53v12z5tj1m; Wed, 30 Apr 2025 17:21:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZnkW51Ps8z3MBB; Wed, 30 Apr 2025 17:21:37 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1746033697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nZtdDjJ28aM7IlKbO1zIPgc17yxtzbH5LUfGgM1hfDE=; b=DoQX9UMMi9xcKpChuy5Zcxwf5V4J8MvopDw3b5BVV7puVeZnhyMq0/z+f7QO8JgCwI7pcN 93R5bsQcSkBBrtuInosCE6PdEGjVgfhhOlX+g4XnP/aS6yoNniHSwl0A2oyhsYJgeB4Wum em+BzEAqIwpkiAEXzCIY+EE+HTuX5K4F7eTqe9o8UdpAXN0G/HH+fVphQH+RvO1ajmY8P0 CEVUTpLF1PsRHkRXWA8gno5m2BA9Sx1jgJvruw14HEPSB7oaOOUTC2aKt958kqHnHLuxnx SsXTFUslX3C9Pmosg82GN1bEu5/1MgQIRSZtObroJTBPRBdttI6Qj1dtxIqMWQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1746033697; a=rsa-sha256; cv=none; b=mtxD1dcmKWSdpa3isUXYq92AlvgfmhB5FrNpFaER89dZZiwSPx1KDX+Y5p4pqV2G2mH6NH Z0mM26UOtjdOUI+84i12GAEY6hG2SMxq+aKUaHyOwp/yg6YQjVLHYlNZrGDUes3+tZrpkJ cjvXfzPqmo/WpuQyp1NUE0xcPcWE/AFwLgMRrZFuSvMAOxjbaLR05ulx8dLehmWkLSRsLF VK9BOQMu2Pd9DFMzlBQjlZrJJ4WmlCsxUJVcwDVKijnTaNEzHwukTOxROe0Pss3QdRQQFA IrFEE2vpXEEg62aXL97e61uakZx2UsIrg0B/BdrXTAM/MZf8kUk7ehvsr9fV5g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1746033697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nZtdDjJ28aM7IlKbO1zIPgc17yxtzbH5LUfGgM1hfDE=; b=TQMWueRbnQzGrhwbqI2nkgKDyq0IpizjERwvhr/DBYzwZyTAZv9wm+tEWfPuO29dDFqh0E OdAQAPiHid19bpUQF8H4iH0xCxbYvvx0aytdyAB343thhta8fRYfMM3yiVoEmD1LtBBq8v 0RabTxMjv1e9LHtzf3WQF4lP7Ot3eGxAztjAHogamhM9ZsXvT+J7riVvl2+W5IiQih+KbH TSsv9KDjDLlaFiwhLkOqsQASDUWT65n8H6L3zfqN0VyCGvLd/WyNWQvLz0X31MlE7s8bwt OJESX1pUksKkmogte3szsmmqREJJzjc6Uw8TngohhTdu33Ais9J0U+tEsYKdhQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZnkW510Fpzd7; Wed, 30 Apr 2025 17:21:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 53UHLbes093540; Wed, 30 Apr 2025 17:21:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 53UHLbXO093537; Wed, 30 Apr 2025 17:21:37 GMT (envelope-from git) Date: Wed, 30 Apr 2025 17:21:37 GMT Message-Id: <202504301721.53UHLbXO093537@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 3e1b3b3d15cd - stable/14 - mpi3mr: poll reply queue and add MPI3MR_DEV_REMOVE_HS_COMPLETED flag List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 3e1b3b3d15cd40337f2842b4483a66513f988e49 Auto-Submitted: auto-generated The branch stable/14 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=3e1b3b3d15cd40337f2842b4483a66513f988e49 commit 3e1b3b3d15cd40337f2842b4483a66513f988e49 Author: Chandrakanth patil AuthorDate: 2024-03-14 17:47:06 +0000 Commit: Warner Losh CommitDate: 2025-04-30 17:05:51 +0000 mpi3mr: poll reply queue and add MPI3MR_DEV_REMOVE_HS_COMPLETED flag An outstanding IO counter per target check has been added before deleting the target from the OS which will poll the reply queue if there are any outstanding IOs are found. A new flag, named "MPI3MR_DEV_REMOVE_HS_COMPLETED," is added. If a remove event for a target occurs and before the deletion of the target resource if the add event for another target arrives reusing the same target ID then this flag will prevent the removal of the target reference. This flag ensures synchronization between the interrupt top and bottom half during target removal and addition events. Reviewed by: imp Approved by: imp Differential revision: https://reviews.freebsd.org/D44423 (cherry picked from commit 701d776c98e015160c2d0e3f4fdaf980d1d5dc83) --- sys/dev/mpi3mr/mpi3mr.c | 19 ++++++++----------- sys/dev/mpi3mr/mpi3mr.h | 3 +-- sys/dev/mpi3mr/mpi3mr_cam.c | 27 ++++++++++++++++++--------- 3 files changed, 27 insertions(+), 22 deletions(-) diff --git a/sys/dev/mpi3mr/mpi3mr.c b/sys/dev/mpi3mr/mpi3mr.c index fa7623b56285..a5f4215f29ca 100644 --- a/sys/dev/mpi3mr/mpi3mr.c +++ b/sys/dev/mpi3mr/mpi3mr.c @@ -3478,6 +3478,7 @@ static void mpi3mr_dev_rmhs_complete_iou(struct mpi3mr_softc *sc, { U16 cmd_idx = drv_cmd->host_tag - MPI3MR_HOSTTAG_DEVRMCMD_MIN; struct delayed_dev_rmhs_node *delayed_dev_rmhs = NULL; + struct mpi3mr_target *tgtdev = NULL; mpi3mr_dprint(sc, MPI3MR_EVENT, "%s :dev_rmhs_iouctrl_complete:handle(0x%04x), ioc_status(0x%04x), loginfo(0x%08x)\n", @@ -3498,6 +3499,13 @@ static void mpi3mr_dev_rmhs_complete_iou(struct mpi3mr_softc *sc, "%s :dev removal handshake failed after all retries: handle(0x%04x)\n", __func__, drv_cmd->dev_handle); } else { + mtx_lock_spin(&sc->target_lock); + TAILQ_FOREACH(tgtdev, &sc->cam_sc->tgt_list, tgt_next) { + if (tgtdev->dev_handle == drv_cmd->dev_handle) + tgtdev->state = MPI3MR_DEV_REMOVE_HS_COMPLETED; + } + mtx_unlock_spin(&sc->target_lock); + mpi3mr_dprint(sc, MPI3MR_INFO, "%s :dev removal handshake completed successfully: handle(0x%04x)\n", __func__, drv_cmd->dev_handle); @@ -3605,18 +3613,7 @@ static void mpi3mr_dev_rmhs_send_tm(struct mpi3mr_softc *sc, U16 handle, U8 retrycount = 5; struct mpi3mr_drvr_cmd *drv_cmd = cmdparam; struct delayed_dev_rmhs_node *delayed_dev_rmhs = NULL; - struct mpi3mr_target *tgtdev = NULL; - mtx_lock_spin(&sc->target_lock); - TAILQ_FOREACH(tgtdev, &sc->cam_sc->tgt_list, tgt_next) { - if ((tgtdev->dev_handle == handle) && - (iou_rc == MPI3_CTRL_OP_REMOVE_DEVICE)) { - tgtdev->state = MPI3MR_DEV_REMOVE_HS_STARTED; - break; - } - } - mtx_unlock_spin(&sc->target_lock); - if (drv_cmd) goto issue_cmd; do { diff --git a/sys/dev/mpi3mr/mpi3mr.h b/sys/dev/mpi3mr/mpi3mr.h index ca1f42a4a7e5..7dc1b3ba3fc2 100644 --- a/sys/dev/mpi3mr/mpi3mr.h +++ b/sys/dev/mpi3mr/mpi3mr.h @@ -447,8 +447,7 @@ enum mpi3mr_cmd_state { enum mpi3mr_target_state { MPI3MR_DEV_CREATED = 1, - MPI3MR_DEV_REMOVE_HS_STARTED = 2, - MPI3MR_DEV_DELETED = 3, + MPI3MR_DEV_REMOVE_HS_COMPLETED = 2, }; struct mpi3mr_cmd { diff --git a/sys/dev/mpi3mr/mpi3mr_cam.c b/sys/dev/mpi3mr/mpi3mr_cam.c index ed47d69832ba..6d9869e74235 100644 --- a/sys/dev/mpi3mr/mpi3mr_cam.c +++ b/sys/dev/mpi3mr/mpi3mr_cam.c @@ -1823,6 +1823,17 @@ int mpi3mr_remove_device_from_os(struct mpi3mr_softc *sc, U16 handle) target->flags |= MPI3MRSAS_TARGET_INREMOVAL; + if (mpi3mr_atomic_read(&target->outstanding)) { + mpi3mr_dprint(sc, MPI3MR_ERROR, "there are [%2d] outstanding IOs on target: %d" + "Poll reply queue once\n", mpi3mr_atomic_read(&target->outstanding), + target->per_id); + mpi3mr_poll_pend_io_completions(sc); + if (mpi3mr_atomic_read(&target->outstanding)) + mpi3mr_dprint(sc, MPI3MR_ERROR, "[%2d] outstanding IOs present on target: %d" + "despite poll\n", mpi3mr_atomic_read(&target->outstanding), + target->per_id); + } + while (mpi3mr_atomic_read(&target->outstanding) && (i < 30)) { i++; if (!(i % 2)) { @@ -1849,18 +1860,16 @@ out: void mpi3mr_remove_device_from_list(struct mpi3mr_softc *sc, struct mpi3mr_target *target, bool must_delete) { + if ((must_delete == false) && + (target->state != MPI3MR_DEV_REMOVE_HS_COMPLETED)) + return; + mtx_lock_spin(&sc->target_lock); - if ((target->state == MPI3MR_DEV_REMOVE_HS_STARTED) || - (must_delete == true)) { - TAILQ_REMOVE(&sc->cam_sc->tgt_list, target, tgt_next); - target->state = MPI3MR_DEV_DELETED; - } + TAILQ_REMOVE(&sc->cam_sc->tgt_list, target, tgt_next); mtx_unlock_spin(&sc->target_lock); - if (target->state == MPI3MR_DEV_DELETED) { - free(target, M_MPI3MR); - target = NULL; - } + free(target, M_MPI3MR); + target = NULL; return; }