From nobody Mon Nov 11 16:39:40 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XnFd91XHmz5cJPf; Mon, 11 Nov 2024 16:39:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XnFd9038Sz4GTc; Mon, 11 Nov 2024 16:39:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1731343181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=v7AbXEOj9blkf8CuDQOVAJ95SToZFwLMzOqBtA5jLT8=; b=HetJNdt6sHvj6gFIp5mr0poPnQp5owA1j7LACUGMYYzAp20SQ8BBtUF1MWQgEMxKKJMgRH YOqgaxznOnCxvsUcu1Jilf6C1BB/JCig7+BwH0JuwazIImcetZR9vyq1LI9sqIAc2AV+W8 X+syDVKODpJ3m6gsUPhxWfbWVMSZKxeQc+QJ295yu+09NpeqfKyssJSQlwQya/5sx6Z1sG C/DO+q5ZMC7LCOWe4+7ZE+ag+yWB7/92O0tBfVG0DFbqND2b7jsSlIjQZ2vIBEo2FRnHau RkWz+4vM1fEhaijx5pPqtzfYHuTqI1eHLVKAzVD0BcK2UycTGH2Vo9sLqDiaoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1731343181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=v7AbXEOj9blkf8CuDQOVAJ95SToZFwLMzOqBtA5jLT8=; b=Hh0M5lNbK/wV90gy9tfT6vEM2sWq4WFHmr7MBt7wCb1XSvexATFRn4jN6MysHsUaQ29rxn Q3CaFzZUmG5haaLo/KyqE9Qdt2o4EXG5RE7MI6YfsEZRtcynm8plW1W9tQ4vDNWg9UFhpb mojcsBE47R+IjHH58Lk8IpBWk/WTTu5dATcJH0qqhonD0k8j0VSXi7FYC+rfWRRdIX83LU JCT8qeGqeFsEnaaUz6VRWdkDVgIl8GYi1t6iuRLFjt/ZBCFeP6MNyPrtkY7Kuo17N/FQKu RCExNJsyywk5BASg6ThRkD7doIfrUcy61XkWkDF2ohiw1qeiSwa8NviKfQquwA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1731343181; a=rsa-sha256; cv=none; b=qWddBct4AKCF+O1F2RZyi25X/qFKadjOg2ZuhA48+eEV7qeMcLzPeRF+byU3ra+BhfHnS3 59Le3UTcWor/Gf7IqO/X9Avw6mL09IApN2wPhJxyMHs9whIl3kMORpTNPTw7+lZZEDacCl OhGKVKSxKxPEsYVSr3w47PMHQTW1VLqLkc5lBifrgjhkPHBqOvp4eap6M+mIKS5IARaDFN l6vZs8f3V6g5NzmGMFmrSDKzpqXOeVvdcmlYuS7G1DzSVlkNaAFzOzheqdLJpIoT5e3Ehw KF+/aRxqeGf2PhNCKBbQv1wV55H9wKPooug8M2pWyXH0EuCmV1GgzAXnkHwcLQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XnFd854G9zZ4L; Mon, 11 Nov 2024 16:39:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4ABGdeKJ070494; Mon, 11 Nov 2024 16:39:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4ABGde9B070491; Mon, 11 Nov 2024 16:39:40 GMT (envelope-from git) Date: Mon, 11 Nov 2024 16:39:40 GMT Message-Id: <202411111639.4ABGde9B070491@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 4d3b659f24a8 - main - nvmf: Track SQ flow control List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4d3b659f24a88f76557019e6f3192760bdebfbdf Auto-Submitted: auto-generated The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=4d3b659f24a88f76557019e6f3192760bdebfbdf commit 4d3b659f24a88f76557019e6f3192760bdebfbdf Author: John Baldwin AuthorDate: 2024-11-11 16:39:05 +0000 Commit: John Baldwin CommitDate: 2024-11-11 16:39:05 +0000 nvmf: Track SQ flow control This isn't really needed since the host driver never submits more commands to a queue than it can hold, but I noticed that the recently-added SQ head and tail sysctl nodes were not updating. This fixes that and also uses these values to assert that there we never submit a command while a queue pair is full. Sponsored by: Chelsio Communications --- sys/dev/nvmf/host/nvmf_qpair.c | 33 ++++++++++++++++++++++++++++----- sys/dev/nvmf/nvmf_transport.c | 8 ++++++++ sys/dev/nvmf/nvmf_transport.h | 1 + 3 files changed, 37 insertions(+), 5 deletions(-) diff --git a/sys/dev/nvmf/host/nvmf_qpair.c b/sys/dev/nvmf/host/nvmf_qpair.c index 1aeb0535eacf..b03ecfa081d3 100644 --- a/sys/dev/nvmf/host/nvmf_qpair.c +++ b/sys/dev/nvmf/host/nvmf_qpair.c @@ -115,8 +115,23 @@ nvmf_dispatch_command(struct nvmf_host_qpair *qp, struct nvmf_host_command *cmd) struct nvmf_softc *sc = qp->sc; struct nvme_command *sqe; struct nvmf_capsule *nc; + uint16_t new_sqtail; int error; + mtx_assert(&qp->lock, MA_OWNED); + + qp->submitted++; + + /* + * Update flow control tracking. This is just a sanity check. + * Since num_commands == qsize - 1, there can never be too + * many commands in flight. + */ + new_sqtail = (qp->sqtail + 1) % (qp->num_commands + 1); + KASSERT(new_sqtail != qp->sqhd, ("%s: qp %p is full", __func__, qp)); + qp->sqtail = new_sqtail; + mtx_unlock(&qp->lock); + nc = cmd->req->nc; sqe = nvmf_capsule_sqe(nc); @@ -180,11 +195,23 @@ nvmf_receive_capsule(void *arg, struct nvmf_capsule *nc) return; } + /* Update flow control tracking. */ + mtx_lock(&qp->lock); + if (qp->sq_flow_control) { + if (nvmf_sqhd_valid(nc)) + qp->sqhd = le16toh(cqe->sqhd); + } else { + /* + * If SQ FC is disabled, just advance the head for + * each response capsule received. + */ + qp->sqhd = (qp->sqhd + 1) % (qp->num_commands + 1); + } + /* * If the queue has been shutdown due to an error, silently * drop the response. */ - mtx_lock(&qp->lock); if (qp->qp == NULL) { device_printf(sc->dev, "received completion for CID %u on shutdown %s\n", cid, @@ -215,8 +242,6 @@ nvmf_receive_capsule(void *arg, struct nvmf_capsule *nc) } else { cmd->req = STAILQ_FIRST(&qp->pending_requests); STAILQ_REMOVE_HEAD(&qp->pending_requests, link); - qp->submitted++; - mtx_unlock(&qp->lock); nvmf_dispatch_command(qp, cmd); } @@ -420,7 +445,5 @@ nvmf_submit_request(struct nvmf_request *req) ("%s: CID already busy", __func__)); qp->active_commands[cmd->cid] = cmd; cmd->req = req; - qp->submitted++; - mtx_unlock(&qp->lock); nvmf_dispatch_command(qp, cmd); } diff --git a/sys/dev/nvmf/nvmf_transport.c b/sys/dev/nvmf/nvmf_transport.c index ea4aee8cc7ae..316d1571e61d 100644 --- a/sys/dev/nvmf/nvmf_transport.c +++ b/sys/dev/nvmf/nvmf_transport.c @@ -180,6 +180,14 @@ nvmf_capsule_cqe(struct nvmf_capsule *nc) return (&nc->nc_cqe); } +bool +nvmf_sqhd_valid(struct nvmf_capsule *nc) +{ + KASSERT(nc->nc_qe_len == sizeof(struct nvme_completion), + ("%s: capsule %p is not a response capsule", __func__, nc)); + return (nc->nc_sqhd_valid); +} + uint8_t nvmf_validate_command_capsule(struct nvmf_capsule *nc) { diff --git a/sys/dev/nvmf/nvmf_transport.h b/sys/dev/nvmf/nvmf_transport.h index 549170b25940..bbd830eba576 100644 --- a/sys/dev/nvmf/nvmf_transport.h +++ b/sys/dev/nvmf/nvmf_transport.h @@ -78,6 +78,7 @@ int nvmf_transmit_capsule(struct nvmf_capsule *nc); void nvmf_abort_capsule_data(struct nvmf_capsule *nc, int error); void *nvmf_capsule_sqe(struct nvmf_capsule *nc); void *nvmf_capsule_cqe(struct nvmf_capsule *nc); +bool nvmf_sqhd_valid(struct nvmf_capsule *nc); /* Controller-specific APIs. */