From owner-svn-src-head@freebsd.org Fri Mar 27 15:28:43 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5D2DD27898E; Fri, 27 Mar 2020 15:28:43 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48pm3G07zZz4K8K; Fri, 27 Mar 2020 15:28:42 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 84D5C234D4; Fri, 27 Mar 2020 15:28:28 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 02RFSSHE002071; Fri, 27 Mar 2020 15:28:28 GMT (envelope-from chuck@FreeBSD.org) Received: (from chuck@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 02RFSSkt002070; Fri, 27 Mar 2020 15:28:28 GMT (envelope-from chuck@FreeBSD.org) Message-Id: <202003271528.02RFSSkt002070@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: chuck set sender to chuck@FreeBSD.org using -f From: Chuck Tuffli Date: Fri, 27 Mar 2020 15:28:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r359367 - head/usr.sbin/bhyve X-SVN-Group: head X-SVN-Commit-Author: chuck X-SVN-Commit-Paths: head/usr.sbin/bhyve X-SVN-Commit-Revision: 359367 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2020 15:28:43 -0000 Author: chuck Date: Fri Mar 27 15:28:27 2020 New Revision: 359367 URL: https://svnweb.freebsd.org/changeset/base/359367 Log: bhyve: fix NVMe emulation update of SQHD The SQHD field of a Completion Queue entry indicates the current Submission Queue head pointer value. The head pointer represents the next entry to be consumed and is updated after consuming the current entry. In the Admin queue processing, the current code updates the head pointer after reporting the value to the host via the SQHD. This gives the impression that the Controller is perpetually one command behind in its processing of the Admin SQ. And while this doesn't appear to bother some initiators, it is wrong. Fix is to update the SQ head pointer prior to writing the SQHD value in the completion. While here, fix missed update of dword 0 (cdw0) in the completion message. Reported by: khng300 Reviewed by: jhb, imp Approved by: jhb (maintainer) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24083 Modified: head/usr.sbin/bhyve/pci_nvme.c Modified: head/usr.sbin/bhyve/pci_nvme.c ============================================================================== --- head/usr.sbin/bhyve/pci_nvme.c Fri Mar 27 15:28:22 2020 (r359366) +++ head/usr.sbin/bhyve/pci_nvme.c Fri Mar 27 15:28:27 2020 (r359367) @@ -1153,7 +1153,8 @@ pci_nvme_handle_admin_cmd(struct pci_nvme_softc* sc, u cmd->opc)); pci_nvme_status_genc(&compl.status, NVME_SC_INVALID_OPCODE); } - + sqhead = (sqhead + 1) % sq->size; + if (NVME_COMPLETION_VALID(compl)) { struct nvme_completion *cp; int phase; @@ -1170,7 +1171,6 @@ pci_nvme_handle_admin_cmd(struct pci_nvme_softc* sc, u cq->tail = (cq->tail + 1) % cq->size; } - sqhead = (sqhead + 1) % sq->size; } DPRINTF(("setting sqhead %u", sqhead)); @@ -1285,8 +1285,9 @@ pci_nvme_set_completion(struct pci_nvme_softc *sc, compl = &cq->qbase[cq->tail]; - compl->sqhd = atomic_load_acq_short(&sq->head); + compl->cdw0 = cdw0; compl->sqid = sqid; + compl->sqhd = atomic_load_acq_short(&sq->head); compl->cid = cid; // toggle phase