From owner-svn-src-all@freebsd.org Sun Jul 19 22:58:11 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D81F736C0AB; Sun, 19 Jul 2020 22:58:11 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B90dH5KQjz48VG; Sun, 19 Jul 2020 22:58:11 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9632023452; Sun, 19 Jul 2020 22:58:11 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 06JMwBTr040034; Sun, 19 Jul 2020 22:58:11 GMT (envelope-from chuck@FreeBSD.org) Received: (from chuck@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 06JMwB0h040033; Sun, 19 Jul 2020 22:58:11 GMT (envelope-from chuck@FreeBSD.org) Message-Id: <202007192258.06JMwB0h040033@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: chuck set sender to chuck@FreeBSD.org using -f From: Chuck Tuffli Date: Sun, 19 Jul 2020 22:58:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r363337 - stable/12/usr.sbin/bhyve X-SVN-Group: stable-12 X-SVN-Commit-Author: chuck X-SVN-Commit-Paths: stable/12/usr.sbin/bhyve X-SVN-Commit-Revision: 363337 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jul 2020 22:58:11 -0000 Author: chuck Date: Sun Jul 19 22:58:11 2020 New Revision: 363337 URL: https://svnweb.freebsd.org/changeset/base/363337 Log: MFC r362749 bhyve: Consolidate NVMe CQ update Modified: stable/12/usr.sbin/bhyve/pci_nvme.c Directory Properties: stable/12/ (props changed) Modified: stable/12/usr.sbin/bhyve/pci_nvme.c ============================================================================== --- stable/12/usr.sbin/bhyve/pci_nvme.c Sun Jul 19 22:55:52 2020 (r363336) +++ stable/12/usr.sbin/bhyve/pci_nvme.c Sun Jul 19 22:58:11 2020 (r363337) @@ -329,16 +329,6 @@ pci_nvme_status_genc(uint16_t *status, uint16_t code) pci_nvme_status_tc(status, NVME_SCT_GENERIC, code); } -static __inline void -pci_nvme_toggle_phase(uint16_t *status, int prev) -{ - - if (prev) - *status &= ~NVME_STATUS_P; - else - *status |= NVME_STATUS_P; -} - /* * Initialize the requested number or IO Submission and Completion Queues. * Admin queues are allocated implicitly. @@ -617,6 +607,7 @@ pci_nvme_init_controller(struct vmctx *ctx, struct pci sc->compl_queues[0].size = acqs; sc->compl_queues[0].qbase = vm_map_gpa(ctx, sc->regs.acq, sizeof(struct nvme_completion) * acqs); + DPRINTF("%s mapping Admin-CQ guest 0x%lx, host: %p", __func__, sc->regs.acq, sc->compl_queues[0].qbase); } @@ -668,6 +659,45 @@ nvme_prp_memcpy(struct vmctx *ctx, uint64_t prp1, uint return (0); } +/* + * Write a Completion Queue Entry update + * + * Write the completion and update the doorbell value + */ +static void +pci_nvme_cq_update(struct pci_nvme_softc *sc, + struct nvme_completion_queue *cq, + uint32_t cdw0, + uint16_t cid, + uint16_t sqid, + uint16_t status) +{ + struct nvme_submission_queue *sq = &sc->submit_queues[sqid]; + struct nvme_completion *cqe; + + assert(cq->qbase != NULL); + + pthread_mutex_lock(&cq->mtx); + + cqe = &cq->qbase[cq->tail]; + + /* Flip the phase bit */ + status |= (cqe->status ^ NVME_STATUS_P) & NVME_STATUS_P_MASK; + + cqe->cdw0 = cdw0; + cqe->sqhd = sq->head; + cqe->sqid = sqid; + cqe->cid = cid; + cqe->status = status; + + cq->tail++; + if (cq->tail >= cq->size) { + cq->tail = 0; + } + + pthread_mutex_unlock(&cq->mtx); +} + static int nvme_opc_delete_io_sq(struct pci_nvme_softc* sc, struct nvme_command* command, struct nvme_completion* compl) @@ -757,6 +787,7 @@ static int nvme_opc_create_io_cq(struct pci_nvme_softc* sc, struct nvme_command* command, struct nvme_completion* compl) { + if (command->cdw11 & NVME_CMD_CDW11_PC) { uint16_t qid = command->cdw10 & 0xffff; struct nvme_completion_queue *ncq; @@ -1190,24 +1221,11 @@ pci_nvme_handle_admin_cmd(struct pci_nvme_softc* sc, u sqhead = (sqhead + 1) % sq->size; if (NVME_COMPLETION_VALID(compl)) { - struct nvme_completion *cp; - int phase; - - pthread_mutex_lock(&cq->mtx); - - cp = &(cq->qbase)[cq->tail]; - cp->cdw0 = compl.cdw0; - cp->sqid = 0; - cp->sqhd = sqhead; - cp->cid = cmd->cid; - - phase = NVME_STATUS_GET_P(cp->status); - cp->status = compl.status; - pci_nvme_toggle_phase(&cp->status, phase); - - cq->tail = (cq->tail + 1) % cq->size; - - pthread_mutex_unlock(&cq->mtx); + pci_nvme_cq_update(sc, &sc->compl_queues[0], + compl.cdw0, + cmd->cid, + 0, /* SQID */ + compl.status); } } @@ -1310,32 +1328,16 @@ pci_nvme_set_completion(struct pci_nvme_softc *sc, uint32_t cdw0, uint16_t status) { struct nvme_completion_queue *cq = &sc->compl_queues[sq->cqid]; - struct nvme_completion *compl; - int phase; DPRINTF("%s sqid %d cqid %u cid %u status: 0x%x 0x%x", __func__, sqid, sq->cqid, cid, NVME_STATUS_GET_SCT(status), NVME_STATUS_GET_SC(status)); - pthread_mutex_lock(&cq->mtx); - - assert(cq->qbase != NULL); - - compl = &cq->qbase[cq->tail]; - - compl->cdw0 = cdw0; - compl->sqid = sqid; - compl->sqhd = sq->head; - compl->cid = cid; - - // toggle phase - phase = NVME_STATUS_GET_P(compl->status); - compl->status = status; - pci_nvme_toggle_phase(&compl->status, phase); - - cq->tail = (cq->tail + 1) % cq->size; - - pthread_mutex_unlock(&cq->mtx); + pci_nvme_cq_update(sc, cq, + 0, /* CDW0 */ + cid, + sqid, + status); if (cq->head != cq->tail) { if (cq->intr_en & NVME_CQ_INTEN) {