From owner-svn-src-all@FreeBSD.ORG Tue Mar 26 18:27:23 2013 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 900163F3; Tue, 26 Mar 2013 18:27:23 +0000 (UTC) (envelope-from jimharris@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 82A2636E; Tue, 26 Mar 2013 18:27:23 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.6/8.14.6) with ESMTP id r2QIRN3j061223; Tue, 26 Mar 2013 18:27:23 GMT (envelope-from jimharris@svn.freebsd.org) Received: (from jimharris@localhost) by svn.freebsd.org (8.14.6/8.14.5/Submit) id r2QIRNoP061222; Tue, 26 Mar 2013 18:27:23 GMT (envelope-from jimharris@svn.freebsd.org) Message-Id: <201303261827.r2QIRNoP061222@svn.freebsd.org> From: Jim Harris Date: Tue, 26 Mar 2013 18:27:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r248733 - head/sys/dev/nvme X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2013 18:27:23 -0000 Author: jimharris Date: Tue Mar 26 18:27:22 2013 New Revision: 248733 URL: http://svnweb.freebsd.org/changeset/base/248733 Log: Break out the code for completing an nvme_tracker object into a separate function. This allows for completions outside the normal completion path, for example when an ABORT command fails due to the controller reporting the targeted command does not exist. This is mainly for protection against a faulty controller, but we need to clean up our internal request nonetheless. Sponsored by: Intel Modified: head/sys/dev/nvme/nvme_qpair.c Modified: head/sys/dev/nvme/nvme_qpair.c ============================================================================== --- head/sys/dev/nvme/nvme_qpair.c Tue Mar 26 18:23:35 2013 (r248732) +++ head/sys/dev/nvme/nvme_qpair.c Tue Mar 26 18:27:22 2013 (r248733) @@ -38,7 +38,14 @@ static void _nvme_qpair_submit_request(s struct nvme_request *req); static boolean_t -nvme_completion_check_retry(const struct nvme_completion *cpl) +nvme_completion_is_error(struct nvme_completion *cpl) +{ + + return (cpl->sf_sc != 0 || cpl->sf_sct != 0); +} + +static boolean_t +nvme_completion_is_retry(const struct nvme_completion *cpl) { /* * TODO: spec is not clear how commands that are aborted due @@ -96,69 +103,78 @@ nvme_qpair_construct_tracker(struct nvme tr->qpair = qpair; } -void -nvme_qpair_process_completions(struct nvme_qpair *qpair) +static void +nvme_qpair_complete_tracker(struct nvme_qpair *qpair, struct nvme_tracker *tr, + struct nvme_completion *cpl, boolean_t print_on_error) { - struct nvme_tracker *tr; struct nvme_request *req; - struct nvme_completion *cpl; boolean_t retry, error; - qpair->num_intr_handler_calls++; + req = tr->req; + error = nvme_completion_is_error(cpl); + retry = error && nvme_completion_is_retry(cpl); - while (1) { - cpl = &qpair->cpl[qpair->cq_head]; + if (error && print_on_error) { + nvme_dump_completion(cpl); + nvme_dump_command(&req->cmd); + } - if (cpl->p != qpair->phase) - break; + qpair->act_tr[cpl->cid] = NULL; - tr = qpair->act_tr[cpl->cid]; - req = tr->req; + KASSERT(cpl->cid == req->cmd.cid, ("cpl cid does not match cmd cid\n")); - KASSERT(tr, - ("completion queue has entries but no active trackers\n")); + if (req->cb_fn && !retry) + req->cb_fn(req->cb_arg, cpl); - error = cpl->sf_sc || cpl->sf_sct; - retry = error && nvme_completion_check_retry(cpl); + mtx_lock(&qpair->lock); + callout_stop(&tr->timer); - if (error) { - nvme_dump_completion(cpl); - nvme_dump_command(&tr->req->cmd); - } + if (retry) + nvme_qpair_submit_cmd(qpair, tr); + else { + if (req->payload_size > 0 || req->uio != NULL) + bus_dmamap_unload(qpair->dma_tag, + tr->payload_dma_map); + + nvme_free_request(req); - qpair->act_tr[cpl->cid] = NULL; + SLIST_INSERT_HEAD(&qpair->free_tr, tr, slist); - KASSERT(cpl->cid == req->cmd.cid, - ("cpl cid does not match cmd cid\n")); + if (!STAILQ_EMPTY(&qpair->queued_req)) { + req = STAILQ_FIRST(&qpair->queued_req); + STAILQ_REMOVE_HEAD(&qpair->queued_req, stailq); + _nvme_qpair_submit_request(qpair, req); + } + } - if (req->cb_fn && !retry) - req->cb_fn(req->cb_arg, cpl); + mtx_unlock(&qpair->lock); +} - qpair->sq_head = cpl->sqhd; +void +nvme_qpair_process_completions(struct nvme_qpair *qpair) +{ + struct nvme_tracker *tr; + struct nvme_completion *cpl; - mtx_lock(&qpair->lock); - callout_stop(&tr->timer); + qpair->num_intr_handler_calls++; - if (retry) - nvme_qpair_submit_cmd(qpair, tr); - else { - if (req->payload_size > 0 || req->uio != NULL) - bus_dmamap_unload(qpair->dma_tag, - tr->payload_dma_map); + while (1) { + cpl = &qpair->cpl[qpair->cq_head]; - nvme_free_request(req); + if (cpl->p != qpair->phase) + break; - SLIST_INSERT_HEAD(&qpair->free_tr, tr, slist); + tr = qpair->act_tr[cpl->cid]; - if (!STAILQ_EMPTY(&qpair->queued_req)) { - req = STAILQ_FIRST(&qpair->queued_req); - STAILQ_REMOVE_HEAD(&qpair->queued_req, stailq); - _nvme_qpair_submit_request(qpair, req); - } + if (tr != NULL) { + nvme_qpair_complete_tracker(qpair, tr, cpl, TRUE); + qpair->sq_head = cpl->sqhd; + } else { + printf("cpl does not map to outstanding cmd\n"); + nvme_dump_completion(cpl); + KASSERT(0, ("received completion for unknown cmd\n")); } - mtx_unlock(&qpair->lock); - if (++qpair->cq_head == qpair->num_entries) { qpair->cq_head = 0; qpair->phase = !qpair->phase;