From nobody Fri Sep 22 21:57:30 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RsmMt6k8Vz4tVB8; Fri, 22 Sep 2023 21:57:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RsmMt5gVwz3TCx; Fri, 22 Sep 2023 21:57:30 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695419850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sLQwbJIjMi4Bt0BILyL9dAthqbZc5/98R9Jc6fEgCTE=; b=rbqCeedr2uBlTWESbhsdjW6DGdr/adXwyg97QvLNMDFQ5L+1Nc0tDi9BjKf2nzZ4WwjA5+ AGx3EQ31/SokPSEAH3mHjXzuOLD0LOa2PO68UPiU3fp9KHSHEXPkXnapHGKqmfMICp4221 2TGT2UH8fh3c/1Y5HQ4EA5NrKYckn6Vw2HixKfyVYbxWdQNUfdXMPbUe6WUXe49cg6r1Nf Yl9Q4ivMqd3h0MQrxM74a/VwpQTFVTOdOYpEFimL7gWpSSwe8jpSUH7EnlYhWImsyL23kX GwFXWwS8D9yIveM1gcg4nHIC6KIXsZXEDDWtK6rVwNV6FC3hxz5dcf8dK2SQMw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1695419850; a=rsa-sha256; cv=none; b=tbmVsvZOtBY8+er78Ns/RzVGwum6C+gI9bPk/+UoAC7BrXKRZI+PP8txMl0khe39hpebKX JWmNGYkdUdmKSDqA8S5LI3BO4YnicRRxXa2jjrrmf/c/rQayZJ/OAxHrezNmb1CZnxOHJ2 2mL5pQkmUxinXOS5KTM+dr0la5/Ug456o9PpgFThEePySIyC1V4lmPgwq3PdIDHf9V0V4p NesQVKAOL4800Al+nvrmmFC5yhNR/LVL+KmGbvf7hhqEs2rbspyJSRT2CIOoFtfyXXAsH/ werdK5wIg6nKm7lBVFLEFVOyaYIZRNBGUZGzUTO9jo3GX+Loav6GFjeOTS89Fw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695419850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sLQwbJIjMi4Bt0BILyL9dAthqbZc5/98R9Jc6fEgCTE=; b=TD9lKmKL7KdioINsahbT/N7vpfDtYeNaEDRnqm3VTAVPL0N5HuecZwOC6LkQiiG7AoYL8x yrTaCSuUadFBu73g96lChgJuAluwYD/HDh2EdT8pQqWErJzOnsKmVHOEkrMaQYSucl5aUi lbzo7NFmI3lepiqU3BpGqDvMdxLGOgt5MWfCjgB8zEgV8EGHdz3SYInMBAL3ndNRX/MmTn X5MhyG4y/2f0YYzrlCv11mAIeOV6d4lMYJq0P6KPc8OgFu5JgBUk8UxLqXQXr2xNk/10W3 N2oNFCGwhuRnvkrzvc1m6XnkJHWnA3WgHKvDdF4fu3/XQdgkgGQ8Wfz5P2lsqQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RsmMt4NdBzdxN; Fri, 22 Sep 2023 21:57:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 38MLvUCg047895; Fri, 22 Sep 2023 21:57:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 38MLvUgH047892; Fri, 22 Sep 2023 21:57:30 GMT (envelope-from git) Date: Fri, 22 Sep 2023 21:57:30 GMT Message-Id: <202309222157.38MLvUgH047892@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Eric Joyner Subject: git: 8ac750e095a3 - stable/13 - irdma(4): remove artificial completion generator List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: erj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 8ac750e095a3c9a5bd45b92d5a0b5aeb2572348d Auto-Submitted: auto-generated The branch stable/13 has been updated by erj: URL: https://cgit.FreeBSD.org/src/commit/?id=8ac750e095a3c9a5bd45b92d5a0b5aeb2572348d commit 8ac750e095a3c9a5bd45b92d5a0b5aeb2572348d Author: Bartosz Sobczak AuthorDate: 2023-08-21 11:28:58 +0000 Commit: Eric Joyner CommitDate: 2023-09-22 21:55:18 +0000 irdma(4): remove artificial completion generator Removing artificial completion generator as there had been no indication of the code being required for E810 cards. Further more it was found that the code may have unpleasant side effects on user experience when using ucmatose tool. Signed-off-by: Bartosz Sobczak Signed-off-by: Eric Joyner Reviewed by: erj@ Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D41593 (cherry picked from commit ffafa6a4d157e49e6b12567958c4ab0c9151c080) --- contrib/ofed/libirdma/irdma_umain.h | 8 -- contrib/ofed/libirdma/irdma_uverbs.c | 161 +---------------------------------- 2 files changed, 1 insertion(+), 168 deletions(-) diff --git a/contrib/ofed/libirdma/irdma_umain.h b/contrib/ofed/libirdma/irdma_umain.h index 312310dcb754..1fa18dd7a847 100644 --- a/contrib/ofed/libirdma/irdma_umain.h +++ b/contrib/ofed/libirdma/irdma_umain.h @@ -108,11 +108,6 @@ struct verbs_cq { }; }; -struct irdma_cmpl_gen { - LIST_ENTRY(irdma_cmpl_gen) list; - struct irdma_cq_poll_info cpi; -}; - struct irdma_ucq { struct verbs_cq verbs_cq; struct verbs_mr vmr; @@ -129,7 +124,6 @@ struct irdma_ucq { struct list_head resize_list; /* for extended CQ completion fields */ struct irdma_cq_poll_info cur_cqe; - struct list_head_cmpl cmpl_generated; }; struct irdma_uqp { @@ -149,7 +143,6 @@ struct irdma_uqp { struct irdma_qp_uk qp; enum ibv_qp_type qp_type; struct irdma_sge *recv_sges; - pthread_t flush_thread; }; /* irdma_uverbs.c */ @@ -206,5 +199,4 @@ void irdma_async_event(struct ibv_context *context, void irdma_set_hw_attrs(struct irdma_hw_attrs *attrs); void *irdma_mmap(int fd, off_t offset); void irdma_munmap(void *map); -void *irdma_flush_thread(void *arg); #endif /* IRDMA_UMAIN_H */ diff --git a/contrib/ofed/libirdma/irdma_uverbs.c b/contrib/ofed/libirdma/irdma_uverbs.c index 50fd2f23476d..962af399ed31 100644 --- a/contrib/ofed/libirdma/irdma_uverbs.c +++ b/contrib/ofed/libirdma/irdma_uverbs.c @@ -458,7 +458,6 @@ ucreate_cq(struct ibv_context *context, info.cq_size = get_cq_size(attr_ex->cqe, hw_rev, cqe_64byte_ena); iwucq->comp_vector = attr_ex->comp_vector; LIST_INIT(&iwucq->resize_list); - LIST_INIT(&iwucq->cmpl_generated); total_size = get_cq_total_bytes(info.cq_size, cqe_64byte_ena); cq_pages = total_size >> IRDMA_HW_PAGE_SHIFT; @@ -614,142 +613,6 @@ irdma_process_resize_list(struct irdma_ucq *iwucq, return cq_cnt; } -static void -irdma_remove_cmpls_list(struct irdma_ucq *iwucq) -{ - struct irdma_cmpl_gen *cmpl_node, *next; - - LIST_FOREACH_SAFE(cmpl_node, &iwucq->cmpl_generated, list, next) { - LIST_REMOVE(cmpl_node, list); - free(cmpl_node); - } -} - -static int -irdma_generated_cmpls(struct irdma_ucq *iwucq, struct irdma_cq_poll_info *cq_poll_info) -{ - struct irdma_cmpl_gen *cmpl; - - if (!iwucq || LIST_EMPTY(&iwucq->cmpl_generated)) - return ENOENT; - cmpl = LIST_FIRST(&iwucq->cmpl_generated); - LIST_REMOVE(cmpl, list); - memcpy(cq_poll_info, &cmpl->cpi, sizeof(*cq_poll_info)); - - free(cmpl); - - return 0; -} - -/** - * irdma_set_cpi_common_values - fill in values for polling info struct - * @cpi: resulting structure of cq_poll_info type - * @qp: QPair - * @qp_num: id of the QP - */ -static void -irdma_set_cpi_common_values(struct irdma_cq_poll_info *cpi, - struct irdma_qp_uk *qp, __u32 qp_num) -{ - cpi->comp_status = IRDMA_COMPL_STATUS_FLUSHED; - cpi->error = 1; - cpi->major_err = IRDMA_FLUSH_MAJOR_ERR; - cpi->minor_err = FLUSH_GENERAL_ERR; - cpi->qp_handle = (irdma_qp_handle) (uintptr_t)qp; - cpi->qp_id = qp_num; -} - -static bool -irdma_cq_empty(struct irdma_ucq *iwucq) -{ - struct irdma_cq_uk *ukcq; - __u64 qword3; - __le64 *cqe; - __u8 polarity; - - ukcq = &iwucq->cq; - cqe = IRDMA_GET_CURRENT_CQ_ELEM(ukcq); - get_64bit_val(cqe, 24, &qword3); - polarity = (__u8) FIELD_GET(IRDMA_CQ_VALID, qword3); - - return polarity != ukcq->polarity; -} - -/** - * irdma_generate_flush_completions - generate completion from WRs - * @iwuqp: pointer to QP - */ -static void -irdma_generate_flush_completions(struct irdma_uqp *iwuqp) -{ - struct irdma_qp_uk *qp = &iwuqp->qp; - struct irdma_ring *sq_ring = &qp->sq_ring; - struct irdma_ring *rq_ring = &qp->rq_ring; - struct irdma_cmpl_gen *cmpl; - __le64 *sw_wqe; - __u64 wqe_qword; - __u32 wqe_idx; - - if (pthread_spin_lock(&iwuqp->send_cq->lock)) - return; - if (irdma_cq_empty(iwuqp->send_cq)) { - while (IRDMA_RING_MORE_WORK(*sq_ring)) { - cmpl = malloc(sizeof(*cmpl)); - if (!cmpl) { - pthread_spin_unlock(&iwuqp->send_cq->lock); - return; - } - - wqe_idx = sq_ring->tail; - irdma_set_cpi_common_values(&cmpl->cpi, qp, qp->qp_id); - cmpl->cpi.wr_id = qp->sq_wrtrk_array[wqe_idx].wrid; - sw_wqe = qp->sq_base[wqe_idx].elem; - get_64bit_val(sw_wqe, 24, &wqe_qword); - cmpl->cpi.op_type = (__u8) FIELD_GET(IRDMAQPSQ_OPCODE, wqe_qword); - /* remove the SQ WR by moving SQ tail */ - IRDMA_RING_SET_TAIL(*sq_ring, sq_ring->tail + qp->sq_wrtrk_array[sq_ring->tail].quanta); - LIST_INSERT_HEAD(&iwuqp->send_cq->cmpl_generated, cmpl, list); - } - } - pthread_spin_unlock(&iwuqp->send_cq->lock); - if (pthread_spin_lock(&iwuqp->recv_cq->lock)) - return; - if (irdma_cq_empty(iwuqp->recv_cq)) { - while (IRDMA_RING_MORE_WORK(*rq_ring)) { - cmpl = malloc(sizeof(*cmpl)); - if (!cmpl) { - pthread_spin_unlock(&iwuqp->recv_cq->lock); - return; - } - - wqe_idx = rq_ring->tail; - irdma_set_cpi_common_values(&cmpl->cpi, qp, qp->qp_id); - cmpl->cpi.wr_id = qp->rq_wrid_array[wqe_idx]; - cmpl->cpi.op_type = IRDMA_OP_TYPE_REC; - /* remove the RQ WR by moving RQ tail */ - IRDMA_RING_SET_TAIL(*rq_ring, rq_ring->tail + 1); - LIST_INSERT_HEAD(&iwuqp->recv_cq->cmpl_generated, cmpl, list); - } - } - pthread_spin_unlock(&iwuqp->recv_cq->lock); -} - -void * -irdma_flush_thread(void *arg) -{ - __u8 i = 5; - struct irdma_uqp *iwuqp = arg; - - while (--i) { - if (pthread_spin_lock(&iwuqp->lock)) - break; - irdma_generate_flush_completions(arg); - pthread_spin_unlock(&iwuqp->lock); - sleep(1); - } - pthread_exit(NULL); -} - /** * irdma_udestroy_cq - destroys cq * @cq: ptr to cq to be destroyed @@ -770,8 +633,6 @@ irdma_udestroy_cq(struct ibv_cq *cq) if (ret) goto err; - if (!LIST_EMPTY(&iwucq->cmpl_generated)) - irdma_remove_cmpls_list(iwucq); irdma_process_resize_list(iwucq, NULL); ret = ibv_cmd_destroy_cq(cq); if (ret) @@ -1015,15 +876,6 @@ __irdma_upoll_cq(struct irdma_ucq *iwucq, int num_entries, while (npolled < num_entries) { ret = irdma_poll_one(&iwucq->cq, cur_cqe, entry ? entry + npolled : NULL); - if (ret == ENOENT) { - ret = irdma_generated_cmpls(iwucq, cur_cqe); - if (!ret) { - if (entry) - irdma_process_cqe(entry + npolled, cur_cqe); - else - irdma_process_cqe_ext(cur_cqe); - } - } if (!ret) { ++npolled; cq_new_cqe = true; @@ -1709,14 +1561,7 @@ irdma_umodify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mask) return ret; } else { - int ret; - - ret = ibv_cmd_modify_qp(qp, attr, attr_mask, &cmd, sizeof(cmd)); - if (ret) - return ret; - if (attr_mask & IBV_QP_STATE && attr->qp_state == IBV_QPS_ERR) - pthread_create(&iwuqp->flush_thread, NULL, irdma_flush_thread, iwuqp); - return 0; + return ibv_cmd_modify_qp(qp, attr, attr_mask, &cmd, sizeof(cmd)); } } @@ -1769,10 +1614,6 @@ irdma_udestroy_qp(struct ibv_qp *qp) int ret; iwuqp = container_of(qp, struct irdma_uqp, ibv_qp); - if (iwuqp->flush_thread) { - pthread_cancel(iwuqp->flush_thread); - pthread_join(iwuqp->flush_thread, NULL); - } ret = pthread_spin_destroy(&iwuqp->lock); if (ret) goto err;