From owner-dev-commits-src-all@freebsd.org Mon Jul 12 13:10:06 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0BAED65EF5D; Mon, 12 Jul 2021 13:10:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNkdT281wz3h0p; Mon, 12 Jul 2021 13:10:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9CB4D7563; Mon, 12 Jul 2021 13:10:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16CDA42L097837; Mon, 12 Jul 2021 13:10:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16CDA4aG097835; Mon, 12 Jul 2021 13:10:04 GMT (envelope-from git) Date: Mon, 12 Jul 2021 13:10:04 GMT Message-Id: <202107121310.16CDA4aG097835@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Hans Petter Selasky Subject: git: f60da09dbb15 - main - ibcore: Add some functions and definitions for selecting and querying retryable ucontext cleanup. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f60da09dbb152d7c8ee1719197d98149a8b0c017 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2021 13:10:06 -0000 The branch main has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=f60da09dbb152d7c8ee1719197d98149a8b0c017 commit f60da09dbb152d7c8ee1719197d98149a8b0c017 Author: Hans Petter Selasky AuthorDate: 2021-06-16 13:02:00 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-12 12:22:34 +0000 ibcore: Add some functions and definitions for selecting and querying retryable ucontext cleanup. Linux commit: 1c77483e4c50339b0306572167ccbff6b55d051b MFC after: 1 week Reviewed by: kib Sponsored by: Mellanox Technologies // NVIDIA Networking --- sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c | 1 + sys/ofed/include/rdma/ib_verbs.h | 56 ++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c b/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c index ab6e27d40d33..df3f8657755c 100644 --- a/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c +++ b/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c @@ -365,6 +365,7 @@ ssize_t ib_uverbs_get_context(struct ib_uverbs_file *file, ucontext->tgid = get_pid(task_pid_group_leader(current)); rcu_read_unlock(); ucontext->closing = 0; + ucontext->cleanup_retryable = false; #ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING ucontext->umem_tree = RB_ROOT; diff --git a/sys/ofed/include/rdma/ib_verbs.h b/sys/ofed/include/rdma/ib_verbs.h index 45a25dc06c45..723db4a659cf 100644 --- a/sys/ofed/include/rdma/ib_verbs.h +++ b/sys/ofed/include/rdma/ib_verbs.h @@ -1364,6 +1364,20 @@ struct ib_fmr_attr { struct ib_umem; +enum rdma_remove_reason { + /* + * Userspace requested uobject deletion or initial try + * to remove uobject via cleanup. Call could fail + */ + RDMA_REMOVE_DESTROY, + /* Context deletion. This call should delete the actual object itself */ + RDMA_REMOVE_CLOSE, + /* Driver is being hot-unplugged. This call should delete the actual object itself */ + RDMA_REMOVE_DRIVER_REMOVE, + /* uobj is being cleaned-up before being committed */ + RDMA_REMOVE_ABORT, +}; + struct ib_ucontext { struct ib_device *device; struct list_head pd_list; @@ -1379,6 +1393,8 @@ struct ib_ucontext { struct list_head rwq_ind_tbl_list; int closing; + bool cleanup_retryable; + pid_t tgid; #ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING struct rb_root umem_tree; @@ -2213,6 +2229,46 @@ static inline bool ib_is_udata_cleared(struct ib_udata *udata, return ret; } +/** + * ib_is_destroy_retryable - Check whether the uobject destruction + * is retryable. + * @ret: The initial destruction return code + * @why: remove reason + * @uobj: The uobject that is destroyed + * + * This function is a helper function that IB layer and low-level drivers + * can use to consider whether the destruction of the given uobject is + * retry-able. + * It checks the original return code, if it wasn't success the destruction + * is retryable according to the ucontext state (i.e. cleanup_retryable) and + * the remove reason. (i.e. why). + * Must be called with the object locked for destroy. + */ +static inline bool ib_is_destroy_retryable(int ret, enum rdma_remove_reason why, + struct ib_uobject *uobj) +{ + return ret && (why == RDMA_REMOVE_DESTROY || + uobj->context->cleanup_retryable); +} + +/** + * ib_destroy_usecnt - Called during destruction to check the usecnt + * @usecnt: The usecnt atomic + * @why: remove reason + * @uobj: The uobject that is destroyed + * + * Non-zero usecnts will block destruction unless destruction was triggered by + * a ucontext cleanup. + */ +static inline int ib_destroy_usecnt(atomic_t *usecnt, + enum rdma_remove_reason why, + struct ib_uobject *uobj) +{ + if (atomic_read(usecnt) && ib_is_destroy_retryable(-EBUSY, why, uobj)) + return -EBUSY; + return 0; +} + /** * ib_modify_qp_is_ok - Check that the supplied attribute mask * contains all required attributes and no attributes not allowed for