From owner-svn-src-all@freebsd.org Tue Nov 10 13:31:02 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49FAE43085C; Tue, 10 Nov 2020 13:31:02 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CVpfG1Cwrz3vkj; Tue, 10 Nov 2020 13:31:02 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1CF8B2326A; Tue, 10 Nov 2020 13:31:02 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AADV1oD047392; Tue, 10 Nov 2020 13:31:01 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0AADV1oA047391; Tue, 10 Nov 2020 13:31:01 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <202011101331.0AADV1oA047391@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Tue, 10 Nov 2020 13:31:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r367563 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 367563 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2020 13:31:02 -0000 Author: hselasky Date: Tue Nov 10 13:31:01 2020 New Revision: 367563 URL: https://svnweb.freebsd.org/changeset/base/367563 Log: MFC r367269: Fix for referencing file via its vnode in ibore. Use the native vnode lookup functions, instead of going via the LinuxKPI, because the file referenced is typically created outside the LinuxKPI, and the LinuxKPI's fdget() can only resolve file descriptor numbers which were created by itself. The vnode pointer is used as an identifier to identify XRCD handles which are sharing resources. This patch fixes the so-called XRCD support in ibcore for FreeBSD. Refer to ibv_open_xrcd(3) for more information how the file descriptor argument is used. Reviewed by: kib@ Sponsored by: Mellanox Technologies // NVIDIA Networking Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Tue Nov 10 13:22:52 2020 (r367562) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Tue Nov 10 13:31:01 2020 (r367563) @@ -666,11 +666,11 @@ err_put: struct xrcd_table_entry { struct rb_node node; struct ib_xrcd *xrcd; - struct inode *inode; + struct vnode *vnode; }; static int xrcd_table_insert(struct ib_uverbs_device *dev, - struct inode *inode, + struct vnode *vnode, struct ib_xrcd *xrcd) { struct xrcd_table_entry *entry, *scan; @@ -682,15 +682,15 @@ static int xrcd_table_insert(struct ib_uverbs_device * return -ENOMEM; entry->xrcd = xrcd; - entry->inode = inode; + entry->vnode = vnode; while (*p) { parent = *p; scan = rb_entry(parent, struct xrcd_table_entry, node); - if (inode < scan->inode) { + if ((uintptr_t)vnode < (uintptr_t)scan->vnode) { p = &(*p)->rb_left; - } else if (inode > scan->inode) { + } else if ((uintptr_t)vnode > (uintptr_t)scan->vnode) { p = &(*p)->rb_right; } else { kfree(entry); @@ -700,12 +700,12 @@ static int xrcd_table_insert(struct ib_uverbs_device * rb_link_node(&entry->node, parent, p); rb_insert_color(&entry->node, &dev->xrcd_tree); - igrab(inode); + vrefact(vnode); return 0; } static struct xrcd_table_entry *xrcd_table_search(struct ib_uverbs_device *dev, - struct inode *inode) + struct vnode *vnode) { struct xrcd_table_entry *entry; struct rb_node *p = dev->xrcd_tree.rb_node; @@ -713,9 +713,9 @@ static struct xrcd_table_entry *xrcd_table_search(stru while (p) { entry = rb_entry(p, struct xrcd_table_entry, node); - if (inode < entry->inode) + if ((uintptr_t)vnode < (uintptr_t)entry->vnode) p = p->rb_left; - else if (inode > entry->inode) + else if ((uintptr_t)vnode > (uintptr_t)entry->vnode) p = p->rb_right; else return entry; @@ -724,11 +724,11 @@ static struct xrcd_table_entry *xrcd_table_search(stru return NULL; } -static struct ib_xrcd *find_xrcd(struct ib_uverbs_device *dev, struct inode *inode) +static struct ib_xrcd *find_xrcd(struct ib_uverbs_device *dev, struct vnode *vnode) { struct xrcd_table_entry *entry; - entry = xrcd_table_search(dev, inode); + entry = xrcd_table_search(dev, vnode); if (!entry) return NULL; @@ -736,13 +736,13 @@ static struct ib_xrcd *find_xrcd(struct ib_uverbs_devi } static void xrcd_table_delete(struct ib_uverbs_device *dev, - struct inode *inode) + struct vnode *vnode) { struct xrcd_table_entry *entry; - entry = xrcd_table_search(dev, inode); + entry = xrcd_table_search(dev, vnode); if (entry) { - iput(inode); + vrele(vnode); rb_erase(&entry->node, &dev->xrcd_tree); kfree(entry); } @@ -758,8 +758,7 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil struct ib_udata udata; struct ib_uxrcd_object *obj; struct ib_xrcd *xrcd = NULL; - struct fd f = {NULL}; - struct inode *inode = NULL; + struct vnode *vnode = NULL; int ret = 0; int new_xrcd = 0; @@ -776,15 +775,15 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil mutex_lock(&file->device->xrcd_tree_mutex); if (cmd.fd != -1) { + cap_rights_t cap_no_rights; + cap_rights_init(&cap_no_rights); + /* search for file descriptor */ - f = fdget(cmd.fd); - if (!f.file) { - ret = -EBADF; + ret = -fgetvp(curthread, cmd.fd, &cap_no_rights, &vnode); + if (ret != 0) goto err_tree_mutex_unlock; - } - inode = f.file->f_dentry->d_inode; - xrcd = find_xrcd(file->device, inode); + xrcd = find_xrcd(file->device, vnode); if (!xrcd && !(cmd.oflags & O_CREAT)) { /* no file descriptor. Need CREATE flag */ ret = -EAGAIN; @@ -814,7 +813,7 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil goto err; } - xrcd->inode = inode; + xrcd->vnode = vnode; xrcd->device = ib_dev; atomic_set(&xrcd->usecnt, 0); mutex_init(&xrcd->tgt_qp_mutex); @@ -831,10 +830,10 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil memset(&resp, 0, sizeof resp); resp.xrcd_handle = obj->uobject.id; - if (inode) { + if (vnode != NULL) { if (new_xrcd) { - /* create new inode/xrcd table entry */ - ret = xrcd_table_insert(file->device, inode, xrcd); + /* create new vnode/xrcd table entry */ + ret = xrcd_table_insert(file->device, vnode, xrcd); if (ret) goto err_insert_xrcd; } @@ -847,8 +846,8 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil goto err_copy; } - if (f.file) - fdput(f); + if (vnode != NULL) + vrele(vnode); mutex_lock(&file->mutex); list_add_tail(&obj->uobject.list, &file->ucontext->xrcd_list); @@ -861,9 +860,9 @@ ssize_t ib_uverbs_open_xrcd(struct ib_uverbs_file *fil return in_len; err_copy: - if (inode) { + if (vnode != NULL) { if (new_xrcd) - xrcd_table_delete(file->device, inode); + xrcd_table_delete(file->device, vnode); atomic_dec(&xrcd->usecnt); } @@ -877,8 +876,8 @@ err: put_uobj_write(&obj->uobject); err_tree_mutex_unlock: - if (f.file) - fdput(f); + if (vnode != NULL) + vrele(vnode); mutex_unlock(&file->device->xrcd_tree_mutex); @@ -893,7 +892,7 @@ ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *fi struct ib_uverbs_close_xrcd cmd; struct ib_uobject *uobj; struct ib_xrcd *xrcd = NULL; - struct inode *inode = NULL; + struct vnode *vnode = NULL; struct ib_uxrcd_object *obj; int live; int ret = 0; @@ -909,7 +908,7 @@ ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *fi } xrcd = uobj->object; - inode = xrcd->inode; + vnode = xrcd->vnode; obj = container_of(uobj, struct ib_uxrcd_object, uobject); if (atomic_read(&obj->refcnt)) { put_uobj_write(uobj); @@ -917,14 +916,14 @@ ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *fi goto out; } - if (!inode || atomic_dec_and_test(&xrcd->usecnt)) { + if (!vnode || atomic_dec_and_test(&xrcd->usecnt)) { ret = ib_dealloc_xrcd(uobj->object); if (!ret) uobj->live = 0; } live = uobj->live; - if (inode && ret) + if (vnode && ret) atomic_inc(&xrcd->usecnt); put_uobj_write(uobj); @@ -932,8 +931,8 @@ ssize_t ib_uverbs_close_xrcd(struct ib_uverbs_file *fi if (ret) goto out; - if (inode && !live) - xrcd_table_delete(file->device, inode); + if (vnode && !live) + xrcd_table_delete(file->device, vnode); idr_remove_uobj(&ib_uverbs_xrcd_idr, uobj); mutex_lock(&file->mutex); @@ -951,16 +950,16 @@ out: void ib_uverbs_dealloc_xrcd(struct ib_uverbs_device *dev, struct ib_xrcd *xrcd) { - struct inode *inode; + struct vnode *vnode; - inode = xrcd->inode; - if (inode && !atomic_dec_and_test(&xrcd->usecnt)) + vnode = xrcd->vnode; + if (vnode && !atomic_dec_and_test(&xrcd->usecnt)) return; ib_dealloc_xrcd(xrcd); - if (inode) - xrcd_table_delete(dev, inode); + if (vnode) + xrcd_table_delete(dev, vnode); } ssize_t ib_uverbs_reg_mr(struct ib_uverbs_file *file,