From owner-svn-src-stable-11@freebsd.org Thu Aug 2 08:15:07 2018 Return-Path: Delivered-To: svn-src-stable-11@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DEDA105D405; Thu, 2 Aug 2018 08:15:07 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2EF47A11B; Thu, 2 Aug 2018 08:15:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A50F770EF; Thu, 2 Aug 2018 08:15:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728F6Bf039443; Thu, 2 Aug 2018 08:15:06 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728F50o039436; Thu, 2 Aug 2018 08:15:05 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020815.w728F50o039436@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:15:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337078 - in stable/11/sys: dev/mlx4/mlx4_ib dev/mlx5/mlx5_ib ofed/drivers/infiniband/core ofed/include/rdma X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys: dev/mlx4/mlx4_ib dev/mlx5/mlx5_ib ofed/drivers/infiniband/core ofed/include/rdma X-SVN-Commit-Revision: 337078 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-11@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for only the 11-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:15:07 -0000 Author: hselasky Date: Thu Aug 2 08:15:05 2018 New Revision: 337078 URL: https://svnweb.freebsd.org/changeset/base/337078 Log: MFC r336372: Add support for prio-tagged traffic for RDMA in ibcore. When receiving a PCP change all GID entries are reloaded. This ensures the relevant GID entries use prio tagging, by setting VLAN present and VLAN ID to zero. The priority for prio tagged traffic is set using the regular rdma_set_service_type() function. Fake the real network device to have a VLAN ID of zero when prio tagging is enabled. This is logic is hidden inside the rdma_vlan_dev_vlan_id() function which must always be used to retrieve the VLAN ID throughout all of ibcore and the infiniband network drivers. The VLAN presence information then propagates through all of ibcore and so incoming connections will have the VLAN bit set. The incoming VLAN ID is then checked against the return value of rdma_vlan_dev_vlan_id(). Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c stable/11/sys/ofed/include/rdma/ib_addr.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c ============================================================================== --- stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c Thu Aug 2 08:15:05 2018 (r337078) @@ -96,8 +96,7 @@ static struct ib_ah *create_iboe_ah(struct ib_pd *pd, return ERR_PTR(ret); eth_zero_addr(ah->av.eth.s_mac); if (gid_attr.ndev) { - if (is_vlan_dev(gid_attr.ndev)) - vlan_tag = vlan_dev_vlan_id(gid_attr.ndev); + vlan_tag = rdma_vlan_dev_vlan_id(gid_attr.ndev); memcpy(ah->av.eth.s_mac, IF_LLADDR(gid_attr.ndev), ETH_ALEN); dev_put(gid_attr.ndev); } Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:15:05 2018 (r337078) @@ -285,14 +285,16 @@ static void ib_gid_to_mlx5_roce_addr(const union ib_gi source_l3_address); void *mlx5_addr_mac = MLX5_ADDR_OF(roce_addr_layout, mlx5_addr, source_mac_47_32); + u16 vlan_id; if (!gid) return; ether_addr_copy(mlx5_addr_mac, IF_LLADDR(attr->ndev)); - if (is_vlan_dev(attr->ndev)) { + vlan_id = rdma_vlan_dev_vlan_id(attr->ndev); + if (vlan_id != 0xffff) { MLX5_SET_RA(mlx5_addr, vlan_valid, 1); - MLX5_SET_RA(mlx5_addr, vlan_id, vlan_dev_vlan_id(attr->ndev)); + MLX5_SET_RA(mlx5_addr, vlan_id, vlan_id); } switch (attr->gid_type) { Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c Thu Aug 2 08:15:05 2018 (r337078) @@ -402,7 +402,20 @@ static struct notifier_block nb_inetaddr = { .notifier_call = inetaddr_event }; +static eventhandler_tag eh_ifnet_event; + static void +roce_ifnet_event(void *arg, struct ifnet *ifp, int event) +{ + if (event != IFNET_EVENT_PCP || is_vlan_dev(ifp)) + return; + + /* make sure GID table is reloaded */ + roce_gid_delete_all_event(ifp); + roce_gid_queue_scan_event(ifp); +} + +static void roce_rescan_device_handler(struct work_struct *_work) { struct roce_rescan_work *work = @@ -445,11 +458,18 @@ int __init roce_gid_mgmt_init(void) */ register_netdevice_notifier(&nb_inetaddr); + eh_ifnet_event = EVENTHANDLER_REGISTER(ifnet_event, + roce_ifnet_event, NULL, EVENTHANDLER_PRI_ANY); + return 0; } void __exit roce_gid_mgmt_cleanup(void) { + + if (eh_ifnet_event != NULL) + EVENTHANDLER_DEREGISTER(ifnet_event, eh_ifnet_event); + unregister_inetaddr_notifier(&nb_inetaddr); unregister_netdevice_notifier(&nb_inetaddr); Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:15:05 2018 (r337078) @@ -402,12 +402,8 @@ static bool find_gid_index(const union ib_gid *gid, if (ctx->gid_type != gid_attr->gid_type) return false; - - if ((!!(ctx->vlan_id != 0xffff) == !is_vlan_dev(gid_attr->ndev)) || - (is_vlan_dev(gid_attr->ndev) && - vlan_dev_vlan_id(gid_attr->ndev) != ctx->vlan_id)) + if (rdma_vlan_dev_vlan_id(gid_attr->ndev) != ctx->vlan_id) return false; - return true; } @@ -484,7 +480,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 po if (rdma_protocol_roce(device, port_num)) { struct ib_gid_attr dgid_attr; - const u16 vlan_id = wc->wc_flags & IB_WC_WITH_VLAN ? + const u16 vlan_id = (wc->wc_flags & IB_WC_WITH_VLAN) ? wc->vlan_id : 0xffff; if (!(wc->wc_flags & IB_WC_GRH)) Modified: stable/11/sys/ofed/include/rdma/ib_addr.h ============================================================================== --- stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:15:05 2018 (r337078) @@ -163,6 +163,8 @@ static inline u16 rdma_vlan_dev_vlan_id(const struct n { uint16_t tag; + if (dev->if_pcp != IFNET_PCP_NONE) + return 0x0000; /* prio-tagged traffic */ if (VLAN_TAG(__DECONST(struct ifnet *, dev), &tag) != 0) return 0xffff; return tag; @@ -342,8 +344,10 @@ static inline u16 rdma_get_vlan_id(union ib_gid *dgid) return vid < 0x1000 ? vid : 0xffff; } -static inline struct net_device *rdma_vlan_dev_real_dev(const struct net_device *dev) +static inline struct net_device *rdma_vlan_dev_real_dev(struct net_device *dev) { + if (dev->if_pcp != IFNET_PCP_NONE) + return dev; /* prio-tagged traffic */ return VLAN_TRUNKDEV(__DECONST(struct ifnet *, dev)); }