From nobody Sat Feb 25 23:57:02 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PPNwH1zpnz3v29s; Sat, 25 Feb 2023 23:57:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PPNwH1Sd9z3CKd; Sat, 25 Feb 2023 23:57:03 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1677369423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=60yqR3Zh+oGYQh+Lhh/qk0uuKsv59jSudj/tdtDgdGA=; b=aLwHixy96pX04mYeI99NEm9rZeo3X+JPOZ6g6LQebbz8VvqmIdHOe7kUkeDKYDthB+mMR/ iK3A6eZb00HbokWsdwg/fhNPgU/X/dVYNTIPEuLUXHVNtjUckWWhg1DG55AbD4VD/NOMHO DRgmn8GXR0uBG6s8aR1KrgPiBJzEJXGIHSq5GjktP77PNvLpcTklvL4IP9/2BvoTyX+qL9 9t0RdtoqOoQj0/EEo8VQKontOyFnO+hvlI5MnXz4/gCCfRTZeVuDy2r0/MUPg7iHGAKj3e uHEZddRTGD4+45Kb4sTSpgNHEXFaNUgUCPT45glUX5lvI/WSl8xmnby3qt+yhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1677369423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=60yqR3Zh+oGYQh+Lhh/qk0uuKsv59jSudj/tdtDgdGA=; b=JtE9cAEK32cQrVsTisJ1/oOWzeVbLQ/EdnWJmTLaStVfXW0joOCDiXuibkq/iy1ZATeYRa +MkcGY020NTH05GrrUfE1u0/U2rYp4grHZiBztWX3GkDYFa81d8CUv8EHxU1q2Y+Kgsi5K VSV6PRpORmM683SfKVczdHG76KXaBeYV9YXUIIxsKlXRR8Kk6tLoCbLNmfC72VRSrbipWa etDiyZlUkQCx29hbzIGlkkQ7135sXjH6H2wUNBk83oViOzJoOkGrNLRwhUeTtyCfr1QPKa 0rqPX10ryBGS2Zltb5pwLUCDHuUbQSCryxqN/M6ErOxXOcjJAS4BZn0U2ilqxg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1677369423; a=rsa-sha256; cv=none; b=PA8mbHAf4xt/lIZFp4fG0p7GKkq+Y0z/qFtQYJhr2VVCxSOaFMgcxcsJZv/SfWr8NFUmAE 3srYbvO1uBRipv+SCESakkP8jWd4G36xovR7gELRxBSKb2zIkmeMl6UYg2mgSKJZujvx89 vWGWgIDcyR2nVEg/jQ7anJj+y/Y92AgkVczGbDE+I9YpxF3dGyoA6nYQ3MQbIL+Bl2jLrs SQBfsfT0sC0g/LHNd2QhKyH38s8H2NV0/qSmPUh7yQtCwQ6BZqhIb3zc07omY7K2SroU30 wvAxLZIWRBYDTVjKODoFxXk79Zx7l6YwDo1H8yOhJUs1CK5GKUWooAGSTLIMig== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PPNwH0W5yzNVS; Sat, 25 Feb 2023 23:57:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 31PNv20v071968; Sat, 25 Feb 2023 23:57:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 31PNv23r071967; Sat, 25 Feb 2023 23:57:02 GMT (envelope-from git) Date: Sat, 25 Feb 2023 23:57:02 GMT Message-Id: <202302252357.31PNv23r071967@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Piotr Kubaj Subject: git: 83d6d8877ef7 - main - powerpc64*: port mlx5, OFED, KTLS and krping List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: pkubaj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 83d6d8877ef7dad4f4e8f409a01c9f28139cd026 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by pkubaj: URL: https://cgit.FreeBSD.org/src/commit/?id=83d6d8877ef7dad4f4e8f409a01c9f28139cd026 commit 83d6d8877ef7dad4f4e8f409a01c9f28139cd026 Author: Piotr Kubaj AuthorDate: 2023-02-25 21:09:41 +0000 Commit: Piotr Kubaj CommitDate: 2023-02-25 23:56:37 +0000 powerpc64*: port mlx5, OFED, KTLS and krping Summary: This review ports mlx5 driver, kernel's OFED stack (userland is already enabled), KTLS and krping to powerpc64 and powerpc64le. krping requires a small change since it uses assembly for amd64 / i386. NOTE: On powerpc64le RDMA works fine in the userspace with libmlx5, but on powerpc64 it does not. The problem is that contrib/ofed/libmlx5/doorbell.h checks for SIZEOF_LONG but this macro exists on neither powerpc64* nor amd64. Thus, the file silently goes to the fallback function written for 32-bit architectures. It works fine on little-endian architectures, but causes a hard fail on big-endian. It's possible it may also cause some runtime issues on little-endian. Thus, on powerpc64 I verified that RDMA works with krping. Reviewers: #powerpc, hselasky Subscribers: bdrewery, imp, emaste, jhibbits Differential Revision: https://reviews.freebsd.org/D38786 --- share/mk/src.opts.mk | 4 ++-- sys/conf/kern.opts.mk | 3 ++- sys/contrib/rdma/krping/krping.c | 4 ++++ sys/modules/Makefile | 13 +++++++++++++ sys/powerpc/conf/GENERIC64 | 10 ++++++++++ sys/powerpc/conf/GENERIC64LE | 10 ++++++++++ 6 files changed, 41 insertions(+), 3 deletions(-) diff --git a/share/mk/src.opts.mk b/share/mk/src.opts.mk index fe2159cd4f5e..8008a94ae329 100644 --- a/share/mk/src.opts.mk +++ b/share/mk/src.opts.mk @@ -318,8 +318,8 @@ BROKEN_OPTIONS+=LOADER_UBOOT BROKEN_OPTIONS+=LOADER_GELI LOADER_LUA .endif -# Kernel TLS is enabled by default on amd64 and aarch64 -.if ${__T} == "aarch64" || ${__T} == "amd64" +# Kernel TLS is enabled by default on amd64, aarch64 and powerpc64* +.if ${__T} == "aarch64" || ${__T} == "amd64" || ${__T:Mpowerpc64*} != "" __DEFAULT_YES_OPTIONS+=OPENSSL_KTLS .else __DEFAULT_NO_OPTIONS+=OPENSSL_KTLS diff --git a/sys/conf/kern.opts.mk b/sys/conf/kern.opts.mk index 5cda70156394..93edc1b6e63f 100644 --- a/sys/conf/kern.opts.mk +++ b/sys/conf/kern.opts.mk @@ -84,7 +84,8 @@ BROKEN_OPTIONS+= ZFS # Things that don't work because the kernel doesn't have the support # for them. -.if ${MACHINE} != "i386" && ${MACHINE} != "amd64" +.if ${MACHINE} != "i386" && ${MACHINE} != "amd64" && \ + ${MACHINE_ARCH:Mpowerpc64*} == "" BROKEN_OPTIONS+= OFED .endif diff --git a/sys/contrib/rdma/krping/krping.c b/sys/contrib/rdma/krping/krping.c index 182e5f111028..98f129597464 100644 --- a/sys/contrib/rdma/krping/krping.c +++ b/sys/contrib/rdma/krping/krping.c @@ -69,9 +69,13 @@ MODULE_DEPEND(krping, linuxkpi, 1, 1, 1); static __inline uint64_t get_cycles(void) { +#if defined(__amd64__) || defined(__i386__) uint32_t low, high; __asm __volatile("rdtsc" : "=a" (low), "=d" (high)); return (low | ((u_int64_t)high << 32)); +#elif defined(__powerpc64__) + return __builtin_readcyclecounter(); +#endif } typedef uint64_t cycles_t; diff --git a/sys/modules/Makefile b/sys/modules/Makefile index 9c2664460067..4397c7beb56e 100644 --- a/sys/modules/Makefile +++ b/sys/modules/Makefile @@ -843,8 +843,21 @@ _nvme= nvme .if ${MACHINE_ARCH:Mpowerpc64*} != "" _ipmi= ipmi _ixl= ixl +_mlx5= mlx5 _nvram= opal_nvram +.if ${MK_OFED} != "no" || defined(ALL_MODULES) +_ibcore= ibcore +_ipoib= ipoib +_iser= iser +_mlx5ib= mlx5ib +_rdma= rdma +.endif +.if (${MK_INET_SUPPORT} != "no" && ${MK_INET6_SUPPORT} != "no") || \ + defined(ALL_MODULES) +_mlx5en= mlx5en +.endif .endif + .if ${MACHINE_CPUARCH} == "powerpc" && ${MACHINE_ARCH} != "powerpcspe" # Don't build powermac_nvram for powerpcspe, it's never supported. _nvram+= powermac_nvram diff --git a/sys/powerpc/conf/GENERIC64 b/sys/powerpc/conf/GENERIC64 index ab36e2032203..6c40b031bee6 100644 --- a/sys/powerpc/conf/GENERIC64 +++ b/sys/powerpc/conf/GENERIC64 @@ -49,6 +49,7 @@ options TCP_BLACKBOX # Enhanced TCP event logging options TCP_HHOOK # hhook(9) framework for TCP options TCP_RFC7413 # TCP Fast Open options SCTP_SUPPORT # Allow kldload of SCTP +options KERN_TLS # TLS transmit & receive offload options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -128,6 +129,8 @@ device pci options PCI_HP # PCI-Express native HotPlug device agp +options COMPAT_LINUXKPI + # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers @@ -189,6 +192,12 @@ device fxp # Intel EtherExpress PRO/100B (82557, 82558) device re # RealTek 8139C+/8169/8169S/8110S device rl # RealTek 8129/8139 +# Nvidia/Mellanox Connect-X 4 and later, Ethernet only +# mlx5ib requires ibcore infra and is not included by default +device mlx5 # Base driver +device mlxfw # Firmware update +device mlx5en # Ethernet driver + # Pseudo devices. device crypto # core crypto support device loop # Network loopback @@ -199,6 +208,7 @@ device md # Memory "disks" device ofwd # Open Firmware disks device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module +device xz # lzma decompression # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! diff --git a/sys/powerpc/conf/GENERIC64LE b/sys/powerpc/conf/GENERIC64LE index ffb43628d2b1..99d1975442f7 100644 --- a/sys/powerpc/conf/GENERIC64LE +++ b/sys/powerpc/conf/GENERIC64LE @@ -46,6 +46,7 @@ options TCP_BLACKBOX # Enhanced TCP event logging options TCP_HHOOK # hhook(9) framework for TCP options TCP_RFC7413 # TCP Fast Open options SCTP_SUPPORT # Allow kldload of SCTP +options KERN_TLS # TLS transmit & receive offload options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists @@ -123,6 +124,8 @@ device pci options PCI_HP # PCI-Express native HotPlug device agp +options COMPAT_LINUXKPI + # ATA controllers device ahci # AHCI-compatible SATA controllers device ata # Legacy ATA/SATA controllers @@ -184,6 +187,12 @@ device fxp # Intel EtherExpress PRO/100B (82557, 82558) device re # RealTek 8139C+/8169/8169S/8110S device rl # RealTek 8129/8139 +# Nvidia/Mellanox Connect-X 4 and later, Ethernet only +# mlx5ib requires ibcore infra and is not included by default +device mlx5 # Base driver +device mlxfw # Firmware update +device mlx5en # Ethernet driver + # Pseudo devices. device crypto # core crypto support device loop # Network loopback @@ -194,6 +203,7 @@ device md # Memory "disks" device ofwd # Open Firmware disks device gif # IPv6 and IPv4 tunneling device firmware # firmware assist module +device xz # lzma decompression # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this!