Date: Tue, 5 Mar 2024 09:31:45 +0200 From: Konstantin Belousov <kib@freebsd.org> To: net@freebsd.org Cc: yohadt@nvidia.com, ddaniel@nvidia.com Subject: IPSEC inline offload Message-ID: <ZebKYU9R2L7hl6gR@kib.kiev.ua>
next in thread | raw e-mail | index | archive | help
NVIDIA networking developed support for inline IPSEC offload on FreeBSD. It consists of the common infrastructure part, and changes for mlx5en(4) driver fitting the kernel counterpart. Inline offload moves (almost) all IPSEC processing of the packet from CPU and crypto accelerators, to the network card. Put it differently, on Tx card is fed with plain-text packets and sends out encapsulated and encrypted/authenticated packet on wire. On Rx, card verifies authentification and then does decryption/decapsulation, pushing the plain text packet to the host' IP stack. Some additions were made to the PF_KEY socket interface to allow more fine control of the offload, and to get some visibility into the offload operation. Otherwise, kernel with offload support is compatible with the existing interfaces, in particular, existing IKE daemons work with it as is. Both SPD and SAD are pushed to the card. Right now, driver supports only IPv4 AES-GCM transport mode, UDP ESP encapsulation (NAT-T), and several stateless offloads, most significant of which is TSO. RoCE over offloaded IPSEC works seamlessly. No other modes are supported. This is not a fundamental restriction of the design, only a current state in the development efforts. Architectural limitation of the hardware plus current infrastructure design is lack of support for fragmentation. The customer which driven the efforts does not need it, and for most interesting workloads it is not needed anyway. Also see above about TSO support. In some not scientific comparisions we made, between two FreeBSD hosts, single-stream TCP performance measured with iperf3: non-offloaded IPSEC vs non-offloaded IPSEC 3Mbit/sec offloaded IPSEC (traffic src) vs non-offloaded IPSEC 3.2Gbit/sec Patches are currently split into two jumbo reviews: infra https://reviews.freebsd.org/D44219 mlx5 driver https://reviews.freebsd.org/D44220
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ZebKYU9R2L7hl6gR>