From nobody Fri Jul 19 19:23:02 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQfj06CS6z5QCCy; Fri, 19 Jul 2024 19:23:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQfj00dhcz4GM3; Fri, 19 Jul 2024 19:23:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 46JJN3Ow016473; Fri, 19 Jul 2024 22:23:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 46JJN3Ow016473 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 46JJN2R2016472; Fri, 19 Jul 2024 22:23:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 19 Jul 2024 22:23:02 +0300 From: Konstantin Belousov To: Kristof Provost Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: ef2a572bf6bd - main - ipsec_offload: kernel infrastructure Message-ID: References: <202407121125.46CBP8eo093121@gitrepo.freebsd.org> List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4WQfj00dhcz4GM3 On Fri, Jul 19, 2024 at 04:26:43PM +0200, Kristof Provost wrote: > On 12 Jul 2024, at 13:25, Konstantin Belousov wrote: > > The branch main has been updated by kib: > > > > URL: https://cgit.FreeBSD.org/src/commit/?id=ef2a572bf6bdcac97ef29ce631d2f50f938e1ec8 > > > > commit ef2a572bf6bdcac97ef29ce631d2f50f938e1ec8 > > Author: Konstantin Belousov > > AuthorDate: 2021-08-22 19:38:04 +0000 > > Commit: Konstantin Belousov > > CommitDate: 2024-07-12 04:27:58 +0000 > > > > ipsec_offload: kernel infrastructure > > > > Inline IPSEC offload moves almost whole IPSEC processing from the > > CPU/MCU and possibly crypto accelerator, to the network card. > > > > The transmitted packet content is not touched by CPU during TX > > operations, kernel only does the required policy and security > > association lookups to find out that given flow is offloaded, and > > then > > packet is transmitted as plain text to the card. For driver > > convenience, > > a metadata is attached to the packet identifying SA which must > > process > > the packet. Card does encryption of the payload, padding, calculates > > authentication, and does the reformat according to the policy. > > > > Similarly, on receive, card does the decapsulation, decryption, and > > authentification. Kernel receives the identifier of SA that was > > used to process the packet, together with the plain-text packet. > > > > Overall, payload octets are only read or written by card DMA engine, > > removing a lot of memory subsystem overhead, and saving CPU time > > because > > IPSEC algos calculations are avoided. > > > > If driver declares support for inline IPSEC offload (with the > > IFCAP2_IPSEC_OFFLOAD capability set and registering method table > > struct > > if_ipsec_accel_methods), kernel offers the SPD and SAD to driver. > > Driver decides which policies and SAs can be offloaded based on > > hardware capacity, and acks/nacks each SA for given interface to > > kernel. Kernel needs to keep this information to make a decision to > > skip software processing on TX, and to assume processing already > > done > > on RX. This shadow SPD/SAD database of offloads is rooted from > > policies (struct secpolicy accel_ifps, struct ifp_handle_sp) and SAs > > (struct secasvar accel_ipfs, struct ifp_handle_sav). > > > > Some extensions to the PF_KEY socket allow to limit interfaces for > > which given SP/SA could be offloaded (proposed for offload). Also, > > additional statistics extensions allow to observe > > allocation/octet/use > > counters for specific SA. > > > > Since SPs and SAs are typically instantiated in non-sleepable > > context, > > while offloading them into card is expected to require costly async > > manipulations of the card state, calls to the driver for offload and > > termination are executed in the threaded taskqueue. It also solves > > the issue of allocating resources needed for the offload database. > > Neither ipf_handle_sp nor ipf_handle_sav do not add reference to the > > owning SP/SA, the offload must be terminated before last reference > > is > > dropped. ipsec_accel only adds transient references to ensure safe > > pointer ownership by taskqueue. > > > > Maintaining the SA counters for hardware-accelerated packets is the > > duty of the driver. The helper ipsec_accel_drv_sa_lifetime_update() > > is provided to hide accel infrastructure from drivers which would > > use > > expected callout to query hardware periodically for updates. > > > > Reviewed by: rscheff (transport, stack integration), np > > Sponsored by: NVIDIA networking > > Differential revision: https://reviews.freebsd.org/D44219 > > --- > > sys/conf/files | 2 + > > sys/conf/options | 1 + > > sys/modules/ipsec/Makefile | 5 +- > > sys/netipsec/ipsec.c | 17 + > > sys/netipsec/ipsec.h | 11 + > > sys/netipsec/ipsec_input.c | 11 + > > sys/netipsec/ipsec_offload.c | 1061 > > ++++++++++++++++++++++++++++++++++++++++++ > > sys/netipsec/ipsec_offload.h | 191 ++++++++ > > sys/netipsec/ipsec_output.c | 15 + > > sys/netipsec/ipsec_pcb.c | 38 +- > > sys/netipsec/key.c | 270 ++++++++++- > > sys/netipsec/key.h | 6 + > > sys/netipsec/key_debug.c | 5 + > > sys/netipsec/keydb.h | 14 + > > 14 files changed, 1628 insertions(+), 19 deletions(-) > > > I’m seeing messages like `ipsec_accel_sa_install_newkey: spi 0x1001 flags > 0x40 seq 0` running the test suite now. > Are those stray debug printfs? Not stray, but also they should not be there by default indeed. https://reviews.freebsd.org/D46045