From nobody Sat Oct 15 18:23:49 2022 X-Original-To: freebsd-arch@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MqWqK4H9Qz4fR4m for ; Sat, 15 Oct 2022 18:23:57 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward503p.mail.yandex.net (forward503p.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MqWqH4Ktdz45Sj; Sat, 15 Oct 2022 18:23:55 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from myt5-69594d4a41fa.qloud-c.yandex.net (myt5-69594d4a41fa.qloud-c.yandex.net [IPv6:2a02:6b8:c12:3ca5:0:640:6959:4d4a]) by forward503p.mail.yandex.net (Yandex) with ESMTP id C54EA110142C; Sat, 15 Oct 2022 21:23:51 +0300 (MSK) Received: by myt5-69594d4a41fa.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id 3sy1yoZu5K-Noj8OJZP; Sat, 15 Oct 2022 21:23:50 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfw.ru; s=mail; t=1665858231; bh=9Jh7ceQImBD0fPheGRw36e79zS3y76q7HK22f1aGowQ=; h=Message-Id:To:Date:References:Cc:In-Reply-To:From:Subject; b=fXQiclDDDuAaEIH/Zu839D5F+KvHT5JyvAZgsxyZ1v6zyb2UEKTLSs1yyO6tdyTZo KtITz2Sf+1bVmc0pjRh0j6TCeSdFDAf41AQ5psXTg2XrO5FqAuDwEzSNLnq0Urrvf0 3gw0dTCcg3lxEsyi1tndm5RjvlHnrAoa/JxPN0TM= Content-Type: text/plain; charset=utf-8 List-Id: Discussion related to FreeBSD architecture List-Archive: https://lists.freebsd.org/archives/freebsd-arch List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arch@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: Re-importing WireGuard driver and utilities From: "Alexander V. Chernikov" In-Reply-To: <742c4fe8-4c25-d7e5-1df3-b2851d90e630@FreeBSD.org> Date: Sat, 15 Oct 2022 19:23:49 +0100 Cc: freebsd-arch Content-Transfer-Encoding: quoted-printable Message-Id: <04EAB90E-06B7-43DC-8D70-8A403E789458@ipfw.ru> References: <742c4fe8-4c25-d7e5-1df3-b2851d90e630@FreeBSD.org> To: John Baldwin X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4MqWqH4Ktdz45Sj X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ipfw.ru header.s=mail header.b=fXQiclDD; dmarc=none; spf=pass (mx1.freebsd.org: domain of melifaro@ipfw.ru designates 2a02:6b8:0:1472:2741:0:8b7:122 as permitted sender) smtp.mailfrom=melifaro@ipfw.ru X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:6b8:0:1472::/64]; R_DKIM_ALLOW(-0.20)[ipfw.ru:s=mail]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arch@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[ipfw.ru:+]; RCVD_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:208722, ipnet:2a02:6b8::/32, country:FI]; FREEFALL_USER(0.00)[melifaro]; ARC_NA(0.00)[]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[ipfw.ru]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 13 Oct 2022, at 18:55, John Baldwin wrote: >=20 > Over the past several months, I have spent some time reviewing the > WireGuard driver including its interactions with the rest of the > kernel and its use of crypto in the kernel. This work was sponsored > by the FreeBSD Foundation and had a few goals: >=20 > 1) Review the driver generally and how it interacted with other parts > of the kernel. >=20 > 2) Review the use of crypto in the driver and evaluate the feasibility > of using existing kernel crypto code where possible. Specifically > included in this goal was aiming to make use of accelerated > software crypto via the ossl(4) driver for datapath crypto. >=20 > 3) Work with upstream to fix any issues encoutered and evaluate the > potential for reintegrating into the source tree. >=20 > In terms of the driver in general, I found a few minor nits and > developed fixes for those that were accepted upstream. I did make one > non-trivial change (also accepted upstream) to fix a CPU scheduling > inefficiency that dated back to the originally-imported driver. > Initially, each input or output packet resulted in scheduling > encryption or decryption tasks on every CPU in the system. I modified > this part of the driver to follow the Linux driver and instead > schedule a single task on a single CPU (chosen via round-robin) for > each packet. This improved throughput in my (simple) tests over epair > interfaces far more than switching to use ossl(4) for the datapath > crypto. Another user also reported that this change alone reduced the > power overhead of FreeBSD VMs using WireGuard in ESXi to be nearly on > par with Linux due to avoiding wasted CPU cycles on tasks that did no > actual work. >=20 > For the crypto used in the driver, I have extended existing crypto > APIs in the kernel to support the algorithms used by WireGuard that > weren't already present (and these changes are already in main). In > general I have not imported any new crypto code into the kernel, but > have added wrappers (often with APIs compatible with Linux) for > existing (but previously unused) routines from libsodium. >=20 > That said, in my review of the driver I also found that the existing > crypto implementations in the WireGuard driver were fine from a > correctness standpoint. Still, I prefer to minimize the number of > copies of crypto algorithm implementations when possible to minimize > future maintenance. >=20 > One bit of crypto is still used directly by the WireGuard driver which > is the use of the Blake2 hashes. In particular, the construction of > the Blake2 hashes requires the length of the desired digest as an > input into initializing the authentication state (similar to how > CBC-MAC used in AES-CCM encodes L in block 0). Here FreeBSD's > in-kernel Blake2 implementation is somewhat broken as it hardcodes > only a single digest length. While OCF itself supports a notion of > variable digest lengths via the csp_mlen field, the software > auth_xform interface does not pass this length down to the Init() or > SetKey() routines, so the auth_xform wrappers of Blake2 are not able > to support variable digest lengths. This could be fixed in the future > by altering the auth_xform interface. However, WireGuard's use of > Blake2 is not a good fit for OCF. Instead, adding an explicit > "library" API around the existing Blake2 implementation is probably > the most straightforward path if we want to eliminate the last of > WireGuard's internal crypto. >=20 > On the third question, I feel that the current driver is stable and > suitable for bring back into the source tree. >=20 > One question compared to the previous driver import is how to manage > the configuration of WireGuard tunnels. The previous driver added new > commands to ifconfig(8) that largely mapped one to one with commands > passed to the upstream wg(8) tool. Upstream WireGuard has since > relicensed their upstream wg(8) tool under a dual MIT/GPL license > permitting direct use on FreeBSD. Using the existing tooling means > that existing recipes for configuring WireGuard on Linux using wg(8) > also apply directly on FreeBSD. It also provides a more seamless > transition path for existing users of the WireGuard driver and utility > in ports vs adding FreeBSD-specific extensions to ifconfig(8). >=20 > Given all that, Kyle Evans, Ed Maste, and myself would like to move > forward with importing the current WireGuard driver and wg(8) utility > into FreeBSD. =46rom upstream's perspective, the driver should simply > move into the tree and be maintained directly in the tree (e.g. as > sys/dev/wg). The wg(8) tool, on the other hand, will be continued to > be maintained by upstream, and so would be imported as normal > third-party software into contrib/. I love the separation boundary (provided that upstream is happy with = it). Having wg(8) as a separate utility, that=E2=80=99s maintained by = the software author is also great. Do you folks have any opinion on the interface used to control the = driver? Currently, wg(8) uses netlink on Linux, ioctl+nvlist on FreeBSD = and some custom ioctls on OpenBSD. Netlink support recently landed to main & there are plans to merge it to = 13-S so it=E2=80=99s available in 13.2. Any thoughts on switching our interface to netlink? I=E2=80=99m happy to = write the interface & tests if there are no objections. >=20 > I have uploaded the main driver for review (along with some followup > local cleanup changes) to https://reviews.freebsd.org/D36909. The > cleanups are included in the phabricator stack off of that review. >=20 > --=20 > John Baldwin >=20