From nobody Fri Oct 28 21:13:13 2022 X-Original-To: freebsd-arch@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MzZyg0rFMz4ftDS for ; Fri, 28 Oct 2022 21:13:15 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MzZyg0Ch3z3FSw; Fri, 28 Oct 2022 21:13:15 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1666991595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+MeHWNeOcaVOZPyPkbx8RkubyGDwR3SegjSH2Na5418=; b=WhrDKJusNTydwQ/3sOwfh7pPaNIJJsnW7OXQ1LeiltPupht0LCAJ+rcU5sQOMmnhDnb6D4 ICG6NcAHqHjj5PIs3bKB5g6Vyp2rIbam4E+N9kLpf5ZpShdjE+BDNDvjjFlEmCDZ1++ZjE KQS7vkaDZ7cwuCWQM44l5Iu+bDKtheftP+q4LGavz7FMvH2wn0YEoSly71lR4JqETcjrES gi7kE7ztguJsP1Cew1SRO+yBRKuhTaSna1yvhl1gb+pipwM9mDnLyH/wHUTvO8wJLWXg9l znWSVlC2X+RjezRYEbQFAOw8RFl7bZHUycEMgcq8hc9wuRCYaTlJcg5afvE8CA== Received: from [10.0.1.4] (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4MzZyf49Wfz1NJP; Fri, 28 Oct 2022 21:13:14 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <6b80ff72-a652-40ad-1e8a-6b671bc18a5d@FreeBSD.org> Date: Fri, 28 Oct 2022 14:13:13 -0700 List-Id: Discussion related to FreeBSD architecture List-Archive: https://lists.freebsd.org/archives/freebsd-arch List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arch@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.3.1 Subject: Re: Re-importing WireGuard driver and utilities Content-Language: en-US To: "Alexander V. Chernikov" Cc: freebsd-arch References: <742c4fe8-4c25-d7e5-1df3-b2851d90e630@FreeBSD.org> <04EAB90E-06B7-43DC-8D70-8A403E789458@ipfw.ru> From: John Baldwin In-Reply-To: <04EAB90E-06B7-43DC-8D70-8A403E789458@ipfw.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1666991595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+MeHWNeOcaVOZPyPkbx8RkubyGDwR3SegjSH2Na5418=; b=yruhT1VRDmJiYGUhFlmrsopJvRZKZmk+Qy9DjbDHPh0+a8JMOpCH0Qggf1o7R4x7ANIWEv rpcAZlvu31vL8eGIiNp4ft/Pwmch1xfYfc/9lsgsKSTsAmt6VUUUrvpu48pz18n/VthgMD tbLsUTJS35c+pk/Ku7L8bcZY1ozj+DE/Nxj3N0VDzs4QraftmCEJfUE8kTwCjEZAETimvx S+9ezip1+dRX2SS1kJ/qs5CNxOwGQaLrtcutKawiMmt8gRglaU/VhOxuoJ2waB6z9QRB1k 7WlL+CK1EIM7Z0L5CuA5KeSrAUiaHjcw3vg9HOr6S1MLGHYd62qhlbwp8V5KUA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1666991595; a=rsa-sha256; cv=none; b=X+/p4Ql9GetLouM+RD908VGfLuCJvbLYZMaUxDVKN4uJ8xaOHVb6Amz1j8k3jdnX0ZLDgF dJksKSvfbnh7Q92Hq8e24wPjYromI0bdSZ2VcUG8xmUx/irdFCn7OXMZqjS1mxyvpLB0oK 9i+UZAY4Y7+E0h3OYPMqcsfw+KgwAb7ov8OdRxCjeBinB04CdMweRmVtRcA+cAnzGJeup0 rDfS69u9XXdKyThY/hA8G54INWtaSsMFRdS8ocQoRO+ioosHj9jzbWftxAADZWKoOCIVhB kj5WlHG8JRXb4qZJtvqlYcnExdYP5eSNz6cKJmEHENwnktJ8SDP0traiSXtxUQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N On 10/15/22 11:23 AM, Alexander V. Chernikov wrote: > On 13 Oct 2022, at 18:55, John Baldwin wrote: >> >> Over the past several months, I have spent some time reviewing the >> WireGuard driver including its interactions with the rest of the >> kernel and its use of crypto in the kernel. This work was sponsored >> by the FreeBSD Foundation and had a few goals: >> >> 1) Review the driver generally and how it interacted with other parts >> of the kernel. >> >> 2) Review the use of crypto in the driver and evaluate the feasibility >> of using existing kernel crypto code where possible. Specifically >> included in this goal was aiming to make use of accelerated >> software crypto via the ossl(4) driver for datapath crypto. >> >> 3) Work with upstream to fix any issues encoutered and evaluate the >> potential for reintegrating into the source tree. >> >> In terms of the driver in general, I found a few minor nits and >> developed fixes for those that were accepted upstream. I did make one >> non-trivial change (also accepted upstream) to fix a CPU scheduling >> inefficiency that dated back to the originally-imported driver. >> Initially, each input or output packet resulted in scheduling >> encryption or decryption tasks on every CPU in the system. I modified >> this part of the driver to follow the Linux driver and instead >> schedule a single task on a single CPU (chosen via round-robin) for >> each packet. This improved throughput in my (simple) tests over epair >> interfaces far more than switching to use ossl(4) for the datapath >> crypto. Another user also reported that this change alone reduced the >> power overhead of FreeBSD VMs using WireGuard in ESXi to be nearly on >> par with Linux due to avoiding wasted CPU cycles on tasks that did no >> actual work. >> >> For the crypto used in the driver, I have extended existing crypto >> APIs in the kernel to support the algorithms used by WireGuard that >> weren't already present (and these changes are already in main). In >> general I have not imported any new crypto code into the kernel, but >> have added wrappers (often with APIs compatible with Linux) for >> existing (but previously unused) routines from libsodium. >> >> That said, in my review of the driver I also found that the existing >> crypto implementations in the WireGuard driver were fine from a >> correctness standpoint. Still, I prefer to minimize the number of >> copies of crypto algorithm implementations when possible to minimize >> future maintenance. >> >> One bit of crypto is still used directly by the WireGuard driver which >> is the use of the Blake2 hashes. In particular, the construction of >> the Blake2 hashes requires the length of the desired digest as an >> input into initializing the authentication state (similar to how >> CBC-MAC used in AES-CCM encodes L in block 0). Here FreeBSD's >> in-kernel Blake2 implementation is somewhat broken as it hardcodes >> only a single digest length. While OCF itself supports a notion of >> variable digest lengths via the csp_mlen field, the software >> auth_xform interface does not pass this length down to the Init() or >> SetKey() routines, so the auth_xform wrappers of Blake2 are not able >> to support variable digest lengths. This could be fixed in the future >> by altering the auth_xform interface. However, WireGuard's use of >> Blake2 is not a good fit for OCF. Instead, adding an explicit >> "library" API around the existing Blake2 implementation is probably >> the most straightforward path if we want to eliminate the last of >> WireGuard's internal crypto. >> >> On the third question, I feel that the current driver is stable and >> suitable for bring back into the source tree. >> >> One question compared to the previous driver import is how to manage >> the configuration of WireGuard tunnels. The previous driver added new >> commands to ifconfig(8) that largely mapped one to one with commands >> passed to the upstream wg(8) tool. Upstream WireGuard has since >> relicensed their upstream wg(8) tool under a dual MIT/GPL license >> permitting direct use on FreeBSD. Using the existing tooling means >> that existing recipes for configuring WireGuard on Linux using wg(8) >> also apply directly on FreeBSD. It also provides a more seamless >> transition path for existing users of the WireGuard driver and utility >> in ports vs adding FreeBSD-specific extensions to ifconfig(8). >> >> Given all that, Kyle Evans, Ed Maste, and myself would like to move >> forward with importing the current WireGuard driver and wg(8) utility >> into FreeBSD. From upstream's perspective, the driver should simply >> move into the tree and be maintained directly in the tree (e.g. as >> sys/dev/wg). The wg(8) tool, on the other hand, will be continued to >> be maintained by upstream, and so would be imported as normal >> third-party software into contrib/. > I love the separation boundary (provided that upstream is happy with it). Having wg(8) as a separate utility, that’s maintained by the software author is also great. > Do you folks have any opinion on the interface used to control the driver? Currently, wg(8) uses netlink on Linux, ioctl+nvlist on FreeBSD and some custom ioctls on OpenBSD. > Netlink support recently landed to main & there are plans to merge it to 13-S so it’s available in 13.2. > Any thoughts on switching our interface to netlink? I’m happy to write the interface & tests if there are no objections. I don't have any strong opinions either way. We'd likely have to keep the ioctl approach under suitable COMPAT_FREEBSD options if we adopted netlink going forward. -- John Baldwin