From nobody Thu Oct 13 17:55:15 2022 X-Original-To: freebsd-arch@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MpHH838jhz4fHDq for ; Thu, 13 Oct 2022 17:55:16 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MpHH826QFz3s8D for ; Thu, 13 Oct 2022 17:55:16 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1665683716; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4BmfBngJC+jvkZDXChdKYu2xuNo9lOY0H/VKheIQkWc=; b=pVkuiao7CQYZ5djgXLlqvNyMmecL7MMr6V72q9qB4gpvVomiRb4d5juipaAHdLt/YcBPLY KxR+8lfpXw63jRCqEegtVYcFocDiWW3a3XnvsoPehPdpV8/EBGIQcwCL3cRl+Wzs+t4Jsj YvikR8e3vBePHwM5tbRmKJJ48OVixWIk6OjCZ4JDoxuGsu0//i/H44yoax11poctC4KvCz Edv2WdDtN33I1gFF4witrw1lqdLQbfvkLvc8I8IeJvV0i3OcHQref0RypYJ2HBfl+MMJJs PUb0NLNrEfJcnz1uRB8hxJb7clIzJduKwzdaXptaFzAuInUFp+60pYtAQn+TYg== Received: from [10.0.1.4] (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4MpHH76sltz193G for ; Thu, 13 Oct 2022 17:55:15 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <742c4fe8-4c25-d7e5-1df3-b2851d90e630@FreeBSD.org> Date: Thu, 13 Oct 2022 10:55:15 -0700 List-Id: Discussion related to FreeBSD architecture List-Archive: https://lists.freebsd.org/archives/freebsd-arch List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arch@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.13.1 Content-Language: en-US From: John Baldwin To: 'freebsd-arch' Subject: Re-importing WireGuard driver and utilities Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1665683716; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4BmfBngJC+jvkZDXChdKYu2xuNo9lOY0H/VKheIQkWc=; b=gizrsvicaUPr53zDW6nZS9cV1MZz85NbNoNnz2dSrwVB878qGoXgAoZvI3xkqZOc15HTy9 0gfYYXHL8wmWpJSVCLulyHE3VLpiS1c4edUuEU2D6YfnVPQsTmh2c4qslOCTGc/URNu/zC SNjibAJzd+kSe62c0fGd9lxuzPzr2xiq1Fhbbw82zvMmmkEC4/pIYP2V0yLkRHKk3cqIqy OSLI5Gd102hfGwfbgpj26W8de6L7I1NhHlfwhE8ERwkNjw/wt1gjM2jxQIkouOjxhqwg4x MuJQuUt7ny2o66SbxV3JTFtfvHoqvbjQwPeKEuTe4szgTEyMUmNEc8bHAWk0zw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1665683716; a=rsa-sha256; cv=none; b=bvQ21E/FEnwQhIGdlG0XYdxz1HSbhT4CsM7RaTZsLtPMAANYnzxbeTRYnAAxTZq7cDGfXC 257MOefqiAJFMmI12DUlme29BfhRslQhZp84avxAclv4zFDGRZwQDxS2EvuqoarVnv8WjU pLR6aU6GIIaOA32emrE6N+L2NUuzpozFA75psDZNkOlaZt08pqHdKuhyioMWJwdSmVdapk 6/rdvjRJkhiWFxmzxVSa86SjdZi9XVEuOfW4AzsOIj3bhIqVpO61QbUnRTzHyL1RFSFxP1 pNzlk91SGLtfyhPfcYsXhX4kHoqOKcvvv6idI/oFbHRdItXxRcg4zDc49XcSIQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N Over the past several months, I have spent some time reviewing the WireGuard driver including its interactions with the rest of the kernel and its use of crypto in the kernel. This work was sponsored by the FreeBSD Foundation and had a few goals: 1) Review the driver generally and how it interacted with other parts of the kernel. 2) Review the use of crypto in the driver and evaluate the feasibility of using existing kernel crypto code where possible. Specifically included in this goal was aiming to make use of accelerated software crypto via the ossl(4) driver for datapath crypto. 3) Work with upstream to fix any issues encoutered and evaluate the potential for reintegrating into the source tree. In terms of the driver in general, I found a few minor nits and developed fixes for those that were accepted upstream. I did make one non-trivial change (also accepted upstream) to fix a CPU scheduling inefficiency that dated back to the originally-imported driver. Initially, each input or output packet resulted in scheduling encryption or decryption tasks on every CPU in the system. I modified this part of the driver to follow the Linux driver and instead schedule a single task on a single CPU (chosen via round-robin) for each packet. This improved throughput in my (simple) tests over epair interfaces far more than switching to use ossl(4) for the datapath crypto. Another user also reported that this change alone reduced the power overhead of FreeBSD VMs using WireGuard in ESXi to be nearly on par with Linux due to avoiding wasted CPU cycles on tasks that did no actual work. For the crypto used in the driver, I have extended existing crypto APIs in the kernel to support the algorithms used by WireGuard that weren't already present (and these changes are already in main). In general I have not imported any new crypto code into the kernel, but have added wrappers (often with APIs compatible with Linux) for existing (but previously unused) routines from libsodium. That said, in my review of the driver I also found that the existing crypto implementations in the WireGuard driver were fine from a correctness standpoint. Still, I prefer to minimize the number of copies of crypto algorithm implementations when possible to minimize future maintenance. One bit of crypto is still used directly by the WireGuard driver which is the use of the Blake2 hashes. In particular, the construction of the Blake2 hashes requires the length of the desired digest as an input into initializing the authentication state (similar to how CBC-MAC used in AES-CCM encodes L in block 0). Here FreeBSD's in-kernel Blake2 implementation is somewhat broken as it hardcodes only a single digest length. While OCF itself supports a notion of variable digest lengths via the csp_mlen field, the software auth_xform interface does not pass this length down to the Init() or SetKey() routines, so the auth_xform wrappers of Blake2 are not able to support variable digest lengths. This could be fixed in the future by altering the auth_xform interface. However, WireGuard's use of Blake2 is not a good fit for OCF. Instead, adding an explicit "library" API around the existing Blake2 implementation is probably the most straightforward path if we want to eliminate the last of WireGuard's internal crypto. On the third question, I feel that the current driver is stable and suitable for bring back into the source tree. One question compared to the previous driver import is how to manage the configuration of WireGuard tunnels. The previous driver added new commands to ifconfig(8) that largely mapped one to one with commands passed to the upstream wg(8) tool. Upstream WireGuard has since relicensed their upstream wg(8) tool under a dual MIT/GPL license permitting direct use on FreeBSD. Using the existing tooling means that existing recipes for configuring WireGuard on Linux using wg(8) also apply directly on FreeBSD. It also provides a more seamless transition path for existing users of the WireGuard driver and utility in ports vs adding FreeBSD-specific extensions to ifconfig(8). Given all that, Kyle Evans, Ed Maste, and myself would like to move forward with importing the current WireGuard driver and wg(8) utility into FreeBSD. From upstream's perspective, the driver should simply move into the tree and be maintained directly in the tree (e.g. as sys/dev/wg). The wg(8) tool, on the other hand, will be continued to be maintained by upstream, and so would be imported as normal third-party software into contrib/. I have uploaded the main driver for review (along with some followup local cleanup changes) to https://reviews.freebsd.org/D36909. The cleanups are included in the phabricator stack off of that review. -- John Baldwin