From nobody Thu Aug 3 17:51:57 2023 X-Original-To: freebsd-ppc@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RGxHh5FFhz4TlMX; Thu, 3 Aug 2023 17:52:00 +0000 (UTC) (envelope-from sanastasio@raptorengineering.com) Received: from raptorengineering.com (mail.raptorengineering.com [23.155.224.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RGxHg623Sz4ky9; Thu, 3 Aug 2023 17:51:59 +0000 (UTC) (envelope-from sanastasio@raptorengineering.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=raptorengineering.com header.s=B8E824E6-0BE2-11E6-931D-288C65937AAD header.b=M6CZ4diZ; spf=pass (mx1.freebsd.org: domain of sanastasio@raptorengineering.com designates 23.155.224.40 as permitted sender) smtp.mailfrom=sanastasio@raptorengineering.com; dmarc=pass (policy=quarantine) header.from=raptorengineering.com Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id F2AD78285406; Thu, 3 Aug 2023 12:51:58 -0500 (CDT) Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id WHesxDq3Ddyv; Thu, 3 Aug 2023 12:51:58 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 38EDC8285487; Thu, 3 Aug 2023 12:51:58 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 38EDC8285487 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD; t=1691085118; bh=4Vxg2XWsA+qqr2qM1uCaK1pdP62hIVl9UNC0n3PncmM=; h=Message-ID:Date:MIME-Version:From:To; b=M6CZ4diZaEnLKE2cZZP2jmAn2rYkI1hjrPXsEgGyt/VlVEWROZbHwcwJI9vVnoDwP gyWg/UV9nlzSAOSwgU2q0/jAbeMBuTjWydgiFomYSf0QOesPyR6ex53NROki6yL6Nq M4ExyFp4OUYBO0k8okMu0bGmxPtUG6XY2Pxvl1CA= X-Virus-Scanned: amavisd-new at rptsys.com Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4Bon8-AhG5I6; Thu, 3 Aug 2023 12:51:58 -0500 (CDT) Received: from [10.11.0.2] (5.edge.rptsys.com [23.155.224.38]) by mail.rptsys.com (Postfix) with ESMTPSA id C86158285406; Thu, 3 Aug 2023 12:51:57 -0500 (CDT) Message-ID: <0c24b4b7-b4c8-242d-6187-15b171c50c19@raptorengineering.com> Date: Thu, 3 Aug 2023 12:51:57 -0500 List-Id: Porting FreeBSD to the PowerPC List-Archive: https://lists.freebsd.org/archives/freebsd-ppc List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ppc@freebsd.org X-BeenThere: freebsd-ppc@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux ppc64le; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US From: Shawn Anastasio Subject: Implementing in-kernel AES crypto acceleration on ppc (POWER8+) To: freebsd-ppc@FreeBSD.org Cc: freebsd-hackers@FreeBSD.org, Timothy Pearson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[raptorengineering.com,quarantine]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[raptorengineering.com:s=B8E824E6-0BE2-11E6-931D-288C65937AAD]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org,freebsd-ppc@FreeBSD.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[raptorengineering.com:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:46246, ipnet:23.155.224.0/24, country:US]; RCVD_COUNT_FIVE(0.00)[5]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: - X-Rspamd-Queue-Id: 4RGxHg623Sz4ky9 Hello all, Raptor Engineering is interested in adding support for in-kernel AES acceleration on ppc64 via the VMX crypto instructions added in ISA 2.07B, and I wanted to reach out to the community with a few questions. 1. As I understand it, FreeBSD already has support for in-kernel crypto acceleration on x86 and ARM via the aesni and armv8_crypto drivers respectively that each implement the cryptodev interface. Am I correct in understanding that adding AES acceleration for Power would just involve creating another driver here, or are there other pieces of the puzzle that I've missed? 2. I see that both the aesni and armv8 drivers make use of the fpu_kern_enter/fpu_kern_leave functions to guard access to vector registers, but it appears that these functions aren't implemented on ppc. Is that correct, or does an in-kernel facility for safely accessing vector registers on ppc already exist? 3. For the accelerated AES implementation itself, I've noticed that cryptogams[*] contains an implementation that is both widely deployed (and thus tested and likely to be correct) and also BSD licensed. Would it be acceptable to import the relevant routines to the FreeBSD kernel and have the new cryptodev driver simply call into them, or are there other considerations involved? 4. Is there a userspace test framework for the cryptodev API that could be used to validate and benchmark the new implementation, or would I have to write that myself? It appears that OpenSSL had support for /dev/crypto at one point, but I'm not sure that is the case any longer. My apologies for the large number of questions, but I look forward to hearing back and working with the FreeBSD community to get this implemented. Thanks, Shawn Anastasio [*] https://github.com/dot-asm/cryptogams From nobody Fri Aug 4 13:36:05 2023 X-Original-To: freebsd-ppc@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RHRZ019Hsz4kTK0; Fri, 4 Aug 2023 13:36:08 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RHRZ00J0cz3CB1; Fri, 4 Aug 2023 13:36:08 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691156168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0uNWYYnTDq/VsgaoM7Gz88oWckbqfTd9WXszadGHlHs=; b=CMv/0qPnN9qJ7UZLopHjtiKSpk1Nxk+DhD985KOyasZYX3xVJe/J+uRVDfhhIiNrGndnzg 3DclY+SVYy1Pm0zj5G9Xf/XWRhOwQa+MEq2QGrCKJDfeRmweDgNtznFjrM/bVBt0p9Zpwu cLa+u1ftgy6xrT80YxMbVOvJOBGiSJxnRxXKQvIcGDJO+NPgx9um8zCH9n4RhU0aR97Zty ie0to8VaWNpotbffaAUHVKr5d345lvNGoKNnuKzvGDarXLtPzsLHjOHTZb2QOvbascXlmA voKu6JJJ+Fsa0OY3fp9TF0E6suFgwAz4ahjJH5jIqwk9wWvWUFDBz/WmgR9DxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691156168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0uNWYYnTDq/VsgaoM7Gz88oWckbqfTd9WXszadGHlHs=; b=Ljwh13sKvW6Y48rxM6qoLL+wugY0MKmKZzL+skwfg1TCH6zKWSWkq/M/Fguaq+j2oBcgcR U6KzfTWRAE3w/waauA7XUJt30dqMdoA563HIo3cYf+usDQSxYJu35A/8hZbMLsMgkOsdPs 4oH/hLSLCqLNiidsYkNnqsqY9YegSwyIq9Bz6ymyl3arSWHxU6Xe4Wqdjg2MW02702f743 S+zfJDUbfV57yH/lGHXIV7QM5+0k9bpAHGtqg4UxXeFfVsS8SU2mk/saB9vR8ncFvNLs4g GANggh9RIykJmp5FKzqZC/iqEdPAjzqzN9fcvLZE1KedVEE7vLVLhG6hR/mpiA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691156168; a=rsa-sha256; cv=none; b=RObrsH7YmWsI2rshvLtDh8nNtHcWeIcg4SZi1t/02ob567xW66NFL1IS73NRFWCTcAI77W bHr2y7S1VJVPjLmSQabrkCmzyAdpFyFzAi0fgHQ7SsHFasbnsPsnChfdpRO9zjJY+E4TmG jDilDYpcGplep/d0cPFFhoPLcOi7UbRgJVJTUcB1/faDECZyOYwKe740/rNYbi3SoOMfBq qPI23ZLAmp/cqYSnWa5cJfWPDXwow32UdG0RL2U+ML/Rk1A8pN9KQuOvKvhohJgXbOFdeH /EJlOqWd2XvZ7rRz//DZHmLL22gk0TN46WVmwO5yHiq0q9Wy1Bux1+MFKPD3Ig== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ralga.knownspace (ip-163-182-7-56.dynamic.fuse.net [163.182.7.56]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhibbits) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RHRYz4kGlz11LF; Fri, 4 Aug 2023 13:36:07 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Date: Fri, 4 Aug 2023 09:36:05 -0400 From: Justin Hibbits To: Shawn Anastasio Cc: freebsd-ppc@FreeBSD.org, freebsd-hackers@FreeBSD.org, Timothy Pearson , John Baldwin Subject: Re: Implementing in-kernel AES crypto acceleration on ppc (POWER8+) Message-ID: <20230804093605.2a61eeed@ralga.knownspace> In-Reply-To: <0c24b4b7-b4c8-242d-6187-15b171c50c19@raptorengineering.com> References: <0c24b4b7-b4c8-242d-6187-15b171c50c19@raptorengineering.com> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; powerpc64le-unknown-linux-gnu) List-Id: Porting FreeBSD to the PowerPC List-Archive: https://lists.freebsd.org/archives/freebsd-ppc List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ppc@freebsd.org X-BeenThere: freebsd-ppc@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Hello, Good to see this! I'll answer inline. On Thu, 3 Aug 2023 12:51:57 -0500 Shawn Anastasio wrote: > Hello all, > > Raptor Engineering is interested in adding support for in-kernel AES > acceleration on ppc64 via the VMX crypto instructions added in ISA > 2.07B, and I wanted to reach out to the community with a few > questions. I would love to see this added. > > 1. As I understand it, FreeBSD already has support for in-kernel > crypto acceleration on x86 and ARM via the aesni and armv8_crypto > drivers respectively that each implement the cryptodev interface. Am I > correct in understanding that adding AES acceleration for Power > would just involve creating another driver here, or are there other > pieces of the puzzle that I've missed? John Baldwin can probably answer this better, but I think your understanding is correct. There might be some plumbing needed as well, but that should be minimal. > > 2. I see that both the aesni and armv8 drivers make use of the > fpu_kern_enter/fpu_kern_leave functions to guard access to vector > registers, but it appears that these functions aren't implemented > on ppc. Is that correct, or does an in-kernel facility for safely > accessing vector registers on ppc already exist? Nope, ppc doesn't have these facilities yet. It shouldn't be hard to implement, we just haven't done it yet. If you're interested in implementing them, you should be able to model it after arm64, largely. > > 3. For the accelerated AES implementation itself, I've noticed that > cryptogams[*] contains an implementation that is both widely > deployed (and thus tested and likely to be correct) and also BSD > licensed. Would it be acceptable to import the relevant routines to > the FreeBSD kernel and have the new cryptodev driver simply call into > them, or are there other considerations involved? I think the right way to do that would be to import the code as-is as third party code, and call into the routines that you need. You can #ifdef out the unneeded bits, but try to keep it as intact as possible from upstream. > > 4. Is there a userspace test framework for the cryptodev API that > could be used to validate and benchmark the new implementation, or > would I have to write that myself? It appears that OpenSSL had > support for /dev/crypto at one point, but I'm not sure that is the > case any longer. John Baldwin might have some ideas here, too. > > My apologies for the large number of questions, but I look forward to > hearing back and working with the FreeBSD community to get this > implemented. > > Thanks, > Shawn Anastasio > > [*] https://github.com/dot-asm/cryptogams > - Justin From nobody Fri Aug 4 14:55:50 2023 X-Original-To: freebsd-ppc@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RHTL01hdJz4TkM8; Fri, 4 Aug 2023 14:55:52 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RHTL018dGz3Mv9; Fri, 4 Aug 2023 14:55:52 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691160952; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eikqwniskib4EfMJ7FVNqh51DaXqjA8jRpz04YBFqt4=; b=d/U29cNjbbuKpfKuFGn6x03+QwkdxkmuAxF21oc+q4mVTPr3CqYazrjScYH8JsuP3s8/Kc 16BM1C9oqFBsX85z6zWYDlHAU5iRd5FMdHgoJXncJ4Th/wp0rPHkQG2EMfmgocq5tcUnSC cR3mf4mOpbaiCQfzzC+lZhYI8M3IdFf0T+p2dKdWHxu/5y19L2ziANOk4sTyvmFYn24Fi6 Lp/kVtNCp91NaKP8XBDd4i65WPdUksT4AHzit5k3d+8fAt2iJHnuVjC79u/cPqX0p9Xr4d 8MvMm+tWyQfRZFz4QOeT9XCxXJ+3aN/o/DMcD5zixGBqWng61qVD3f0kbjduNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691160952; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eikqwniskib4EfMJ7FVNqh51DaXqjA8jRpz04YBFqt4=; b=J8/hyZIBcFGGiEFLLDA3pFUmGtcTzWYg0ZV8dL0BbuaqnxzVM+F9k3BD5TjFrTeRTmH9ia VgGvKipc2aUTYMtjL8silwbWVDTxfsh/v6Tvk+lBKglP8iO01MYJ+0VyJdjMu+K8XswHmM dlHNTqaevurQzvl56RIx8Kl8phmIA6ZOM+OQW5ApDqizOq08mYxTqfUG/A3aQyyY0kz0eG z+H3LSGe/g1zlIGgDk1gojr1Sln6UFKOJucA4hNSP+zJTi1filjX+Myp82tbS6jQvoBJX/ gbNySyKKYnbDJU/kUU+wIM3BhCQxfJbJbNJAFd5FVg1k5j9RoSMKGsKTlJvCEA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691160952; a=rsa-sha256; cv=none; b=xHn3UwiuhTQ/CwVhuvWbgmdCybxAPReRXS7BrfOHewAir5np0kS3FKaivhh+Aw06tU4wL3 pnt6Q0HhumboQLspV9fs9FMNkhHtD15g2fnHozT3hTUWqms8fWje1bcVT3MyXVxLso6dDD jeWF5g78gZGccDJPiRNNfyZmRLYRr1Hb0XY10G4LB9SzvF7uiyV1cdxhsLV64zAKmC2qTk xFniAOzDRgbMPtfUd2kKidOJFyTeYDB5Bc9+si3yY+Lh7wjUg3z/Mml495hPgq7vEl3Fhp /hJShUxLnzFwQmz6Th61bgfkr4KsMd3m+WTSIClzyvw2ROGoqscOwIOgBq/01A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [IPV6:2601:648:8680:16b0:cdd2:66b:dfd1:f731] (unknown [IPv6:2601:648:8680:16b0:cdd2:66b:dfd1:f731]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RHTKz3YJqz11Nn; Fri, 4 Aug 2023 14:55:51 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Fri, 4 Aug 2023 07:55:50 -0700 List-Id: Porting FreeBSD to the PowerPC List-Archive: https://lists.freebsd.org/archives/freebsd-ppc List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ppc@freebsd.org X-BeenThere: freebsd-ppc@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US To: Justin Hibbits , Shawn Anastasio Cc: freebsd-ppc@FreeBSD.org, freebsd-hackers@FreeBSD.org, Timothy Pearson References: <0c24b4b7-b4c8-242d-6187-15b171c50c19@raptorengineering.com> <20230804093605.2a61eeed@ralga.knownspace> From: John Baldwin Subject: Re: Implementing in-kernel AES crypto acceleration on ppc (POWER8+) In-Reply-To: <20230804093605.2a61eeed@ralga.knownspace> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 8/4/23 6:36 AM, Justin Hibbits wrote: > Hello, > > Good to see this! I'll answer inline. > > On Thu, 3 Aug 2023 12:51:57 -0500 > Shawn Anastasio wrote: > >> Hello all, >> >> Raptor Engineering is interested in adding support for in-kernel AES >> acceleration on ppc64 via the VMX crypto instructions added in ISA >> 2.07B, and I wanted to reach out to the community with a few >> questions. > > I would love to see this added. > >> >> 1. As I understand it, FreeBSD already has support for in-kernel >> crypto acceleration on x86 and ARM via the aesni and armv8_crypto >> drivers respectively that each implement the cryptodev interface. Am I >> correct in understanding that adding AES acceleration for Power >> would just involve creating another driver here, or are there other >> pieces of the puzzle that I've missed? > > John Baldwin can probably answer this better, but I think your > understanding is correct. There might be some plumbing needed as well, > but that should be minimal. Recently for accelerated software crypto we have been using the existing assembly routines from OpenSSL (which has ppc routines IIRC) in the ossl(4) driver. For powerpc you would need to provide any ppc-specific things the OpenSSL assembly routines need (e.g. on x86 they use an array of words holding feature bits corresponding to output from cpuid), and mostly just add build glue. This driver also requires fpu_kern_*, but in general ossl(4) is preferred going forward and will eventually replace aesni and armv8crypto entirely. >> 2. I see that both the aesni and armv8 drivers make use of the >> fpu_kern_enter/fpu_kern_leave functions to guard access to vector >> registers, but it appears that these functions aren't implemented >> on ppc. Is that correct, or does an in-kernel facility for safely >> accessing vector registers on ppc already exist? > > Nope, ppc doesn't have these facilities yet. It shouldn't be hard to > implement, we just haven't done it yet. If you're interested in > implementing them, you should be able to model it after arm64, largely. Yes, this is a prerequisite. If you implement this you can also enable assembly for ZFS on powerpc as well. >> 3. For the accelerated AES implementation itself, I've noticed that >> cryptogams[*] contains an implementation that is both widely >> deployed (and thus tested and likely to be correct) and also BSD >> licensed. Would it be acceptable to import the relevant routines to >> the FreeBSD kernel and have the new cryptodev driver simply call into >> them, or are there other considerations involved? > > I think the right way to do that would be to import the code as-is as > third party code, and call into the routines that you need. You can > #ifdef out the unneeded bits, but try to keep it as intact as possible > from upstream. As mentioned above, I would prefer using the OpenSSL sources already in the tree via ossl(4). >> 4. Is there a userspace test framework for the cryptodev API that >> could be used to validate and benchmark the new implementation, or >> would I have to write that myself? It appears that OpenSSL had >> support for /dev/crypto at one point, but I'm not sure that is the >> case any longer. > > John Baldwin might have some ideas here, too. There is a cryptocheck tool in tools/tools/crypto that I tend to use. It generates "random" but deterministic (since it uses a fixed seed for libc's PRNG) tests of most of the algorithms supported by the cryptodev API with various key sizes, nonce sizes, payload, and AAD sizes. It performs each test once with OpenSSL's userland crypto and a second time with /dev/crypto and compares the results reporting any mismatches. There isn't a manpage, but there are some comments in the source. There are also some tests in the test suite that make use of the NIST known answer test vectors (which have to be installed via a nist-kat package). -- John Baldwin From nobody Fri Aug 4 16:13:42 2023 X-Original-To: freebsd-ppc@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RHW3t19xLz4Tq2m; Fri, 4 Aug 2023 16:13:46 +0000 (UTC) (envelope-from sanastasio@raptorengineering.com) Received: from raptorengineering.com (mail.raptorengineering.com [23.155.224.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RHW3s5PS1z3b4Z; Fri, 4 Aug 2023 16:13:45 +0000 (UTC) (envelope-from sanastasio@raptorengineering.com) Authentication-Results: mx1.freebsd.org; none Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 3BF6A828563C; Fri, 4 Aug 2023 11:13:44 -0500 (CDT) Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id RoeiEfZMSDX9; Fri, 4 Aug 2023 11:13:43 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 132348285659; Fri, 4 Aug 2023 11:13:43 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 132348285659 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD; t=1691165623; bh=iYMQtGDtrhMtXm8RFLhaJ9G9LL1lXuHVXruwfcBZUWU=; h=Message-ID:Date:MIME-Version:To:From; b=DAbml8koVUnmq202yDWkFhOwlVXdnxeZAdmcp6H2auGqDYmLTMUfJ+DHonVu/vFHX yuEk3+1d+6mwoEdPFO6aVagULmGHqyHGjMbG7DAgyGkzVsp5PNx6emFr6lz7exjpiE /v7+6RMqn7XjofIml/ReZt0zbv1FvNEmRM5/2688= X-Virus-Scanned: amavisd-new at rptsys.com Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id j5cfwuFE8iFX; Fri, 4 Aug 2023 11:13:42 -0500 (CDT) Received: from [10.11.0.2] (5.edge.rptsys.com [23.155.224.38]) by mail.rptsys.com (Postfix) with ESMTPSA id 97737828563C; Fri, 4 Aug 2023 11:13:42 -0500 (CDT) Message-ID: Date: Fri, 4 Aug 2023 11:13:42 -0500 List-Id: Porting FreeBSD to the PowerPC List-Archive: https://lists.freebsd.org/archives/freebsd-ppc List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ppc@freebsd.org X-BeenThere: freebsd-ppc@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux ppc64le; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: Implementing in-kernel AES crypto acceleration on ppc (POWER8+) Content-Language: en-US To: John Baldwin , Justin Hibbits Cc: freebsd-ppc@FreeBSD.org, freebsd-hackers@FreeBSD.org, Timothy Pearson References: <0c24b4b7-b4c8-242d-6187-15b171c50c19@raptorengineering.com> <20230804093605.2a61eeed@ralga.knownspace> From: Shawn Anastasio In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4RHW3s5PS1z3b4Z X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:46246, ipnet:23.155.224.0/24, country:US] On 8/4/23 9:55 AM, John Baldwin wrote: > On 8/4/23 6:36 AM, Justin Hibbits wrote: >> Hello, >> >> Good to see this! I'll answer inline. >> >> On Thu, 3 Aug 2023 12:51:57 -0500 >> Shawn Anastasio wrote: >> >>> Hello all, >>> >>> Raptor Engineering is interested in adding support for in-kernel AES >>> acceleration on ppc64 via the VMX crypto instructions added in ISA >>> 2.07B, and I wanted to reach out to the community with a few >>> questions. >> >> I would love to see this added. >> >>> >>> 1. As I understand it, FreeBSD already has support for in-kernel >>> crypto acceleration on x86 and ARM via the aesni and armv8_crypto >>> drivers respectively that each implement the cryptodev interface. Am I >>> correct in understanding that adding AES acceleration for Power >>> would just involve creating another driver here, or are there other >>> pieces of the puzzle that I've missed? >> >> John Baldwin can probably answer this better, but I think your >> understanding is correct. There might be some plumbing needed as well, >> but that should be minimal. > > Recently for accelerated software crypto we have been using the existing > assembly routines from OpenSSL (which has ppc routines IIRC) in the > ossl(4) driver. For powerpc you would need to provide any ppc-specific > things the OpenSSL assembly routines need (e.g. on x86 they use an array > of words holding feature bits corresponding to output from cpuid), and > mostly just add build glue. This driver also requires fpu_kern_*, but > in general ossl(4) is preferred going forward and will eventually > replace aesni and armv8crypto entirely. Thank you for pointing this out. It seems that sys/crypto/openssl conveniently already has the relevant ppc assembly files imported from the openssl source tree, so it looks like I'll just have to implement the ppc-specific openssl glue, in addition to the fpu_enter/leave functions. >>> 2. I see that both the aesni and armv8 drivers make use of the >>> fpu_kern_enter/fpu_kern_leave functions to guard access to vector >>> registers, but it appears that these functions aren't implemented >>> on ppc. Is that correct, or does an in-kernel facility for safely >>> accessing vector registers on ppc already exist? >> >> Nope, ppc doesn't have these facilities yet. It shouldn't be hard to >> implement, we just haven't done it yet. If you're interested in >> implementing them, you should be able to model it after arm64, largely. > > Yes, this is a prerequisite. If you implement this you can also enable > assembly for ZFS on powerpc as well. Makes sense, thank you both for the confirmation. I'll get started on mocking up an implementation of these functions. >>> 3. For the accelerated AES implementation itself, I've noticed that >>> cryptogams[*] contains an implementation that is both widely >>> deployed (and thus tested and likely to be correct) and also BSD >>> licensed. Would it be acceptable to import the relevant routines to >>> the FreeBSD kernel and have the new cryptodev driver simply call into >>> them, or are there other considerations involved? >> >> I think the right way to do that would be to import the code as-is as >> third party code, and call into the routines that you need. You can >> #ifdef out the unneeded bits, but try to keep it as intact as possible >> from upstream. > > As mentioned above, I would prefer using the OpenSSL sources already > in the tree via ossl(4). > Sounds good. >>> 4. Is there a userspace test framework for the cryptodev API that >>> could be used to validate and benchmark the new implementation, or >>> would I have to write that myself? It appears that OpenSSL had >>> support for /dev/crypto at one point, but I'm not sure that is the >>> case any longer. >> >> John Baldwin might have some ideas here, too. > > There is a cryptocheck tool in tools/tools/crypto that I tend to use. > It generates "random" but deterministic (since it uses a fixed seed > for libc's PRNG) tests of most of the algorithms supported by the > cryptodev API with various key sizes, nonce sizes, payload, and AAD > sizes. It performs each test once with OpenSSL's userland crypto > and a second time with /dev/crypto and compares the results reporting > any mismatches. There isn't a manpage, but there are some comments > in the source. > > There are also some tests in the test suite that make use of the > NIST known answer test vectors (which have to be installed via a nist-kat > package). Perfect, this seems like exactly what I was looking for. I'll begin work on the fpu_enter/leave functions and keep you all updated. Thank you both for the prompt and detailed responses! - Shawn