From owner-svn-src-head@freebsd.org Fri Mar 27 18:30:40 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9114827D3BF; Fri, 27 Mar 2020 18:30:40 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48pr5B1TXwz4VjY; Fri, 27 Mar 2020 18:30:38 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8DC8825ED5; Fri, 27 Mar 2020 18:25:25 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 02RIPP5X010864; Fri, 27 Mar 2020 18:25:25 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 02RIPOmR010857; Fri, 27 Mar 2020 18:25:24 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <202003271825.02RIPOmR010857@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Fri, 27 Mar 2020 18:25:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r359374 - in head: . share/man/man4 share/man/man7 share/man/man9 sys/crypto/aesni sys/crypto/armv8 sys/crypto/blake2 sys/crypto/ccp sys/crypto/via sys/dev/cesa sys/dev/cxgbe sys/dev/cx... X-SVN-Group: head X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: in head: . share/man/man4 share/man/man7 share/man/man9 sys/crypto/aesni sys/crypto/armv8 sys/crypto/blake2 sys/crypto/ccp sys/crypto/via sys/dev/cesa sys/dev/cxgbe sys/dev/cxgbe/crypto sys/dev/cxgbe/... X-SVN-Commit-Revision: 359374 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2020 18:30:41 -0000 Author: jhb Date: Fri Mar 27 18:25:23 2020 New Revision: 359374 URL: https://svnweb.freebsd.org/changeset/base/359374 Log: Refactor driver and consumer interfaces for OCF (in-kernel crypto). - The linked list of cryptoini structures used in session initialization is replaced with a new flat structure: struct crypto_session_params. This session includes a new mode to define how the other fields should be interpreted. Available modes include: - COMPRESS (for compression/decompression) - CIPHER (for simply encryption/decryption) - DIGEST (computing and verifying digests) - AEAD (combined auth and encryption such as AES-GCM and AES-CCM) - ETA (combined auth and encryption using encrypt-then-authenticate) Additional modes could be added in the future (e.g. if we wanted to support TLS MtE for AES-CBC in the kernel we could add a new mode for that. TLS modes might also affect how AAD is interpreted, etc.) The flat structure also includes the key lengths and algorithms as before. However, code doesn't have to walk the linked list and switch on the algorithm to determine which key is the auth key vs encryption key. The 'csp_auth_*' fields are always used for auth keys and settings and 'csp_cipher_*' for cipher. (Compression algorithms are stored in csp_cipher_alg.) - Drivers no longer register a list of supported algorithms. This doesn't quite work when you factor in modes (e.g. a driver might support both AES-CBC and SHA2-256-HMAC separately but not combined for ETA). Instead, a new 'crypto_probesession' method has been added to the kobj interface for symmteric crypto drivers. This method returns a negative value on success (similar to how device_probe works) and the crypto framework uses this value to pick the "best" driver. There are three constants for hardware (e.g. ccr), accelerated software (e.g. aesni), and plain software (cryptosoft) that give preference in that order. One effect of this is that if you request only hardware when creating a new session, you will no longer get a session using accelerated software. Another effect is that the default setting to disallow software crypto via /dev/crypto now disables accelerated software. Once a driver is chosen, 'crypto_newsession' is invoked as before. - Crypto operations are now solely described by the flat 'cryptop' structure. The linked list of descriptors has been removed. A separate enum has been added to describe the type of data buffer in use instead of using CRYPTO_F_* flags to make it easier to add more types in the future if needed (e.g. wired userspace buffers for zero-copy). It will also make it easier to re-introduce separate input and output buffers (in-kernel TLS would benefit from this). Try to make the flags related to IV handling less insane: - CRYPTO_F_IV_SEPARATE means that the IV is stored in the 'crp_iv' member of the operation structure. If this flag is not set, the IV is stored in the data buffer at the 'crp_iv_start' offset. - CRYPTO_F_IV_GENERATE means that a random IV should be generated and stored into the data buffer. This cannot be used with CRYPTO_F_IV_SEPARATE. If a consumer wants to deal with explicit vs implicit IVs, etc. it can always generate the IV however it needs and store partial IVs in the buffer and the full IV/nonce in crp_iv and set CRYPTO_F_IV_SEPARATE. The layout of the buffer is now described via fields in cryptop. crp_aad_start and crp_aad_length define the boundaries of any AAD. Previously with GCM and CCM you defined an auth crd with this range, but for ETA your auth crd had to span both the AAD and plaintext (and they had to be adjacent). crp_payload_start and crp_payload_length define the boundaries of the plaintext/ciphertext. Modes that only do a single operation (COMPRESS, CIPHER, DIGEST) should only use this region and leave the AAD region empty. If a digest is present (or should be generated), it's starting location is marked by crp_digest_start. Instead of using the CRD_F_ENCRYPT flag to determine the direction of the operation, cryptop now includes an 'op' field defining the operation to perform. For digests I've added a new VERIFY digest mode which assumes a digest is present in the input and fails the request with EBADMSG if it doesn't match the internally-computed digest. GCM and CCM already assumed this, and the new AEAD mode requires this for decryption. The new ETA mode now also requires this for decryption, so IPsec and GELI no longer do their own authentication verification. Simple DIGEST operations can also do this, though there are no in-tree consumers. To eventually support some refcounting to close races, the session cookie is now passed to crypto_getop() and clients should no longer set crp_sesssion directly. - Assymteric crypto operation structures should be allocated via crypto_getkreq() and freed via crypto_freekreq(). This permits the crypto layer to track open asym requests and close races with a driver trying to unregister while asym requests are in flight. - crypto_copyback, crypto_copydata, crypto_apply, and crypto_contiguous_subsegment now accept the 'crp' object as the first parameter instead of individual members. This makes it easier to deal with different buffer types in the future as well as separate input and output buffers. It's also simpler for driver writers to use. - bus_dmamap_load_crp() loads a DMA mapping for a crypto buffer. This understands the various types of buffers so that drivers that use DMA do not have to be aware of different buffer types. - Helper routines now exist to build an auth context for HMAC IPAD and OPAD. This reduces some duplicated work among drivers. - Key buffers are now treated as const throughout the framework and in device drivers. However, session key buffers provided when a session is created are expected to remain alive for the duration of the session. - GCM and CCM sessions now only specify a cipher algorithm and a cipher key. The redundant auth information is not needed or used. - For cryptosoft, split up the code a bit such that the 'process' callback now invokes a function pointer in the session. This function pointer is set based on the mode (in effect) though it simplifies a few edge cases that would otherwise be in the switch in 'process'. It does split up GCM vs CCM which I think is more readable even if there is some duplication. - I changed /dev/crypto to support GMAC requests using CRYPTO_AES_NIST_GMAC as an auth algorithm and updated cryptocheck to work with it. - Combined cipher and auth sessions via /dev/crypto now always use ETA mode. The COP_F_CIPHER_FIRST flag is now a no-op that is ignored. This was actually documented as being true in crypto(4) before, but the code had not implemented this before I added the CIPHER_FIRST flag. - I have not yet updated /dev/crypto to be aware of explicit modes for sessions. I will probably do that at some point in the future as well as teach it about IV/nonce and tag lengths for AEAD so we can support all of the NIST KAT tests for GCM and CCM. - I've split up the exising crypto.9 manpage into several pages of which many are written from scratch. - I have converted all drivers and consumers in the tree and verified that they compile, but I have not tested all of them. I have tested the following drivers: - cryptosoft - aesni (AES only) - blake2 - ccr and the following consumers: - cryptodev - IPsec - ktls_ocf - GELI (lightly) I have not tested the following: - ccp - aesni with sha - hifn - kgssapi_krb5 - ubsec - padlock - safe - armv8_crypto (aarch64) - glxsb (i386) - sec (ppc) - cesa (armv7) - cryptocteon (mips64) - nlmsec (mips64) Discussed with: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D23677 Added: head/share/man/man9/crypto_asym.9 (contents, props changed) head/share/man/man9/crypto_driver.9 (contents, props changed) head/share/man/man9/crypto_request.9 (contents, props changed) head/share/man/man9/crypto_session.9 (contents, props changed) Deleted: head/sys/opencrypto/cryptosoft.h Modified: head/ObsoleteFiles.inc head/share/man/man4/crypto.4 head/share/man/man7/crypto.7 head/share/man/man9/Makefile head/share/man/man9/bus_dma.9 head/share/man/man9/crypto.9 head/sys/crypto/aesni/aesni.c head/sys/crypto/aesni/aesni.h head/sys/crypto/aesni/aesni_wrap.c head/sys/crypto/armv8/armv8_crypto.c head/sys/crypto/blake2/blake2_cryptodev.c head/sys/crypto/ccp/ccp.c head/sys/crypto/ccp/ccp.h head/sys/crypto/ccp/ccp_hardware.c head/sys/crypto/via/padlock.c head/sys/crypto/via/padlock.h head/sys/crypto/via/padlock_cipher.c head/sys/crypto/via/padlock_hash.c head/sys/dev/cesa/cesa.c head/sys/dev/cesa/cesa.h head/sys/dev/cxgbe/adapter.h head/sys/dev/cxgbe/crypto/t4_crypto.c head/sys/dev/cxgbe/crypto/t4_keyctx.c head/sys/dev/cxgbe/tom/t4_tls.c head/sys/dev/glxsb/glxsb.c head/sys/dev/glxsb/glxsb.h head/sys/dev/glxsb/glxsb_hash.c head/sys/dev/hifn/hifn7751.c head/sys/dev/hifn/hifn7751var.h head/sys/dev/safe/safe.c head/sys/dev/safe/safevar.h head/sys/dev/sec/sec.c head/sys/dev/sec/sec.h head/sys/dev/ubsec/ubsec.c head/sys/dev/ubsec/ubsecvar.h head/sys/geom/eli/g_eli.c head/sys/geom/eli/g_eli.h head/sys/geom/eli/g_eli_crypto.c head/sys/geom/eli/g_eli_integrity.c head/sys/geom/eli/g_eli_privacy.c head/sys/kern/subr_bus_dma.c head/sys/kern/uipc_ktls.c head/sys/kgssapi/krb5/kcrypto_aes.c head/sys/kgssapi/krb5/kcrypto_des.c head/sys/kgssapi/krb5/kcrypto_des3.c head/sys/mips/cavium/cryptocteon/cavium_crypto.c head/sys/mips/cavium/cryptocteon/cryptocteon.c head/sys/mips/cavium/cryptocteon/cryptocteonvar.h head/sys/mips/nlm/dev/sec/nlmrsa.c head/sys/mips/nlm/dev/sec/nlmsec.c head/sys/mips/nlm/dev/sec/nlmseclib.c head/sys/mips/nlm/dev/sec/nlmseclib.h head/sys/mips/nlm/hal/nlmsaelib.h head/sys/netipsec/xform.h head/sys/netipsec/xform_ah.c head/sys/netipsec/xform_esp.c head/sys/netipsec/xform_ipcomp.c head/sys/opencrypto/criov.c head/sys/opencrypto/crypto.c head/sys/opencrypto/cryptodev.c head/sys/opencrypto/cryptodev.h head/sys/opencrypto/cryptodev_if.m head/sys/opencrypto/cryptosoft.c head/sys/opencrypto/ktls_ocf.c head/sys/opencrypto/xform_gmac.c head/sys/sys/bus_dma.h head/sys/sys/param.h head/tests/sys/opencrypto/cryptodev.py head/tests/sys/opencrypto/cryptodevh.py head/tests/sys/opencrypto/cryptotest.py head/tools/tools/crypto/cryptocheck.c Modified: head/ObsoleteFiles.inc ============================================================================== --- head/ObsoleteFiles.inc Fri Mar 27 18:20:36 2020 (r359373) +++ head/ObsoleteFiles.inc Fri Mar 27 18:25:23 2020 (r359374) @@ -36,6 +36,11 @@ # xargs -n1 | sort | uniq -d; # done +# 20200327: OCF refactoring +OLD_FILES+=usr/share/man/man9/crypto_find_driver.9 +OLD_FILES+=usr/share/man/man9/crypto_register.9 +OLD_FILES+=usr/share/man/man9/crypto_unregister.9 + # 20200323: INTERNALLIB don't install headers anymore OLD_FILES+=usr/include/libelftc.h OLD_FILES+=usr/include/libifconfig.h Modified: head/share/man/man4/crypto.4 ============================================================================== --- head/share/man/man4/crypto.4 Fri Mar 27 18:20:36 2020 (r359373) +++ head/share/man/man4/crypto.4 Fri Mar 27 18:25:23 2020 (r359374) @@ -60,7 +60,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 17, 2019 +.Dd March 27, 2020 .Dt CRYPTO 4 .Os .Sh NAME @@ -156,7 +156,7 @@ file desriptor. The symmetric-key operation mode provides a context-based API to traditional symmetric-key encryption (or privacy) algorithms, or to keyed and unkeyed one-way hash (HMAC and MAC) algorithms. -The symmetric-key mode also permits fused operation, +The symmetric-key mode also permits encrypt-then-authenticate fused operation, where the hardware performs both a privacy algorithm and an integrity-check algorithm in a single pass over the data: either a fused encrypt/HMAC-generate operation, or a fused HMAC-verify/decrypt operation. @@ -314,16 +314,14 @@ supplies the length of the input buffer; the fields .Fa cr_op-\*[Gt]iv supply the addresses of the input buffer, output buffer, one-way hash, and initialization vector, respectively. -If a session is using both a privacy algorithm and a hash algorithm, -the request will generate a hash of the input buffer before -generating the output buffer by default. -If the -.Dv COP_F_CIPHER_FIRST -flag is included in the -.Fa cr_op-\*[Gt]flags -field, -then the request will generate a hash of the output buffer after -executing the privacy algorithm. +.Pp +If a session is using either fused encrypt-then-authenticate or +an AEAD algorithm, +decryption operations require the associated hash as an input. +If the hash is incorrect, the +operation will fail with +.Dv EBADMSG +and the output buffer will remain unchanged. .It Dv CIOCCRYPTAEAD Fa struct crypt_aead *cr_aead .Bd -literal struct crypt_aead { Modified: head/share/man/man7/crypto.7 ============================================================================== --- head/share/man/man7/crypto.7 Fri Mar 27 18:20:36 2020 (r359373) +++ head/share/man/man7/crypto.7 Fri Mar 27 18:25:23 2020 (r359374) @@ -27,7 +27,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 2, 2015 +.Dd March 27, 2020 .Dt CRYPTO 7 .Os .Sh NAME @@ -68,19 +68,13 @@ This algorithm implements Cipher-block chaining. .El .Pp This algorithm implements Galois/Counter Mode. -This is the cipher part of an AEAD +This cipher uses AEAD .Pq Authenticated Encryption with Associated Data mode. -This requires use of the use of a proper authentication mode, one of -.Dv CRYPTO_AES_128_NIST_GMAC , -.Dv CRYPTO_AES_192_NIST_GMAC -or -.Dv CRYPTO_AES_256_NIST_GMAC , -that corresponds with the number of bits in the key that you are using. .Pp -The associated data (if any) must be provided by the authentication mode op. -The authentication tag will be read/written from/to the offset crd_inject -specified in the descriptor for the authentication mode. +The authentication tag will be read/written from/to the offset +.Va crp_digest_start +specified in the request. .Pp Note: You must provide an IV on every call. .It Dv CRYPTO_AES_ICM @@ -117,22 +111,6 @@ as defined in NIST SP 800-38E. .Pp NOTE: The ciphertext stealing part is not implemented which is why this cipher is listed as having a block size of 16 instead of 1. -.El -.Pp -Authentication algorithms: -.Bl -tag -width ".Dv CRYPTO_AES_256_NIST_GMAC" -.It CRYPTO_AES_128_NIST_GMAC -See -.Dv CRYPTO_AES_NIST_GCM_16 -in the cipher mode section. -.It CRYPTO_AES_192_NIST_GMAC -See -.Dv CRYPTO_AES_NIST_GCM_16 -in the cipher mode section. -.It CRYPTO_AES_256_NIST_GMAC -See -.Dv CRYPTO_AES_NIST_GCM_16 -in the cipher mode section. .El .Sh SEE ALSO .Xr crypto 4 , Modified: head/share/man/man9/Makefile ============================================================================== --- head/share/man/man9/Makefile Fri Mar 27 18:20:36 2020 (r359373) +++ head/share/man/man9/Makefile Fri Mar 27 18:25:23 2020 (r359374) @@ -71,6 +71,10 @@ MAN= accept_filter.9 \ cr_seeothergids.9 \ cr_seeotheruids.9 \ crypto.9 \ + crypto_asym.9 \ + crypto_driver.9 \ + crypto_request.9 \ + crypto_session.9 \ CTASSERT.9 \ DB_COMMAND.9 \ DECLARE_GEOM_CLASS.9 \ @@ -889,20 +893,33 @@ MLINKS+=cpuset.9 CPUSET_T_INITIALIZER.9 \ cpuset.9 CPU_COPY_STORE_REL.9 MLINKS+=critical_enter.9 critical.9 \ critical_enter.9 critical_exit.9 -MLINKS+=crypto.9 crypto_dispatch.9 \ - crypto.9 crypto_done.9 \ - crypto.9 crypto_freereq.9 \ - crypto.9 crypto_freesession.9 \ - crypto.9 crypto_get_driverid.9 \ - crypto.9 crypto_getreq.9 \ - crypto.9 crypto_kdispatch.9 \ - crypto.9 crypto_kdone.9 \ - crypto.9 crypto_kregister.9 \ - crypto.9 crypto_newsession.9 \ - crypto.9 crypto_register.9 \ - crypto.9 crypto_unblock.9 \ - crypto.9 crypto_unregister.9 \ - crypto.9 crypto_unregister_all.9 +MLINKS+=crypto_asym.9 crypto_kdispatch.9 \ + crypto_asym.9 crypto_kdone.9 \ + crypto_asym.9 crypto_kregister.9 \ + crypto_asym.9 CRYPTODEV_KPROCESS.9 +MLINKS+=crypto_driver.9 crypto_apply.9 \ + crypto_driver.9 crypto_contiguous_segment.9 \ + crypto_driver.9 crypto_copyback.9 \ + crypto_driver.9 crypto_copydata.9 \ + crypto_driver.9 crypto_done.9 \ + crypto_driver.9 crypto_get_driverid.9 \ + crypto_driver.9 crypto_get_driver_session.9 \ + crypto_driver.9 crypto_unblock.9 \ + crypto_driver.9 crypto_unregister_all.9 \ + crypto_driver.9 CRYPTODEV_FREESESSION.9 \ + crypto_driver.9 CRYPTODEV_NEWSESSION.9 \ + crypto_driver.9 CRYPTODEV_PROBESESSION.9 \ + crypto_driver.9 CRYPTODEV_PROCESS.9 \ + crypto_driver.9 hmac_init_ipad.9 \ + crypto_driver.9 hmac_init_opad.9 +MLINKS+=crypto_request.9 crypto_dispatch.9 \ + crypto_request.9 crypto_freereq.9 \ + crypto_request.9 crypto_getreq.9 +MLINKS+=crypto_session.9 crypto_auth_hash.9 \ + crypto_session.9 crypto_cipher.9 \ + crypto_session.9 crypto_get_params.9 \ + crypto_session.9 crypto_newsession.9 \ + crypto_session.9 crypto_freesession.9 MLINKS+=DB_COMMAND.9 DB_SHOW_ALL_COMMAND.9 \ DB_COMMAND.9 DB_SHOW_COMMAND.9 MLINKS+=DECLARE_MODULE.9 DECLARE_MODULE_TIED.9 Modified: head/share/man/man9/bus_dma.9 ============================================================================== --- head/share/man/man9/bus_dma.9 Fri Mar 27 18:20:36 2020 (r359373) +++ head/share/man/man9/bus_dma.9 Fri Mar 27 18:25:23 2020 (r359374) @@ -53,7 +53,7 @@ .\" $FreeBSD$ .\" $NetBSD: bus_dma.9,v 1.25 2002/10/14 13:43:16 wiz Exp $ .\" -.Dd August 11, 2018 +.Dd March 27, 2020 .Dt BUS_DMA 9 .Os .Sh NAME @@ -68,6 +68,7 @@ .Nm bus_dmamap_load , .Nm bus_dmamap_load_bio , .Nm bus_dmamap_load_ccb , +.Nm bus_dmamap_load_crp , .Nm bus_dmamap_load_mbuf , .Nm bus_dmamap_load_mbuf_sg , .Nm bus_dmamap_load_uio , @@ -118,6 +119,10 @@ "union ccb *ccb" "bus_dmamap_callback_t *callback" "void *callback_arg" \ "int flags" .Ft int +.Fn bus_dmamap_load_crp "bus_dma_tag_t dmat" "bus_dmamap_t map" \ +"struct crypto *crp" "bus_dmamap_callback_t *callback" "void *callback_arg" \ +"int flags" +.Ft int .Fn bus_dmamap_load_mbuf "bus_dma_tag_t dmat" "bus_dmamap_t map" \ "struct mbuf *mbuf" "bus_dmamap_callback2_t *callback" "void *callback_arg" \ "int flags" @@ -387,9 +392,10 @@ the load of a .Vt bus_dmamap_t via .Fn bus_dmamap_load , -.Fn bus_dmamap_load_bio +.Fn bus_dmamap_load_bio , +.Fn bus_dmamap_load_ccb , or -.Fn bus_dmamap_load_ccb . +.Fn bus_dmamap_load_crp . Callbacks are of the format: .Bl -tag -width indent .It Ft void @@ -879,6 +885,15 @@ XPT_CONT_TARGET_IO .It XPT_SCSI_IO .El +.It Fn bus_dmamap_load_crp "dmat" "map" "crp" "callback" "callback_arg" "flags" +This is a variation of +.Fn bus_dmamap_load +which maps buffers pointed to by +.Fa crp +for DMA transfers. +The +.Dv BUS_DMA_NOWAIT +flag is implied, thus no callback deferral will happen. .It Fn bus_dmamap_load_mbuf "dmat" "map" "mbuf" "callback2" "callback_arg" \ "flags" This is a variation of Modified: head/share/man/man9/crypto.9 ============================================================================== --- head/share/man/man9/crypto.9 Fri Mar 27 18:20:36 2020 (r359373) +++ head/share/man/man9/crypto.9 Fri Mar 27 18:25:23 2020 (r359374) @@ -17,7 +17,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 17, 2019 +.Dd March 27, 2020 .Dt CRYPTO 9 .Os .Sh NAME @@ -25,120 +25,50 @@ .Nd API for cryptographic services in the kernel .Sh SYNOPSIS .In opencrypto/cryptodev.h -.Ft int32_t -.Fn crypto_get_driverid "device_t dev" "size_t session_size" "int flags" -.Ft int -.Fn crypto_register "uint32_t driverid" "int alg" "uint16_t maxoplen" "uint32_t flags" -.Ft int -.Fn crypto_kregister "uint32_t driverid" "int kalg" "uint32_t flags" -.Ft int -.Fn crypto_unregister "uint32_t driverid" "int alg" -.Ft int -.Fn crypto_unregister_all "uint32_t driverid" -.Ft void -.Fn crypto_done "struct cryptop *crp" -.Ft void -.Fn crypto_kdone "struct cryptkop *krp" -.Ft int -.Fn crypto_find_driver "const char *match" -.Ft int -.Fn crypto_newsession "crypto_session_t *cses" "struct cryptoini *cri" "int crid" -.Ft int -.Fn crypto_freesession "crypto_session_t cses" -.Ft int -.Fn crypto_dispatch "struct cryptop *crp" -.Ft int -.Fn crypto_kdispatch "struct cryptkop *krp" -.Ft int -.Fn crypto_unblock "uint32_t driverid" "int what" -.Ft "struct cryptop *" -.Fn crypto_getreq "int num" -.Ft void -.Fn crypto_freereq "struct cryptop *crp" -.Bd -literal -#define CRYPTO_SYMQ 0x1 -#define CRYPTO_ASYMQ 0x2 - -#define EALG_MAX_BLOCK_LEN 16 - -struct cryptoini { - int cri_alg; - int cri_klen; - int cri_mlen; - caddr_t cri_key; - uint8_t cri_iv[EALG_MAX_BLOCK_LEN]; - struct cryptoini *cri_next; -}; - -struct cryptodesc { - int crd_skip; - int crd_len; - int crd_inject; - int crd_flags; - struct cryptoini CRD_INI; -#define crd_iv CRD_INI.cri_iv -#define crd_key CRD_INI.cri_key -#define crd_alg CRD_INI.cri_alg -#define crd_klen CRD_INI.cri_klen - struct cryptodesc *crd_next; -}; - -struct cryptop { - TAILQ_ENTRY(cryptop) crp_next; - crypto_session_t crp_session; - int crp_ilen; - int crp_olen; - int crp_etype; - int crp_flags; - caddr_t crp_buf; - caddr_t crp_opaque; - struct cryptodesc *crp_desc; - int (*crp_callback) (struct cryptop *); - caddr_t crp_mac; -}; - -struct crparam { - caddr_t crp_p; - u_int crp_nbits; -}; - -#define CRK_MAXPARAM 8 - -struct cryptkop { - TAILQ_ENTRY(cryptkop) krp_next; - u_int krp_op; /* ie. CRK_MOD_EXP or other */ - u_int krp_status; /* return status */ - u_short krp_iparams; /* # of input parameters */ - u_short krp_oparams; /* # of output parameters */ - uint32_t krp_hid; - struct crparam krp_param[CRK_MAXPARAM]; - int (*krp_callback)(struct cryptkop *); -}; -.Ed .Sh DESCRIPTION .Nm -is a framework for drivers of cryptographic hardware to register with -the kernel so -.Dq consumers -(other kernel subsystems, and -users through the +is a framework for in-kernel cryptography. +It permits in-kernel consumers to encrypt and decrypt data +and also enables userland applications to use cryptographic hardware +through the .Pa /dev/crypto -device) are able to make use of it. -Drivers register with the framework the algorithms they support, -and provide entry points (functions) the framework may call to -establish, use, and tear down sessions. -Sessions are used to cache cryptographic information in a particular driver -(or associated hardware), so initialization is not needed with every request. -Consumers of cryptographic services pass a set of -descriptors that instruct the framework (and the drivers registered -with it) of the operations that should be applied on the data (more -than one cryptographic operation can be requested). +device. .Pp -Keying operations are supported as well. -Unlike the symmetric operators described above, -these sessionless commands perform mathematical operations using -input and output parameters. +.Nm +supports two modes of operation: +one mode for symmetric-keyed cryptographic requests and digest, +and a second mode for asymmetric-key requests and modular arithmetic. +.Ss Symmetric-Key Mode +Symmetric-key operations include encryption and decryption operations +using block and stream ciphers as well as computation and verification +of message authentication codes (MACs). +In this mode, +consumers allocate sessions to describe a transform as discussed in +.Xr crypto_session 9 . +Consumers then allocate request objects to describe each transformation +such as encrypting a network packet or decrypting a disk sector. +Requests are described in +.Xr crypto_request 9 . .Pp +Device drivers are responsible for processing requests submitted by +consumers. +.Xr crypto_driver 9 +describes the interfaces drivers use to register with the framework, +helper routines the framework provides to faciliate request processing, +and the interfaces drivers are required to provide. +.Ss Asymmetric-Key Mode +Assymteric-key operations do not use sessions. +Instead, +these operations perform individual mathematical operations using a set +of input and output parameters. +These operations are described in +.Xr crypto_asym 9 . +Drivers that support asymmetric operations use additional interfaces +described in +.Xr crypto_asym 9 +in addition to the base interfaces described in +.Xr crypto_driver 9 . +.Ss Callbacks Since the consumers may not be associated with a process, drivers may not .Xr sleep 9 . @@ -148,88 +78,38 @@ to notify a consumer that a request has been completed callback is specified by the consumer on a per-request basis). The callback is invoked by the framework whether the request was successfully completed or not. -An error indication is provided in the latter case. -A specific error code, +Errors are reported to the callback function. +.Pp +Session initialization does not use callbacks and returns errors +synchronously. +.Ss Session Migration +For symmetric-key operations, +a specific error code, .Er EAGAIN , is used to indicate that a session handle has changed and that the request may be re-submitted immediately with the new session. -Errors are only returned to the invoking function if not -enough information to call the callback is available (meaning, there -was a fatal error in verifying the arguments). -For session initialization and teardown no callback mechanism is used. +The consumer should update its saved copy of the session handle +to the value of +.Fa crp_session +so that future requests use the new session. +.Ss Supported Algorithms +More details on some algorithms may be found in +.Xr crypto 7 . +These algorithms are used for symmetric-mode operations. +Asymmetric-mode operations support operations described in +.Xr crypto_asym 9 . .Pp -The -.Fn crypto_find_driver -returns the driver id of the device whose name matches -.Fa match . -.Fa match -can either be the exact name of a device including the unit -or the driver name without a unit. -In the latter case, -the id of the first device with the matching driver name is returned. -If no matching device is found, -the value -1 is returned. +The following authentication algorithms are supported: .Pp -The -.Fn crypto_newsession -routine is called by consumers of cryptographic services (such as the -.Xr ipsec 4 -stack) that wish to establish a new session with the framework. -The -.Fa cri -argument points to a -.Vt cryptoini -structure containing all the necessary information for -the driver to establish the session. -The -.Fa crid -argument is either a specific driver id or a bitmask of flags. -The flags are -.Dv CRYPTOCAP_F_HARDWARE , -to select hardware devices, -or -.Dv CRYPTOCAP_F_SOFTWARE , -to select software devices. -If both are specified, hardware devices are preferred over software -devices. -On success, the opaque session handle of the new session will be stored in -.Fa *cses . -The -.Vt cryptoini -structure pointed to by -.Fa cri -contains these fields: -.Bl -tag -width ".Va cri_next" -.It Va cri_alg -An algorithm identifier. -Currently supported algorithms are: -.Pp -.Bl -tag -width ".Dv CRYPTO_RIPEMD160_HMAC" -compact -.It Dv CRYPTO_AES_128_NIST_GMAC -.It Dv CRYPTO_AES_192_NIST_GMAC -.It Dv CRYPTO_AES_256_NIST_GMAC -.It Dv CRYPTO_AES_CBC -.It Dv CRYPTO_AES_CCM_16 +.Bl -tag -offset indent -width CRYPTO_AES_CCM_CBC_MAC -compact .It Dv CRYPTO_AES_CCM_CBC_MAC -.It Dv CRYPTO_AES_ICM -.It Dv CRYPTO_AES_NIST_GCM_16 .It Dv CRYPTO_AES_NIST_GMAC -.It Dv CRYPTO_AES_XTS -.It Dv CRYPTO_ARC4 .It Dv CRYPTO_BLAKE2B .It Dv CRYPTO_BLAKE2S -.It Dv CRYPTO_BLF_CBC -.It Dv CRYPTO_CAMELLIA_CBC -.It Dv CRYPTO_CAST_CBC -.It Dv CRYPTO_CHACHA20 -.It Dv CRYPTO_DEFLATE_COMP -.It Dv CRYPTO_DES_CBC -.It Dv CRYPTO_3DES_CBC .It Dv CRYPTO_MD5 .It Dv CRYPTO_MD5_HMAC .It Dv CRYPTO_MD5_KPDK .It Dv CRYPTO_NULL_HMAC -.It Dv CRYPTO_NULL_CBC .It Dv CRYPTO_POLY1305 .It Dv CRYPTO_RIPEMD160 .It Dv CRYPTO_RIPEMD160_HMAC @@ -244,488 +124,38 @@ Currently supported algorithms are: .It Dv CRYPTO_SHA2_384_HMAC .It Dv CRYPTO_SHA2_512 .It Dv CRYPTO_SHA2_512_HMAC -.It Dv CRYPTO_SKIPJACK_CBC .El -.It Va cri_klen -For variable-size key algorithms, the length of the key in bits. -.It Va cri_mlen -If non-zero, truncate the calculated hash to this many bytes. -.It Va cri_key -The key to be used. -.It Va cri_iv -An explicit initialization vector if it does not prefix -the data. -This field is ignored during initialization -.Pq Nm crypto_newsession . -If no IV is explicitly passed (see below on details), a random IV is used -by the device driver processing the request. -.It Va cri_next -Pointer to another -.Vt cryptoini -structure. -This is used to establish dual-algorithm sessions, such as combining a -cipher with a MAC. -.El .Pp -The -.Vt cryptoini -structure and its contents will not be modified or referenced by the -framework or any cryptographic drivers. -The memory associated with -.Fa cri -can be released once -.Fn crypto_newsession -returns. +The following encryption algorithms are supported: .Pp -.Fn crypto_freesession -is called with the session handle returned by -.Fn crypto_newsession -to free the session. -.Pp -.Fn crypto_dispatch -is called to process a request. -The various fields in the -.Vt cryptop -structure are: -.Bl -tag -width ".Va crp_callback" -.It Va crp_session -The session handle. -.It Va crp_ilen -The total length in bytes of the buffer to be processed. -.It Va crp_olen -On return, contains the total length of the result. -For symmetric crypto operations, this will be the same as the input length. -This will be used if the framework needs to allocate a new -buffer for the result (or for re-formatting the input). -.It Va crp_callback -Callback routine invoked when a request is completed via -.Fn crypto_done . -The callback routine should inspect the -.Va crp_etype -to determine if the request was successfully completed. -.It Va crp_etype -The error type, if any errors were encountered, or zero if -the request was successfully processed. -If the -.Er EAGAIN -error code is returned, the session handle has changed (and has been recorded -in the -.Va crp_session -field). -The consumer should record the new session handle and use it in all subsequent -requests. -In this case, the request may be re-submitted immediately. -This mechanism is used by the framework to perform -session migration (move a session from one driver to another, because -of availability, performance, or other considerations). -.Pp -This field is only valid in the context of the callback routine specified by -.Va crp_callback . -Errors are returned to the invoker of -.Fn crypto_process -only when enough information is not present to call the callback -routine (i.e., if the pointer passed is -.Dv NULL -or if no callback routine was specified). -.It Va crp_flags -A bitmask of flags associated with this request. -Currently defined flags are: -.Bl -tag -width ".Dv CRYPTO_F_CBIFSYNC" -.It Dv CRYPTO_F_IMBUF -The buffer is an mbuf chain pointed to by -.Va crp_mbuf . -.It Dv CRYPTO_F_IOV -The buffer is a -.Vt uio -structure pointed to by -.Va crp_uio . -.It Dv CRYPTO_F_BATCH -Batch operation if possible. -.It Dv CRYPTO_F_CBIMM -Do callback immediately instead of doing it from a dedicated kernel thread. -.It Dv CRYPTO_F_DONE -Operation completed. -.It Dv CRYPTO_F_CBIFSYNC -Do callback immediately if operation is synchronous (that the driver -specified the -.Dv CRYPTOCAP_F_SYNC -flag). -.It Dv CRYPTO_F_ASYNC -Try to do the crypto operation in a pool of workers -if the operation is synchronous (that is, if the driver specified the -.Dv CRYPTOCAP_F_SYNC -flag). -It aims to speed up processing by dispatching crypto operations -on different processors. -.It Dv CRYPTO_F_ASYNC_KEEPORDER -Dispatch callbacks in the same order they are posted. -Only relevant if the -.Dv CRYPTO_F_ASYNC -flag is set and if the operation is synchronous. +.Bl -tag -offset indent -width CRYPTO_CAMELLIA_CBC -compact +.It Dv CRYPTO_AES_CBC +.It Dv CRYPTO_AES_ICM +.It Dv CRYPTO_AES_XTS +.It Dv CRYPTO_ARC4 +.It Dv CRYPTO_BLF_CBC +.It Dv CRYPTO_CAMELLIA_CBC +.It Dv CRYPTO_CAST_CBC +.It Dv CRYPTO_CHACHA20 +.It Dv CRYPTO_DES_CBC +.It Dv CRYPTO_3DES_CBC +.It Dv CRYPTO_NULL_CBC +.It Dv CRYPTO_SKIPJACK_CBC .El -.It Va crp_buf -Data buffer unless -.Dv CRYPTO_F_IMBUF -or -.Dv CRYPTO_F_IOV -is set in -.Va crp_flags . -The length in bytes is set in -.Va crp_ilen . -.It Va crp_mbuf -Data buffer mbuf chain when -.Dv CRYPTO_F_IMBUF -is set in -.Va crp_flags . -.It Va crp_uio -.Vt struct uio -data buffer when -.Dv CRYPTO_F_IOV -is set in -.Va crp_flags . -.It Va crp_opaque -Cookie passed through the crypto framework untouched. -It is -intended for the invoking application's use. -.It Va crp_desc -A linked list of descriptors. -Each descriptor provides -information about what type of cryptographic operation should be done -on the input buffer. -The various fields are: -.Bl -tag -width ".Va crd_inject" -.It Va crd_iv -When the flag -.Dv CRD_F_IV_EXPLICIT -is set, this field contains the IV. -.It Va crd_key -When the -.Dv CRD_F_KEY_EXPLICIT -flag is set, the -.Va crd_key -points to a buffer with encryption or authentication key. -.It Va crd_alg -An algorithm to use. -Must be the same as the one given at newsession time. -.It Va crd_klen -The -.Va crd_key -key length. -.It Va crd_skip -The offset in the input buffer where processing should start. -.It Va crd_len -How many bytes, after -.Va crd_skip , -should be processed. -.It Va crd_inject -The -.Va crd_inject -field specifies an offset in bytes from the beginning of the buffer. -For encryption algorithms, this may be where the IV will be inserted -when encrypting or where the IV may be found for -decryption (subject to -.Va crd_flags ) . -For MAC algorithms, this is where the result of the keyed hash will be -inserted. -.It Va crd_flags -The following flags are defined: -.Bl -tag -width 3n -.It Dv CRD_F_ENCRYPT -For encryption algorithms, this bit is set when encryption is required -(when not set, decryption is performed). -.It Dv CRD_F_IV_PRESENT -.\" This flag name has nothing to do w/ it's behavior, fix the name. -For encryption, if this bit is not set the IV used to encrypt the packet -will be written at the location pointed to by -.Va crd_inject . -The IV length is assumed to be equal to the blocksize of the -encryption algorithm. -For encryption, if this bit is set, nothing is done. -For decryption, this flag has no meaning. -Applications that do special -.Dq "IV cooking" , -such as the half-IV mode in -.Xr ipsec 4 , -can use this flag to indicate that the IV should not be written on the packet. -This flag is typically used in conjunction with the -.Dv CRD_F_IV_EXPLICIT -flag. -.It Dv CRD_F_IV_EXPLICIT -This bit is set when the IV is explicitly -provided by the consumer in the -.Va crd_iv -field. -Otherwise, for encryption operations the IV is provided for by -the driver used to perform the operation, whereas for decryption -operations the offset of the IV is provided by the -.Va crd_inject -field. -This flag is typically used when the IV is calculated -.Dq "on the fly" -by the consumer, and does not precede the data. -.It Dv CRD_F_KEY_EXPLICIT -For encryption and authentication (MAC) algorithms, this bit is set when the key -is explicitly provided by the consumer in the -.Va crd_key -field for the given operation. -Otherwise, the key is taken at newsession time from the -.Va cri_key -field. -As calculating the key schedule may take a while, it is recommended that often -used keys are given their own session. -.It Dv CRD_F_COMP -For compression algorithms, this bit is set when compression is required (when -not set, decompression is performed). -.El -.It Va CRD_INI -This -.Vt cryptoini -structure will not be modified by the framework or the device drivers. -Since this information accompanies every cryptographic -operation request, drivers may re-initialize state on-demand -(typically an expensive operation). -Furthermore, the cryptographic -framework may re-route requests as a result of full queues or hardware -failure, as described above. -.It Va crd_next -Point to the next descriptor. -Linked operations are useful in protocols such as -.Xr ipsec 4 , -where multiple cryptographic transforms may be applied on the same -block of data. -.El -.El .Pp -.Fn crypto_getreq -allocates a -.Vt cryptop -structure with a linked list of -.Fa num -.Vt cryptodesc -structures. +The following authenticated encryption with additional data (AEAD) +algorithms are supported: .Pp -.Fn crypto_freereq -deallocates a structure -.Vt cryptop -and any -.Vt cryptodesc -structures linked to it. -Note that it is the responsibility of the -callback routine to do the necessary cleanups associated with the -opaque field in the -.Vt cryptop -structure. -.Pp -.Fn crypto_kdispatch -is called to perform a keying operation. -The various fields in the -.Vt cryptkop -structure are: -.Bl -tag -width ".Va krp_callback" -.It Va krp_op -Operation code, such as -.Dv CRK_MOD_EXP . -.It Va krp_status -Return code. -This -.Va errno Ns -style -variable indicates whether lower level reasons -for operation failure. -.It Va krp_iparams -Number of input parameters to the specified operation. -Note that each operation has a (typically hardwired) number of such parameters. -.It Va krp_oparams -Number of output parameters from the specified operation. -Note that each operation has a (typically hardwired) number of such parameters. -.It Va krp_kvp -An array of kernel memory blocks containing the parameters. -.It Va krp_hid -Identifier specifying which low-level driver is being used. -.It Va krp_callback -Callback called on completion of a keying operation. +.Bl -tag -offset indent -width CRYPTO_AES_NIST_GCM_16 -compact +.It Dv CRYPTO_AES_CCM_16 +.It Dv CRYPTO_AES_NIST_GCM_16 .El -.Sh DRIVER-SIDE API -The -.Fn crypto_get_driverid , -.Fn crypto_get_driver_session , -.Fn crypto_register , -.Fn crypto_kregister , -.Fn crypto_unregister , -.Fn crypto_unblock , -and -.Fn crypto_done -routines are used by drivers that provide support for cryptographic -primitives to register and unregister with the kernel crypto services -framework. .Pp -Drivers must first use the -.Fn crypto_get_driverid -function to acquire a driver identifier, specifying the -.Fa flags -as an argument. -One of -.Dv CRYPTOCAP_F_SOFTWARE -or -.Dv CRYPTOCAP_F_HARDWARE -must be specified. -The -.Dv CRYPTOCAP_F_SYNC -may also be specified, and should be specified if the driver does all of -it's operations synchronously. -Drivers must pass the size of their session structure as the second argument. -An appropriately sized memory will be allocated by the framework, zeroed, and -passed to the driver's -.Fn newsession -method. +The following compression algorithms are supported: .Pp -For each algorithm the driver supports, it must then call -.Fn crypto_register . -The first two arguments are the driver and algorithm identifiers. -The next two arguments specify the largest possible operator length (in bits, -important for public key operations) and flags for this algorithm. -.Pp -.Fn crypto_unregister -is called by drivers that wish to withdraw support for an algorithm. -The two arguments are the driver and algorithm identifiers, respectively. -Typically, drivers for -PCMCIA -crypto cards that are being ejected will invoke this routine for all -algorithms supported by the card. -.Fn crypto_unregister_all -will unregister all algorithms registered by a driver -and the driver will be disabled (no new sessions will be allocated on -that driver, and any existing sessions will be migrated to other -drivers). -The same will be done if all algorithms associated with a driver are -unregistered one by one. -After a call to -.Fn crypto_unregister_all -there will be no threads in either the newsession or freesession function -of the driver. -.Pp -The calling convention for the driver-supplied routines are: -.Pp -.Bl -item -compact -.It -.Ft int -.Fn \*[lp]*newsession\*[rp] "device_t" "crypto_session_t" "struct cryptoini *" ; -.It -.Ft void -.Fn \*[lp]*freesession\*[rp] "device_t" "crypto_session_t" ; -.It -.Ft int -.Fn \*[lp]*process\*[rp] "device_t" "struct cryptop *" "int" ; -.It -.Ft int -.Fn \*[lp]*kprocess\*[rp] "device_t" "struct cryptkop *" "int" ; +.Bl -tag -offset indent -width CRYPTO_DEFLATE_COMP -compact +.It Dv CRYPTO_DEFLATE_COMP .El -.Pp -On invocation, the first argument to -all routines is the -.Fa device_t -that was provided to -.Fn crypto_get_driverid . -The second argument to -.Fn newsession -is the opaque session handle for the new session. -The third argument is identical to that of -.Fn crypto_newsession . -.Pp -Drivers obtain a pointer to their session memory by invoking -.Fn crypto_get_driver_session -on the opaque -.Vt crypto_session_t -handle. -.Pp -The -.Fn freesession -routine takes as arguments the opaque data value and the session handle. -It should clear any context associated with the session (clear hardware -registers, memory, etc.). -If no resources need to be released other than the contents of session memory, -the method is optional. -The -.Nm -framework will zero and release the allocated session memory (after running the -.Fn freesession -method, if one exists). -.Pp -The -.Fn process -routine is invoked with a request to perform crypto processing. -This routine must not block or sleep, but should queue the request and return -immediately or process the request to completion. -In case of an unrecoverable error, the error indication must be placed in the -.Va crp_etype -field of the -.Vt cryptop -structure. -When the request is completed, or an error is detected, the -.Fn process -routine must invoke -.Fn crypto_done . -Session migration may be performed, as mentioned previously. -.Pp -In case of a temporary resource exhaustion, the -.Fn process -routine may return -.Er ERESTART -in which case the crypto services will requeue the request, mark the driver -as -.Dq blocked , -and stop submitting requests for processing. -The driver is then responsible for notifying the crypto services -when it is again able to process requests through the -.Fn crypto_unblock -routine. -This simple flow control mechanism should only be used for short-lived -resource exhaustion as it causes operations to be queued in the crypto -layer. -Doing so is preferable to returning an error in such cases as -it can cause network protocols to degrade performance by treating the -failure much like a lost packet. -.Pp -The -.Fn kprocess -routine is invoked with a request to perform crypto key processing. -This routine must not block, but should queue the request and return -immediately. -Upon processing the request, the callback routine should be invoked. -In case of an unrecoverable error, the error indication must be placed in the -.Va krp_status -field of the -.Vt cryptkop -structure. -When the request is completed, or an error is detected, the -.Fn kprocess -routine should invoked -.Fn crypto_kdone . -.Sh RETURN VALUES -.Fn crypto_register , *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***