From owner-svn-src-head@freebsd.org Tue Jun 23 00:02:28 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C2D1334033E; Tue, 23 Jun 2020 00:02:28 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49rRKw4s0vz4bdL; Tue, 23 Jun 2020 00:02:28 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A1C501673B; Tue, 23 Jun 2020 00:02:28 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05N02SQF046211; Tue, 23 Jun 2020 00:02:28 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05N02Sum046210; Tue, 23 Jun 2020 00:02:28 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <202006230002.05N02Sum046210@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Tue, 23 Jun 2020 00:02:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r362523 - head/sys/opencrypto X-SVN-Group: head X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: head/sys/opencrypto X-SVN-Commit-Revision: 362523 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2020 00:02:28 -0000 Author: jhb Date: Tue Jun 23 00:02:28 2020 New Revision: 362523 URL: https://svnweb.freebsd.org/changeset/base/362523 Log: Store the AAD in a separate buffer for KTLS. For TLS 1.2 this permits reusing one of the existing iovecs without always having to duplicate both. While here, only duplicate the output iovec for TLS 1.3 if it will be used. Reviewed by: gallatin Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D25291 Modified: head/sys/opencrypto/ktls_ocf.c Modified: head/sys/opencrypto/ktls_ocf.c ============================================================================== --- head/sys/opencrypto/ktls_ocf.c Mon Jun 22 23:55:06 2020 (r362522) +++ head/sys/opencrypto/ktls_ocf.c Tue Jun 23 00:02:28 2020 (r362523) @@ -112,25 +112,24 @@ ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, struct cryptop *crp; struct ocf_session *os; struct ocf_operation *oo; - struct iovec *iov, *out_iov; int i, error; uint16_t tls_comp_len; bool inplace; os = tls->cipher; - oo = malloc(sizeof(*oo) + (iovcnt + 2) * sizeof(*iov) * 2, M_KTLS_OCF, - M_WAITOK | M_ZERO); + oo = malloc(sizeof(*oo) + (iovcnt + 1) * sizeof(struct iovec), + M_KTLS_OCF, M_WAITOK | M_ZERO); oo->os = os; - iov = oo->iov; - out_iov = iov + iovcnt + 2; - uio.uio_iov = iov; + uio.uio_iov = iniov; + uio.uio_iovcnt = iovcnt; uio.uio_offset = 0; uio.uio_segflg = UIO_SYSSPACE; uio.uio_td = curthread; - out_uio.uio_iov = out_iov; + out_uio.uio_iov = outiov; + out_uio.uio_iovcnt = iovcnt; out_uio.uio_offset = 0; out_uio.uio_segflg = UIO_SYSSPACE; out_uio.uio_td = curthread; @@ -149,26 +148,18 @@ ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, ad.tls_vmajor = hdr->tls_vmajor; ad.tls_vminor = hdr->tls_vminor; ad.tls_length = htons(tls_comp_len); - iov[0].iov_base = &ad; - iov[0].iov_len = sizeof(ad); - crp->crp_aad_start = 0; + crp->crp_aad = &ad; crp->crp_aad_length = sizeof(ad); - /* Copy iov's. */ - memcpy(iov + 1, iniov, iovcnt * sizeof(*iov)); - uio.uio_iovcnt = iovcnt + 1; - memcpy(out_iov, outiov, iovcnt * sizeof(*out_iov)); - out_uio.uio_iovcnt = iovcnt; - /* Compute payload length and determine if encryption is in place. */ inplace = true; - crp->crp_payload_start = sizeof(ad); + crp->crp_payload_start = 0; for (i = 0; i < iovcnt; i++) { if (iniov[i].iov_base != outiov[i].iov_base) inplace = false; crp->crp_payload_length += iniov[i].iov_len; } - uio.uio_resid = sizeof(ad) + crp->crp_payload_length; + uio.uio_resid = crp->crp_payload_length; out_uio.uio_resid = crp->crp_payload_length; if (inplace) @@ -176,8 +167,11 @@ ktls_ocf_tls12_gcm_encrypt(struct ktls_session *tls, else tag_uio = &out_uio; - tag_uio->uio_iov[tag_uio->uio_iovcnt].iov_base = trailer; - tag_uio->uio_iov[tag_uio->uio_iovcnt].iov_len = AES_GMAC_HASH_LEN; + /* Duplicate iovec and append vector for tag. */ + memcpy(oo->iov, tag_uio->uio_iov, iovcnt * sizeof(struct iovec)); + tag_uio->uio_iov = oo->iov; + tag_uio->uio_iov[iovcnt].iov_base = trailer; + tag_uio->uio_iov[iovcnt].iov_len = AES_GMAC_HASH_LEN; tag_uio->uio_iovcnt++; crp->crp_digest_start = tag_uio->uio_resid; tag_uio->uio_resid += AES_GMAC_HASH_LEN; @@ -238,23 +232,12 @@ ktls_ocf_tls13_gcm_encrypt(struct ktls_session *tls, os = tls->cipher; - oo = malloc(sizeof(*oo) + (iovcnt + 2) * sizeof(*iov) * 2, M_KTLS_OCF, + oo = malloc(sizeof(*oo) + (iovcnt + 1) * sizeof(*iov) * 2, M_KTLS_OCF, M_WAITOK | M_ZERO); oo->os = os; iov = oo->iov; - out_iov = iov + iovcnt + 2; - uio.uio_iov = iov; - uio.uio_offset = 0; - uio.uio_segflg = UIO_SYSSPACE; - uio.uio_td = curthread; - - out_uio.uio_iov = out_iov; - out_uio.uio_offset = 0; - out_uio.uio_segflg = UIO_SYSSPACE; - out_uio.uio_td = curthread; - crp = crypto_getreq(os->sid, M_WAITOK); /* Setup the nonce. */ @@ -266,52 +249,56 @@ ktls_ocf_tls13_gcm_encrypt(struct ktls_session *tls, ad.tls_vmajor = hdr->tls_vmajor; ad.tls_vminor = hdr->tls_vminor; ad.tls_length = hdr->tls_length; - iov[0].iov_base = &ad; - iov[0].iov_len = sizeof(ad); - crp->crp_aad_start = 0; + crp->crp_aad = &ad; crp->crp_aad_length = sizeof(ad); - /* Copy iov's. */ - memcpy(iov + 1, iniov, iovcnt * sizeof(*iov)); - uio.uio_iovcnt = iovcnt + 1; - memcpy(out_iov, outiov, iovcnt * sizeof(*out_iov)); - out_uio.uio_iovcnt = iovcnt; - /* Compute payload length and determine if encryption is in place. */ inplace = true; - crp->crp_payload_start = sizeof(ad); + crp->crp_payload_start = 0; for (i = 0; i < iovcnt; i++) { if (iniov[i].iov_base != outiov[i].iov_base) inplace = false; crp->crp_payload_length += iniov[i].iov_len; } - uio.uio_resid = sizeof(ad) + crp->crp_payload_length; - out_uio.uio_resid = crp->crp_payload_length; - /* - * Always include the full trailer as input to get the - * record_type even if only the first byte is used. - */ + /* Store the record type as the first byte of the trailer. */ trailer[0] = record_type; crp->crp_payload_length++; - iov[iovcnt + 1].iov_base = trailer; - iov[iovcnt + 1].iov_len = AES_GMAC_HASH_LEN + 1; - uio.uio_iovcnt++; - uio.uio_resid += AES_GMAC_HASH_LEN + 1; - if (inplace) { - crp->crp_digest_start = uio.uio_resid - AES_GMAC_HASH_LEN; - } else { - out_iov[iovcnt] = iov[iovcnt + 1]; - out_uio.uio_iovcnt++; - out_uio.uio_resid += AES_GMAC_HASH_LEN + 1; - crp->crp_digest_start = out_uio.uio_resid - AES_GMAC_HASH_LEN; + crp->crp_digest_start = crp->crp_payload_length; + + /* + * Duplicate the input iov to append the trailer. Always + * include the full trailer as input to get the record_type + * even if only the first byte is used. + */ + memcpy(iov, iniov, iovcnt * sizeof(*iov)); + iov[iovcnt].iov_base = trailer; + iov[iovcnt].iov_len = AES_GMAC_HASH_LEN + 1; + uio.uio_iov = iov; + uio.uio_iovcnt = iovcnt + 1; + uio.uio_offset = 0; + uio.uio_resid = crp->crp_payload_length + AES_GMAC_HASH_LEN; + uio.uio_segflg = UIO_SYSSPACE; + uio.uio_td = curthread; + crypto_use_uio(crp, &uio); + + if (!inplace) { + /* Duplicate the output iov to append the trailer. */ + memcpy(out_iov, outiov, iovcnt * sizeof(*out_iov)); + out_iov[iovcnt] = iov[iovcnt]; + + out_uio.uio_iov = out_iov; + out_uio.uio_iovcnt = iovcnt + 1; + out_uio.uio_offset = 0; + out_uio.uio_resid = crp->crp_payload_length + + AES_GMAC_HASH_LEN; + out_uio.uio_segflg = UIO_SYSSPACE; + out_uio.uio_td = curthread; + crypto_use_output_uio(crp, &out_uio); } crp->crp_op = CRYPTO_OP_ENCRYPT | CRYPTO_OP_COMPUTE_DIGEST; crp->crp_flags = CRYPTO_F_CBIMM | CRYPTO_F_IV_SEPARATE; - crypto_use_uio(crp, &uio); - if (!inplace) - crypto_use_output_uio(crp, &out_uio); crp->crp_opaque = oo; crp->crp_callback = ktls_ocf_callback; @@ -368,7 +355,7 @@ ktls_ocf_try(struct socket *so, struct ktls_session *t int error; memset(&csp, 0, sizeof(csp)); - csp.csp_flags |= CSP_F_SEPARATE_OUTPUT; + csp.csp_flags |= CSP_F_SEPARATE_OUTPUT | CSP_F_SEPARATE_AAD; switch (tls->params.cipher_algorithm) { case CRYPTO_AES_NIST_GCM_16: