From owner-svn-src-projects@freebsd.org Sun May 3 00:15:20 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 350272C9C26 for ; Sun, 3 May 2020 00:15:20 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F62J0hZPz3P4p; Sun, 3 May 2020 00:15:20 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ECDED20A2; Sun, 3 May 2020 00:15:19 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0430FJLW097421; Sun, 3 May 2020 00:15:19 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0430FJK8097417; Sun, 3 May 2020 00:15:19 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030015.0430FJK8097417@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 00:15:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360580 - in projects/nfs-over-tls/usr.sbin: rpctlscd rpctlssd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/usr.sbin: rpctlscd rpctlssd X-SVN-Commit-Revision: 360580 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 00:15:20 -0000 Author: rmacklem Date: Sun May 3 00:15:18 2020 New Revision: 360580 URL: https://svnweb.freebsd.org/changeset/base/360580 Log: Fix the daemons so they actually work with jhb@'s patched openssl3. The code now has passed a trivial test, where an NFS mount was TLS1.2 encrypted on the wire. I will be updating the setup document, so others will be able to set up system(s) for testing. I have not yet decided what the correct way to handle a failure to set up the ktls is. For the server, I suspect it is clearing of the flags that say "handshake complete". For the client, I am not sure if the mount should continue unencrypted or the mount attempt should fail? At this time, the daemons build, but report warnings that SSL_CTX_load_XXX is deprecated. It works until I figure out what the preferred OpenSSL 3 call is. Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/Makefile projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c projects/nfs-over-tls/usr.sbin/rpctlssd/Makefile projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/Makefile ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlscd/Makefile Sun May 3 00:12:56 2020 (r360579) +++ projects/nfs-over-tls/usr.sbin/rpctlscd/Makefile Sun May 3 00:15:18 2020 (r360580) @@ -8,7 +8,10 @@ SRCS= rpctlscd.c rpctlscd.h rpctlscd_svc.c rpctlscd_xd CFLAGS+= -I. -LIBADD= ssl crypto +CFLAGS+= -I/usr/ktls/include +LDFLAGS+= -L/usr/ktls/lib + +LIBADD= ssl crypto util CLEANFILES= rpctlscd_svc.c rpctlscd_xdr.c rpctlscd.h Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Sun May 3 00:12:56 2020 (r360579) +++ projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Sun May 3 00:15:18 2020 (r360580) @@ -55,6 +55,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -72,7 +73,7 @@ __FBSDID("$FreeBSD$"); #define _PATH_RPCTLSCDPID "/var/run/rpctlscd.pid" #endif #ifndef _PREFERRED_CIPHERS -#define _PREFERRED_CIPHERS "SHA384:SHA256:!CAMELLIA" +#define _PREFERRED_CIPHERS "AES128-GCM-SHA256" #endif static struct pidfh *rpctls_pfh = NULL; @@ -382,7 +383,6 @@ rpctlscd_disconnect_1_svc(struct rpctlscd_disconnect_a rpctlscd_verbose_out("rpctlscd_disconnect: fd=%d closed\n", slp->s); LIST_REMOVE(slp, next); - SSL_shutdown(slp->ssl); SSL_free(slp->ssl); /* * For RPC-over-TLS, this upcall is expected @@ -560,7 +560,6 @@ rpctls_connect(SSL_CTX *ctx, int s) if (cert == NULL) { rpctlscd_verbose_out("rpctls_connect: get peer" " certificate failed\n"); - SSL_shutdown(ssl); SSL_free(ssl); return (NULL); } @@ -585,17 +584,24 @@ rpctls_connect(SSL_CTX *ctx, int s) "failed %s\n", hostnam, cp, cp2, X509_verify_cert_error_string(ret)); } - SSL_shutdown(ssl); SSL_free(ssl); return (NULL); } -#ifdef notnow + /* Check to see if ktls is enabled on the connection. */ ret = BIO_get_ktls_send(SSL_get_wbio(ssl)); - fprintf(stderr, "ktls_send=%d\n", ret); - ret = BIO_get_ktls_recv(SSL_get_rbio(ssl)); - fprintf(stderr, "ktls_recv=%d\n", ret); + rpctlscd_verbose_out("rpctls_connect: BIO_get_ktls_send=%d\n", ret); + if (ret != 0) { + ret = BIO_get_ktls_recv(SSL_get_rbio(ssl)); + rpctlscd_verbose_out("rpctls_connect: BIO_get_ktls_recv=%d\n", ret); + } +#ifdef notnow + if (ret == 0) { + SSL_free(ssl); + return (NULL); + } #endif + return (ssl); } Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/Makefile ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/Makefile Sun May 3 00:12:56 2020 (r360579) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/Makefile Sun May 3 00:15:18 2020 (r360580) @@ -8,7 +8,10 @@ SRCS= rpctlssd.c rpctlssd.h rpctlssd_svc.c rpctlssd_xd CFLAGS+= -I. -LIBADD= ssl crypto +CFLAGS+= -I/usr/ktls/include +LDFLAGS+= -L/usr/ktls/lib + +LIBADD= ssl crypto util CLEANFILES= rpctlssd_svc.c rpctlssd_xdr.c rpctlssd.h Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Sun May 3 00:12:56 2020 (r360579) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Sun May 3 00:15:18 2020 (r360580) @@ -56,6 +56,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -74,7 +75,7 @@ __FBSDID("$FreeBSD$"); #define _PATH_RPCTLSSDPID "/var/run/rpctlssd.pid" #endif #ifndef _PREFERRED_CIPHERS -#define _PREFERRED_CIPHERS "SHA384:SHA256:!CAMELLIA" +#define _PREFERRED_CIPHERS "AES128-GCM-SHA256" #endif static struct pidfh *rpctls_pfh = NULL; @@ -663,6 +664,21 @@ rpctlssd_verbose_out("%s\n", cp2); rpctlssd_verbose_out("rpctls_server: " "No peer certificate\n"); } + + /* Check to see that ktls is working for the connection. */ + ret = BIO_get_ktls_send(SSL_get_wbio(ssl)); + rpctlssd_verbose_out("rpctls_server: BIO_get_ktls_send=%d\n", ret); + if (ret != 0) { + ret = BIO_get_ktls_recv(SSL_get_rbio(ssl)); + rpctlssd_verbose_out("rpctls_server: BIO_get_ktls_recv=%d\n", ret); + } +#ifdef notnow + if (ret == 0) { + SSL_free(ssl); + return (NULL); + } +#endif + return (ssl); } From owner-svn-src-projects@freebsd.org Sun May 3 00:46:15 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 127382CA830 for ; Sun, 3 May 2020 00:46:15 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F6jy6ymyz3Qpq; Sun, 3 May 2020 00:46:14 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EA255267C; Sun, 3 May 2020 00:46:14 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0430kEvu016434; Sun, 3 May 2020 00:46:14 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0430kEFC016430; Sun, 3 May 2020 00:46:14 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030046.0430kEFC016430@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 00:46:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360584 - in projects/nfs-over-tls/sys: dev/cxgbe/tom kern modules opencrypto sys X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys: dev/cxgbe/tom kern modules opencrypto sys X-SVN-Commit-Revision: 360584 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 00:46:15 -0000 Author: rmacklem Date: Sun May 3 00:46:14 2020 New Revision: 360584 URL: https://svnweb.freebsd.org/changeset/base/360584 Log: Add jhb@'s ktls rx patch to the kernel. I also took out a few modules from sys/modules/Makefile, since those do not build without a full src tree and not just sys. Modified: projects/nfs-over-tls/sys/dev/cxgbe/tom/t4_tom.c projects/nfs-over-tls/sys/kern/uipc_ktls.c projects/nfs-over-tls/sys/kern/uipc_sockbuf.c projects/nfs-over-tls/sys/kern/uipc_socket.c projects/nfs-over-tls/sys/modules/Makefile projects/nfs-over-tls/sys/opencrypto/ktls_ocf.c projects/nfs-over-tls/sys/sys/ktls.h projects/nfs-over-tls/sys/sys/sockbuf.h projects/nfs-over-tls/sys/sys/socket.h projects/nfs-over-tls/sys/sys/socketvar.h Modified: projects/nfs-over-tls/sys/dev/cxgbe/tom/t4_tom.c ============================================================================== --- projects/nfs-over-tls/sys/dev/cxgbe/tom/t4_tom.c Sun May 3 00:37:16 2020 (r360583) +++ projects/nfs-over-tls/sys/dev/cxgbe/tom/t4_tom.c Sun May 3 00:46:14 2020 (r360584) @@ -1080,7 +1080,9 @@ is_tls_sock(struct socket *so, struct adapter *sc) struct inpcb *inp = sotoinpcb(so); int i, rc; - /* XXX: Eventually add a SO_WANT_TLS socket option perhaps? */ + if (so_options_get(so) & SO_WANT_KTLS) + return (1); + rc = 0; ADAPTER_LOCK(sc); for (i = 0; i < sc->tt.num_tls_rx_ports; i++) { Modified: projects/nfs-over-tls/sys/kern/uipc_ktls.c ============================================================================== --- projects/nfs-over-tls/sys/kern/uipc_ktls.c Sun May 3 00:37:16 2020 (r360583) +++ projects/nfs-over-tls/sys/kern/uipc_ktls.c Sun May 3 00:46:14 2020 (r360584) @@ -79,7 +79,8 @@ __FBSDID("$FreeBSD$"); struct ktls_wq { struct mtx mtx; - STAILQ_HEAD(, mbuf_ext_pgs) head; + STAILQ_HEAD(, mbuf_ext_pgs) pgs_head; + STAILQ_HEAD(, socket) so_head; bool running; } __aligned(CACHE_LINE_SIZE); @@ -131,10 +132,16 @@ static counter_u64_t ktls_tasks_active; SYSCTL_COUNTER_U64(_kern_ipc_tls, OID_AUTO, tasks_active, CTLFLAG_RD, &ktls_tasks_active, "Number of active tasks"); -static counter_u64_t ktls_cnt_on; -SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, so_inqueue, CTLFLAG_RD, - &ktls_cnt_on, "Number of TLS records in queue to tasks for SW crypto"); +static counter_u64_t ktls_cnt_tx_queued; +SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, sw_tx_inqueue, CTLFLAG_RD, + &ktls_cnt_tx_queued, + "Number of TLS records in queue to tasks for SW encryption"); +static counter_u64_t ktls_cnt_rx_queued; +SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, sw_rx_inqueue, CTLFLAG_RD, + &ktls_cnt_rx_queued, + "Number of TLS sockets in queue to tasks for SW decryption"); + static counter_u64_t ktls_offload_total; SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, offload_total, CTLFLAG_RD, &ktls_offload_total, @@ -149,6 +156,10 @@ static counter_u64_t ktls_offload_active; SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, active, CTLFLAG_RD, &ktls_offload_active, "Total Active TLS sessions"); +static counter_u64_t ktls_offload_corrupted_records; +SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, corrupted_records, CTLFLAG_RD, + &ktls_offload_corrupted_records, "Total corrupted TLS records received"); + static counter_u64_t ktls_offload_failed_crypto; SYSCTL_COUNTER_U64(_kern_ipc_tls_stats, OID_AUTO, failed_crypto, CTLFLAG_RD, &ktls_offload_failed_crypto, "Total TLS crypto failures"); @@ -334,10 +345,12 @@ ktls_init(void *dummy __unused) int error, i; ktls_tasks_active = counter_u64_alloc(M_WAITOK); - ktls_cnt_on = counter_u64_alloc(M_WAITOK); + ktls_cnt_tx_queued = counter_u64_alloc(M_WAITOK); + ktls_cnt_rx_queued = counter_u64_alloc(M_WAITOK); ktls_offload_total = counter_u64_alloc(M_WAITOK); ktls_offload_enable_calls = counter_u64_alloc(M_WAITOK); ktls_offload_active = counter_u64_alloc(M_WAITOK); + ktls_offload_corrupted_records = counter_u64_alloc(M_WAITOK); ktls_offload_failed_crypto = counter_u64_alloc(M_WAITOK); ktls_switch_to_ifnet = counter_u64_alloc(M_WAITOK); ktls_switch_to_sw = counter_u64_alloc(M_WAITOK); @@ -370,7 +383,8 @@ ktls_init(void *dummy __unused) * work queue for each CPU. */ CPU_FOREACH(i) { - STAILQ_INIT(&ktls_wq[i].head); + STAILQ_INIT(&ktls_wq[i].pgs_head); + STAILQ_INIT(&ktls_wq[i].so_head); mtx_init(&ktls_wq[i].mtx, "ktls work queue", NULL, MTX_DEF); error = kproc_kthread_add(ktls_work_thread, &ktls_wq[i], &ktls_proc, &td, 0, 0, "KTLS", "thr_%d", i); @@ -859,7 +873,7 @@ ktls_try_ifnet(struct socket *so, struct ktls_session } static int -ktls_try_sw(struct socket *so, struct ktls_session *tls) +ktls_try_sw(struct socket *so, struct ktls_session *tls, int direction) { struct rm_priotracker prio; struct ktls_crypto_backend *be; @@ -874,7 +888,7 @@ ktls_try_sw(struct socket *so, struct ktls_session *tl if (ktls_allow_unload) rm_rlock(&ktls_backends_lock, &prio); LIST_FOREACH(be, &ktls_backends, next) { - if (be->try(so, tls) == 0) + if (be->try(so, tls, direction) == 0) break; KASSERT(tls->cipher == NULL, ("ktls backend leaked a cipher pointer")); @@ -900,6 +914,61 @@ ktls_try_sw(struct socket *so, struct ktls_session *tl return (0); } +/* + * KTLS RX stores data in the socket buffer as a list of TLS records, + * where each record is stored as a control message containg the TLS + * header followed by data mbufs containing the decrypted data. This + * is different from KTLS TX which always uses an mb_ext_pgs mbuf for + * both encrypted and decrypted data. TLS records decrypted by a NIC + * should be queued to the socket buffer as records, but encrypted + * data which needs to be decrypted by software arrives as a stream of + * regular mbufs which need to be converted. In addition, there may + * already be pending encrypted data in the socket buffer when KTLS RX + * is enabled. + * + * To manage not-yet-decrypted data for KTLS RX, the following scheme + * is used: + * + * - A single chain of NOTREADY mbufs is hung off of sb_mtls. + * + * - ktls_check_rx checks this chain of mbufs reading the TLS header + * from the first mbuf. Once all of the data for that TLS record is + * queued, the socket is queued to a worker thread. + * + * - The worker thread calls ktls_decrypt to decrypt TLS records in + * the TLS chain. Each TLS record is detached from the TLS chain, + * decrypted, and inserted into the regular socket buffer chain as + * record starting with a control message holding the TLS header and + * a chain of mbufs holding the encrypted data. + */ + +static void +sb_mark_notready(struct sockbuf *sb) +{ + struct mbuf *m; + + m = sb->sb_mb; + sb->sb_mtls = m; + sb->sb_mb = NULL; + sb->sb_mbtail = NULL; + sb->sb_lastrecord = NULL; + for (; m != NULL; m = m->m_next) { + KASSERT(m->m_nextpkt == NULL, ("%s: m_nextpkt != NULL", + __func__)); + KASSERT((m->m_flags & M_NOTAVAIL) == 0, ("%s: mbuf not avail", + __func__)); + KASSERT(sb->sb_acc >= m->m_len, ("%s: sb_acc < m->m_len", + __func__)); + m->m_flags |= M_NOTREADY; + sb->sb_acc -= m->m_len; + sb->sb_tlscc += m->m_len; + sb->sb_mtlstail = m; + } + KASSERT(sb->sb_acc == 0 && sb->sb_tlscc == sb->sb_ccc, + ("%s: acc %u tlscc %u ccc %u", __func__, sb->sb_acc, sb->sb_tlscc, + sb->sb_ccc)); +} + int ktls_enable_rx(struct socket *so, struct tls_enable *en) { @@ -928,16 +997,20 @@ ktls_enable_rx(struct socket *so, struct tls_enable *e if (en->cipher_algorithm == CRYPTO_AES_CBC && !ktls_cbc_enable) return (ENOTSUP); + /* TLS 1.3 is not yet supported. */ + if (en->tls_vmajor == TLS_MAJOR_VER_ONE && + en->tls_vminor == TLS_MINOR_VER_THREE) + return (ENOTSUP); + error = ktls_create_session(so, en, &tls); if (error) return (error); - /* TLS RX offload is only supported on TOE currently. */ #ifdef TCP_OFFLOAD error = ktls_try_toe(so, tls, KTLS_RX); -#else - error = EOPNOTSUPP; + if (error) #endif + error = ktls_try_sw(so, tls, KTLS_RX); if (error) { ktls_cleanup(tls); @@ -946,7 +1019,13 @@ ktls_enable_rx(struct socket *so, struct tls_enable *e /* Mark the socket as using TLS offload. */ SOCKBUF_LOCK(&so->so_rcv); + so->so_rcv.sb_tls_seqno = be64dec(en->rec_seq); so->so_rcv.sb_tls_info = tls; + so->so_rcv.sb_flags |= SB_TLS_RX; + + /* Mark existing data as not ready until it can be decrypted. */ + sb_mark_notready(&so->so_rcv); + ktls_check_rx(&so->so_rcv); SOCKBUF_UNLOCK(&so->so_rcv); counter_u64_add(ktls_offload_total, 1); @@ -997,7 +1076,7 @@ ktls_enable_tx(struct socket *so, struct tls_enable *e #endif error = ktls_try_ifnet(so, tls, false); if (error) - error = ktls_try_sw(so, tls); + error = ktls_try_sw(so, tls, KTLS_TX); if (error) { ktls_cleanup(tls); @@ -1102,7 +1181,7 @@ ktls_set_tx_mode(struct socket *so, int mode) if (mode == TCP_TLS_MODE_IFNET) error = ktls_try_ifnet(so, tls_new, true); else - error = ktls_try_sw(so, tls_new); + error = ktls_try_sw(so, tls_new, KTLS_TX); if (error) { counter_u64_add(ktls_switch_failed, 1); ktls_free(tls_new); @@ -1430,6 +1509,371 @@ ktls_frame(struct mbuf *top, struct ktls_session *tls, } void +ktls_check_rx(struct sockbuf *sb) +{ + struct tls_record_layer hdr; + struct ktls_wq *wq; + struct socket *so; + bool running; + + SOCKBUF_LOCK_ASSERT(sb); + KASSERT(sb->sb_flags & SB_TLS_RX, ("%s: sockbuf %p isn't TLS RX", + __func__, sb)); + so = __containerof(sb, struct socket, so_rcv); + + if (sb->sb_flags & SB_TLS_RX_RUNNING) + return; + + /* Is there enough queued for a TLS header? */ + if (sb->sb_tlscc < sizeof(hdr)) { + if ((sb->sb_state & SBS_CANTRCVMORE) != 0 && sb->sb_tlscc != 0) + so->so_error = EMSGSIZE; + return; + } + + m_copydata(sb->sb_mtls, 0, sizeof(hdr), (void *)&hdr); + + /* Is the entire record queued? */ + if (sb->sb_tlscc < sizeof(hdr) + ntohs(hdr.tls_length)) { + if ((sb->sb_state & SBS_CANTRCVMORE) != 0) + so->so_error = EMSGSIZE; + return; + } + + sb->sb_flags |= SB_TLS_RX_RUNNING; + + soref(so); + wq = &ktls_wq[so->so_rcv.sb_tls_info->wq_index]; + mtx_lock(&wq->mtx); + STAILQ_INSERT_TAIL(&wq->so_head, so, so_ktls_rx_list); + running = wq->running; + mtx_unlock(&wq->mtx); + if (!running) + wakeup(wq); + counter_u64_add(ktls_cnt_rx_queued, 1); +} + +static struct mbuf * +ktls_detach_record(struct sockbuf *sb, int len) +{ + struct mbuf *m, *n, *top; + int remain; + + SOCKBUF_LOCK_ASSERT(sb); + MPASS(len <= sb->sb_tlscc); + + /* + * If TLS chain is the exact size of the record, + * just grab the whole record. + */ + top = sb->sb_mtls; + if (sb->sb_tlscc == len) { + sb->sb_mtls = NULL; + sb->sb_mtlstail = NULL; + goto out; + } + + /* + * While it would be nice to use m_split() here, we need + * to know exactly what m_split() allocates to update the + * accounting, so do it inline instead. + */ + remain = len; + for (m = top; remain > m->m_len; m = m->m_next) + remain -= m->m_len; + + /* Easy case: don't have to split 'm'. */ + if (remain == m->m_len) { + sb->sb_mtls = m->m_next; + if (sb->sb_mtls == NULL) + sb->sb_mtlstail = NULL; + m->m_next = NULL; + goto out; + } + + /* + * Need to allocate an mbuf to hold the remainder of 'm'. Try + * with M_NOWAIT first. + */ + n = m_get(M_NOWAIT, MT_DATA); + if (n == NULL) { + /* + * Use M_WAITOK with socket buffer unlocked. If + * 'sb_mtls' changes while the lock is dropped, return + * NULL to force the caller to retry. + */ + SOCKBUF_UNLOCK(sb); + + n = m_get(M_WAITOK, MT_DATA); + + SOCKBUF_LOCK(sb); + if (sb->sb_mtls != top) { + m_free(n); + return (NULL); + } + } + n->m_flags |= M_NOTREADY; + + /* Store remainder in 'n'. */ + n->m_len = m->m_len - remain; + if (m->m_flags & M_EXT) { + n->m_data = m->m_data + remain; + mb_dupcl(n, m); + } else { + bcopy(mtod(m, caddr_t) + remain, mtod(n, caddr_t), n->m_len); + } + + /* Trim 'm' and update accounting. */ + m->m_len -= n->m_len; + sb->sb_tlscc -= n->m_len; + sb->sb_ccc -= n->m_len; + + /* Account for 'n'. */ + sballoc_ktls_rx(sb, n); + + /* Insert 'n' into the TLS chain. */ + sb->sb_mtls = n; + n->m_next = m->m_next; + if (sb->sb_mtlstail == m) + sb->sb_mtlstail = n; + + /* Detach the record from the TLS chain. */ + m->m_next = NULL; + +out: + MPASS(m_length(top, NULL) == len); + for (m = top; m != NULL; m = m->m_next) + sbfree_ktls_rx(sb, m); + sb->sb_tlsdcc = len; + sb->sb_ccc += len; + SBCHECK(sb); + return (top); +} + +static int +m_segments(struct mbuf *m, int skip) +{ + int count; + + while (skip >= m->m_len) { + skip -= m->m_len; + m = m->m_next; + } + + for (count = 0; m != NULL; count++) + m = m->m_next; + return (count); +} + +static void +ktls_decrypt(struct socket *so) +{ + char tls_header[MBUF_PEXT_HDR_LEN]; + struct ktls_session *tls; + struct sockbuf *sb; + struct tls_record_layer *hdr; + struct iovec *iov; + struct tls_get_record tgr; + struct mbuf *control, *data, *m; + uint64_t seqno; + int error, i, iov_cap, iov_count, remain, tls_len, trail_len; + + hdr = (struct tls_record_layer *)tls_header; + sb = &so->so_rcv; + SOCKBUF_LOCK(sb); + KASSERT(sb->sb_flags & SB_TLS_RX_RUNNING, + ("%s: socket %p not running", __func__, so)); + + tls = sb->sb_tls_info; + MPASS(tls != NULL); + + iov = NULL; + iov_cap = 0; + for (;;) { + /* Is there enough queued for a TLS header? */ + if (sb->sb_tlscc < tls->params.tls_hlen) + break; + + m_copydata(sb->sb_mtls, 0, tls->params.tls_hlen, tls_header); + tls_len = sizeof(*hdr) + ntohs(hdr->tls_length); + + if (hdr->tls_vmajor != tls->params.tls_vmajor || + hdr->tls_vminor != tls->params.tls_vminor) + error = EINVAL; + else if (tls_len < tls->params.tls_hlen || tls_len > + tls->params.tls_hlen + TLS_MAX_MSG_SIZE_V10_2 + + tls->params.tls_tlen) + error = EMSGSIZE; + else + error = 0; + if (__predict_false(error != 0)) { + /* + * We have a corrupted record and are likely + * out of sync. The connection isn't + * recoverable at this point, so abort it. + */ + SOCKBUF_UNLOCK(sb); + counter_u64_add(ktls_offload_corrupted_records, 1); + + CURVNET_SET(so->so_vnet); + so->so_proto->pr_usrreqs->pru_abort(so); + so->so_error = error; + CURVNET_RESTORE(); + goto deref; + } + + /* Is the entire record queued? */ + if (sb->sb_tlscc < tls_len) + break; + + /* + * Split out the portion of the mbuf chain containing + * this TLS record. + */ + data = ktls_detach_record(sb, tls_len); + if (data == NULL) + continue; + MPASS(sb->sb_tlsdcc == tls_len); + + seqno = sb->sb_tls_seqno; + sb->sb_tls_seqno++; + SBCHECK(sb); + SOCKBUF_UNLOCK(sb); + + /* + * Build an I/O vector spanning the TLS record payload + * and trailer but skipping the header. + */ + iov_count = m_segments(data, tls->params.tls_hlen); + if (iov_count > iov_cap) { + free(iov, M_KTLS); + iov = malloc(sizeof(*iov) * iov_count, M_KTLS, + M_WAITOK); + iov_cap = iov_count; + } + remain = tls->params.tls_hlen; + for (m = data; remain >= m->m_len; m = m->m_next) + remain -= m->m_len; + iov[0].iov_base = m->m_data + remain; + iov[0].iov_len = m->m_len - remain; + for (m = m->m_next, i = 1; m != NULL; m = m->m_next, i++) { + iov[i].iov_base = m->m_data; + iov[i].iov_len = m->m_len; + } + MPASS(i == iov_count); + + error = tls->sw_decrypt(tls, hdr, iov, iov_count, seqno, + &trail_len); + if (error) { + counter_u64_add(ktls_offload_failed_crypto, 1); + + SOCKBUF_LOCK(sb); + if (sb->sb_tlsdcc == 0) { + /* + * sbcut/drop/flush discarded these + * mbufs. + */ + m_freem(data); + break; + } + + /* + * Drop this TLS record's data, but keep + * decrypting subsequent records. + */ + sb->sb_ccc -= tls_len; + sb->sb_tlsdcc = 0; + + CURVNET_SET(so->so_vnet); + so->so_error = EBADMSG; + sorwakeup_locked(so); + CURVNET_RESTORE(); + + m_freem(data); + + SOCKBUF_LOCK(sb); + continue; + } + + /* Allocate the control mbuf. */ + tgr.tls_type = hdr->tls_type; + tgr.tls_vmajor = hdr->tls_vmajor; + tgr.tls_vminor = hdr->tls_vminor; + tgr.tls_length = htobe16(tls_len - tls->params.tls_hlen - + trail_len); + control = sbcreatecontrol_how(&tgr, sizeof(tgr), + TLS_GET_RECORD, IPPROTO_TCP, M_WAITOK); + + SOCKBUF_LOCK(sb); + if (sb->sb_tlsdcc == 0) { + /* sbcut/drop/flush discarded these mbufs. */ + MPASS(sb->sb_tlscc == 0); + m_freem(data); + m_freem(control); + break; + } + + /* + * Clear the 'dcc' accounting in preparation for + * adding the decrypted record. + */ + sb->sb_ccc -= tls_len; + sb->sb_tlsdcc = 0; + SBCHECK(sb); + + /* If there is no payload, drop all of the data. */ + if (tgr.tls_length == htobe16(0)) { + m_freem(data); + data = NULL; + } else { + /* Trim header. */ + remain = tls->params.tls_hlen; + while (remain > 0) { + if (data->m_len > remain) { + data->m_data += remain; + data->m_len -= remain; + break; + } + remain -= data->m_len; + data = m_free(data); + } + + /* Trim trailer and clear M_NOTREADY. */ + remain = be16toh(tgr.tls_length); + m = data; + for (m = data; remain > m->m_len; m = m->m_next) { + m->m_flags &= ~M_NOTREADY; + remain -= m->m_len; + } + m->m_len = remain; + m_freem(m->m_next); + m->m_next = NULL; + m->m_flags &= ~M_NOTREADY; + + /* Set EOR on the final mbuf. */ + m->m_flags |= M_EOR; + } + + sbappendcontrol_locked(sb, data, control, 0); + } + + sb->sb_flags &= ~SB_TLS_RX_RUNNING; + + if ((sb->sb_state & SBS_CANTRCVMORE) != 0 && sb->sb_tlscc > 0) + so->so_error = EMSGSIZE; + + sorwakeup_locked(so); + +deref: + SOCKBUF_UNLOCK_ASSERT(sb); + + CURVNET_SET(so->so_vnet); + SOCK_LOCK(so); + sorele(so); + CURVNET_RESTORE(); +} + +void ktls_enqueue_to_free(struct mbuf_ext_pgs *pgs) { struct ktls_wq *wq; @@ -1439,7 +1883,7 @@ ktls_enqueue_to_free(struct mbuf_ext_pgs *pgs) pgs->mbuf = NULL; wq = &ktls_wq[pgs->tls->wq_index]; mtx_lock(&wq->mtx); - STAILQ_INSERT_TAIL(&wq->head, pgs, stailq); + STAILQ_INSERT_TAIL(&wq->pgs_head, pgs, stailq); running = wq->running; mtx_unlock(&wq->mtx); if (!running) @@ -1473,12 +1917,12 @@ ktls_enqueue(struct mbuf *m, struct socket *so, int pa wq = &ktls_wq[pgs->tls->wq_index]; mtx_lock(&wq->mtx); - STAILQ_INSERT_TAIL(&wq->head, pgs, stailq); + STAILQ_INSERT_TAIL(&wq->pgs_head, pgs, stailq); running = wq->running; mtx_unlock(&wq->mtx); if (!running) wakeup(wq); - counter_u64_add(ktls_cnt_on, 1); + counter_u64_add(ktls_cnt_tx_queued, 1); } static __noinline void @@ -1633,36 +2077,46 @@ static void ktls_work_thread(void *ctx) { struct ktls_wq *wq = ctx; - struct mbuf_ext_pgs *p, *n; + struct mbuf_ext_pgs *p, *pn; + struct socket *so, *son; struct ktls_session *tls; struct mbuf *m; - STAILQ_HEAD(, mbuf_ext_pgs) local_head; + STAILQ_HEAD(, mbuf_ext_pgs) local_pgs_head; + STAILQ_HEAD(, socket) local_so_head; #if defined(__aarch64__) || defined(__amd64__) || defined(__i386__) fpu_kern_thread(0); #endif for (;;) { mtx_lock(&wq->mtx); - while (STAILQ_EMPTY(&wq->head)) { + while (STAILQ_EMPTY(&wq->pgs_head) && + STAILQ_EMPTY(&wq->so_head)) { wq->running = false; mtx_sleep(wq, &wq->mtx, 0, "-", 0); wq->running = true; } - STAILQ_INIT(&local_head); - STAILQ_CONCAT(&local_head, &wq->head); + STAILQ_INIT(&local_pgs_head); + STAILQ_CONCAT(&local_pgs_head, &wq->pgs_head); + STAILQ_INIT(&local_so_head); + STAILQ_CONCAT(&local_so_head, &wq->so_head); mtx_unlock(&wq->mtx); - STAILQ_FOREACH_SAFE(p, &local_head, stailq, n) { + STAILQ_FOREACH_SAFE(p, &local_pgs_head, stailq, pn) { if (p->mbuf != NULL) { ktls_encrypt(p); - counter_u64_add(ktls_cnt_on, -1); + counter_u64_add(ktls_cnt_tx_queued, -1); } else { tls = p->tls; ktls_free(tls); m = __containerof(p, struct mbuf, m_ext_pgs); uma_zfree(zone_mbuf, m); } + } + + STAILQ_FOREACH_SAFE(so, &local_so_head, so_ktls_rx_list, son) { + ktls_decrypt(so); + counter_u64_add(ktls_cnt_rx_queued, -1); } } } Modified: projects/nfs-over-tls/sys/kern/uipc_sockbuf.c ============================================================================== --- projects/nfs-over-tls/sys/kern/uipc_sockbuf.c Sun May 3 00:37:16 2020 (r360583) +++ projects/nfs-over-tls/sys/kern/uipc_sockbuf.c Sun May 3 00:46:14 2020 (r360584) @@ -70,6 +70,8 @@ u_long sb_max_adj = static u_long sb_efficiency = 8; /* parameter for sbreserve() */ +static void sbcompress_ktls_rx(struct sockbuf *sb, struct mbuf *m, + struct mbuf *n); static struct mbuf *sbcut_internal(struct sockbuf *sb, int len); static void sbflush_internal(struct sockbuf *sb); @@ -339,7 +341,52 @@ sbfree(struct sockbuf *sb, struct mbuf *m) sb->sb_sndptroff -= m->m_len; } +#ifdef KERN_TLS /* + * Similar to sballoc/sbfree but does not adjust state associated with + * the sb_mb chain such a sb_fnrdy or sb_sndptr*. Also assumes mbufs + * are not ready. + */ +void +sballoc_ktls_rx(struct sockbuf *sb, struct mbuf *m) +{ + + SOCKBUF_LOCK_ASSERT(sb); + + sb->sb_ccc += m->m_len; + sb->sb_tlscc += m->m_len; + + sb->sb_mbcnt += MSIZE; + sb->sb_mcnt += 1; + + if (m->m_flags & M_EXT) { + sb->sb_mbcnt += m->m_ext.ext_size; + sb->sb_ccnt += 1; + } +} + +void +sbfree_ktls_rx(struct sockbuf *sb, struct mbuf *m) +{ + +#if 0 /* XXX: not yet: soclose() call path comes here w/o lock. */ + SOCKBUF_LOCK_ASSERT(sb); +#endif + + sb->sb_ccc -= m->m_len; + sb->sb_tlscc -= m->m_len; + + sb->sb_mbcnt -= MSIZE; + sb->sb_mcnt -= 1; + + if (m->m_flags & M_EXT) { + sb->sb_mbcnt -= m->m_ext.ext_size; + sb->sb_ccnt -= 1; + } +} +#endif + +/* * Socantsendmore indicates that no more data will be sent on the socket; it * would normally be applied to a socket when the user informs the system * that no more data is to be sent, by the protocol code (in case @@ -375,6 +422,10 @@ socantrcvmore_locked(struct socket *so) SOCKBUF_LOCK_ASSERT(&so->so_rcv); so->so_rcv.sb_state |= SBS_CANTRCVMORE; +#ifdef KERN_TLS + if (so->so_rcv.sb_flags & SB_TLS_RX) + ktls_check_rx(&so->so_rcv); +#endif sorwakeup_locked(so); mtx_assert(SOCKBUF_MTX(&so->so_rcv), MA_NOTOWNED); } @@ -775,6 +826,24 @@ sblastmbufchk(struct sockbuf *sb, const char *file, in } panic("%s from %s:%u", __func__, file, line); } + +#ifdef KERN_TLS + m = sb->sb_mtls; + while (m && m->m_next) + m = m->m_next; + + if (m != sb->sb_mtlstail) { + printf("%s: sb_mtls %p sb_mtlstail %p last %p\n", + __func__, sb->sb_mtls, sb->sb_mtlstail, m); + printf("TLS packet tree:\n"); + printf("\t"); + for (m = sb->sb_mtls; m != NULL; m = m->m_next) { + printf("%p ", m); + } + printf("\n"); + panic("%s from %s:%u", __func__, file, line); + } +#endif } #endif /* SOCKBUF_DEBUG */ @@ -852,7 +921,30 @@ sbappend(struct sockbuf *sb, struct mbuf *m, int flags SOCKBUF_UNLOCK(sb); } +#ifdef KERN_TLS /* + * Append an mbuf containing encrypted TLS data. The data + * is marked M_NOTREADY until it has been decrypted and + * stored as a TLS record. + */ +static void +sbappend_ktls_rx(struct sockbuf *sb, struct mbuf *m) +{ + struct mbuf *n; + + SBLASTMBUFCHK(sb); + + /* Remove all packet headers and mbuf tags to get a pure data chain. */ + m_demote(m, 1, 0); + + for (n = m; n != NULL; n = n->m_next) + n->m_flags |= M_NOTREADY; + sbcompress_ktls_rx(sb, m, sb->sb_mtlstail); + ktls_check_rx(sb); +} +#endif + +/* * This version of sbappend() should only be used when the caller absolutely * knows that there will never be more than one record in the socket buffer, * that is, a stream protocol (such as TCP). @@ -863,6 +955,19 @@ sbappendstream_locked(struct sockbuf *sb, struct mbuf SOCKBUF_LOCK_ASSERT(sb); KASSERT(m->m_nextpkt == NULL,("sbappendstream 0")); + +#ifdef KERN_TLS + /* + * Decrypted TLS records are appended as records via + * sbappendrecord(). TCP passes encrypted TLS records to this + * function which must be scheduled for decryption. + */ + if (sb->sb_flags & SB_TLS_RX) { + sbappend_ktls_rx(sb, m); + return; + } +#endif + KASSERT(sb->sb_mb == sb->sb_lastrecord,("sbappendstream 1")); SBLASTMBUFCHK(sb); @@ -901,6 +1006,9 @@ sbcheck(struct sockbuf *sb, const char *file, int line { struct mbuf *m, *n, *fnrdy; u_long acc, ccc, mbcnt; +#ifdef KERN_TLS + u_long tlscc; +#endif SOCKBUF_LOCK_ASSERT(sb); @@ -936,9 +1044,46 @@ sbcheck(struct sockbuf *sb, const char *file, int line mbcnt += m->m_ext.ext_size; } } +#ifdef KERN_TLS + /* + * Account for mbufs "detached" by ktls_detach_record() while + * they are decrypted by ktls_decrypt(). tlsdcc gives a count + * of the detached bytes that are included in ccc. The mbufs + * and clusters are not included in the socket buffer + * accounting. + */ + ccc += sb->sb_tlsdcc; + + tlscc = 0; + for (m = sb->sb_mtls; m; m = m->m_next) { + if (m->m_nextpkt != NULL) { + printf("sb %p TLS mbuf %p with nextpkt\n", sb, m); + goto fail; + } + if ((m->m_flags & M_NOTREADY) == 0) { + printf("sb %p TLS mbuf %p ready\n", sb, m); + goto fail; + } + tlscc += m->m_len; + ccc += m->m_len; + mbcnt += MSIZE; + if (m->m_flags & M_EXT) /*XXX*/ /* pretty sure this is bogus */ + mbcnt += m->m_ext.ext_size; + } + + if (sb->sb_tlscc != tlscc) { + printf("tlscc %ld/%u dcc %u\n", tlscc, sb->sb_tlscc, + sb->sb_tlsdcc); + goto fail; + } +#endif if (acc != sb->sb_acc || ccc != sb->sb_ccc || mbcnt != sb->sb_mbcnt) { printf("acc %ld/%u ccc %ld/%u mbcnt %ld/%u\n", acc, sb->sb_acc, ccc, sb->sb_ccc, mbcnt, sb->sb_mbcnt); +#ifdef KERN_TLS + printf("tlscc %ld/%u dcc %u\n", tlscc, sb->sb_tlscc, + sb->sb_tlsdcc); +#endif goto fail; } return; @@ -1214,14 +1359,72 @@ sbcompress(struct sockbuf *sb, struct mbuf *m, struct SBLASTMBUFCHK(sb); } +#ifdef KERN_TLS /* + * A version of sbcompress() for encrypted TLS RX mbufs. These mbufs + * are appended to the 'sb_mtls' chain instead of 'sb_mb' and are also + * a bit simpler (no EOR markers, always MT_DATA, etc.). + */ +static void +sbcompress_ktls_rx(struct sockbuf *sb, struct mbuf *m, struct mbuf *n) +{ + + SOCKBUF_LOCK_ASSERT(sb); + + while (m) { + KASSERT((m->m_flags & M_EOR) == 0, + ("TLS RX mbuf %p with EOR", m)); + KASSERT(m->m_type == MT_DATA, + ("TLS RX mbuf %p is not MT_DATA", m)); + KASSERT((m->m_flags & M_NOTREADY) != 0, + ("TLS RX mbuf %p ready", m)); + KASSERT((m->m_flags & M_NOMAP) == 0, + ("TLS RX mbuf %p unmapped", m)); + + if (m->m_len == 0) { + m = m_free(m); + continue; + } + + /* + * Even though both 'n' and 'm' are NOTREADY, it's ok + * to coalesce the data. + */ + if (n && + M_WRITABLE(n) && + ((sb->sb_flags & SB_NOCOALESCE) == 0) && + !(n->m_flags & (M_NOMAP)) && + m->m_len <= MCLBYTES / 4 && /* XXX: Don't copy too much */ + m->m_len <= M_TRAILINGSPACE(n)) { + m_copydata(m, 0, m->m_len, mtodo(n, n->m_len)); + n->m_len += m->m_len; + sb->sb_ccc += m->m_len; + sb->sb_tlscc += m->m_len; + m = m_free(m); + continue; + } + if (n) + n->m_next = m; + else + sb->sb_mtls = m; + sb->sb_mtlstail = m; + sballoc_ktls_rx(sb, m); + n = m; + m = m->m_next; + n->m_next = NULL; + } + SBLASTMBUFCHK(sb); +} +#endif + +/* * Free all mbufs in a sockbuf. Check that all resources are reclaimed. */ static void sbflush_internal(struct sockbuf *sb) { - while (sb->sb_mbcnt) { + while (sb->sb_mbcnt || sb->sb_tlsdcc) { /* * Don't call sbcut(sb, 0) if the leading mbuf is non-empty: * we would loop forever. Panic instead. @@ -1259,6 +1462,7 @@ static struct mbuf * sbcut_internal(struct sockbuf *sb, int len) { struct mbuf *m, *next, *mfree; + bool is_tls; KASSERT(len >= 0, ("%s: len is %d but it is supposed to be >= 0", __func__, len)); @@ -1266,10 +1470,25 @@ sbcut_internal(struct sockbuf *sb, int len) __func__, len, sb->sb_ccc)); next = (m = sb->sb_mb) ? m->m_nextpkt : 0; + is_tls = false; mfree = NULL; while (len > 0) { if (m == NULL) { +#ifdef KERN_TLS + if (next == NULL && !is_tls) { + if (sb->sb_tlsdcc != 0) { + MPASS(len >= sb->sb_tlsdcc); + len -= sb->sb_tlsdcc; + sb->sb_ccc -= sb->sb_tlsdcc; + sb->sb_tlsdcc = 0; + if (len == 0) + break; + } + next = sb->sb_mtls; + is_tls = true; + } +#endif KASSERT(next, ("%s: no next, len %d", __func__, len)); m = next; next = m->m_nextpkt; @@ -1288,12 +1507,17 @@ sbcut_internal(struct sockbuf *sb, int len) break; } len -= m->m_len; - sbfree(sb, m); +#ifdef KERN_TLS + if (is_tls) + sbfree_ktls_rx(sb, m); + else +#endif + sbfree(sb, m); /* * Do not put M_NOTREADY buffers to the free list, they * are referenced from outside. */ - if (m->m_flags & M_NOTREADY) + if (m->m_flags & M_NOTREADY && !is_tls) m = m->m_next; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-projects@freebsd.org Sun May 3 00:51:15 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6099D2CA9E3 for ; Sun, 3 May 2020 00:51:15 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F6ql2J8wz3QyP; Sun, 3 May 2020 00:51:15 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4A18026C3; Sun, 3 May 2020 00:51:15 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0430pFQH017594; Sun, 3 May 2020 00:51:15 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0430pFkb017593; Sun, 3 May 2020 00:51:15 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030051.0430pFkb017593@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 00:51:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360585 - projects/nfs-over-tls/sys/amd64/conf X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/amd64/conf X-SVN-Commit-Revision: 360585 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 00:51:15 -0000 Author: rmacklem Date: Sun May 3 00:51:14 2020 New Revision: 360585 URL: https://svnweb.freebsd.org/changeset/base/360585 Log: Add the kernel config I use, which includes "options KERN_TLS", plus a couple of changes so the kernel will build by itself. Added: projects/nfs-over-tls/sys/amd64/conf/MYTEST Added: projects/nfs-over-tls/sys/amd64/conf/MYTEST ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/nfs-over-tls/sys/amd64/conf/MYTEST Sun May 3 00:51:14 2020 (r360585) @@ -0,0 +1,378 @@ +# +# GENERIC -- Generic kernel configuration file for FreeBSD/amd64 +# +# For more information on this file, please read the config(5) manual page, +# and/or the handbook section on Kernel Configuration Files: +# +# https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html +# +# The handbook is also available locally in /usr/share/doc/handbook +# if you've installed the doc distribution, otherwise always see the +# FreeBSD World Wide Web server (https://www.FreeBSD.org/) for the +# latest information. +# +# An exhaustive list of options and more detailed explanations of the +# device lines is also present in the ../../conf/NOTES and NOTES files. +# If you are in doubt as to the purpose or necessity of a line, check first +# in NOTES. +# +# $FreeBSD$ + +cpu HAMMER +ident GENERIC + +makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols +makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support + +options SCHED_ULE # ULE scheduler +options NUMA # Non-Uniform Memory Architecture support +options PREEMPTION # Enable kernel thread preemption +options VIMAGE # Subsystem virtualization, e.g. VNET +options INET # InterNETworking +options INET6 # IPv6 communications protocols +options IPSEC_SUPPORT # Allow kldload of ipsec and tcpmd5 +options TCP_OFFLOAD # TCP offload +options TCP_BLACKBOX # Enhanced TCP event logging +options TCP_HHOOK # hhook(9) framework for TCP +options TCP_RFC7413 # TCP Fast Open +options SCTP # Stream Control Transmission Protocol +options FFS # Berkeley Fast Filesystem +options SOFTUPDATES # Enable FFS soft updates support +options UFS_ACL # Support for access control lists +options UFS_DIRHASH # Improve performance on big directories +options UFS_GJOURNAL # Enable gjournal-based UFS journaling +options QUOTA # Enable disk quotas for UFS +options MD_ROOT # MD is a potential root device +options NFSCL # Network Filesystem Client +options NFSD # Network Filesystem Server +options NFSLOCKD # Network Lock Manager +options NFS_ROOT # NFS usable as /, requires NFSCL +options MSDOSFS # MSDOS Filesystem +options CD9660 # ISO 9660 Filesystem +options PROCFS # Process filesystem (requires PSEUDOFS) +options PSEUDOFS # Pseudo-filesystem framework +options GEOM_RAID # Soft RAID functionality. +options GEOM_LABEL # Provides labelization +options EFIRT # EFI Runtime Services support +options COMPAT_FREEBSD32 # Compatible with i386 binaries +options COMPAT_FREEBSD4 # Compatible with FreeBSD4 +options COMPAT_FREEBSD5 # Compatible with FreeBSD5 +options COMPAT_FREEBSD6 # Compatible with FreeBSD6 +options COMPAT_FREEBSD7 # Compatible with FreeBSD7 +options COMPAT_FREEBSD9 # Compatible with FreeBSD9 +options COMPAT_FREEBSD10 # Compatible with FreeBSD10 +options COMPAT_FREEBSD11 # Compatible with FreeBSD11 +options COMPAT_FREEBSD12 # Compatible with FreeBSD12 +options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI +options KTRACE # ktrace(1) support +options STACK # stack(9) support +options SYSVSHM # SYSV-style shared memory +options SYSVMSG # SYSV-style message queues +options SYSVSEM # SYSV-style semaphores +options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions +options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. +options KBD_INSTALL_CDEV # install a CDEV entry in /dev +options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) +options AUDIT # Security event auditing +options CAPABILITY_MODE # Capsicum capability mode +options CAPABILITIES # Capsicum capabilities +options MAC # TrustedBSD MAC Framework +options KDTRACE_FRAME # Ensure frames are compiled in +options KDTRACE_HOOKS # Kernel DTrace hooks +options DDB_CTF # Kernel ELF linker loads CTF data +options INCLUDE_CONFIG_FILE # Include this file in kernel +options RACCT # Resource accounting framework +options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default +options RCTL # Resource limits +options KERN_TLS + +# Debugging support. Always need this: +options KDB # Enable kernel debugger support. +options KDB_TRACE # Print a stack trace for a panic. +# For full debugger support use (turn off in stable branch): +options BUF_TRACKING # Track buffer history +options DDB # Support DDB. +options FULL_BUF_TRACKING # Track more buffer history +options GDB # Support remote GDB. +options DEADLKRES # Enable the deadlock resolver +options INVARIANTS # Enable calls of extra sanity checking +options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS +options QUEUE_MACRO_DEBUG_TRASH # Trash queue(2) internal pointers on invalidation +options WITNESS # Enable checks to detect deadlocks and cycles +options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed +options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones +options VERBOSE_SYSINIT=0 # Support debug.verbose_sysinit, off by default + +# Kernel Sanitizers +#options COVERAGE # Generic kernel coverage. Used by KCOV +#options KCOV # Kernel Coverage Sanitizer +# Warning: KUBSAN can result in a kernel too large for loader to load +#options KUBSAN # Kernel Undefined Behavior Sanitizer +#options KCSAN # Kernel Concurrency Sanitizer + +# Kernel dump features. +options EKCD # Support for encrypted kernel dumps +options GZIO # gzip-compressed kernel and user dumps +options ZSTDIO # zstd-compressed kernel and user dumps +options DEBUGNET # debugnet networking +options NETDUMP # netdump(4) client support +options NETGDB # netgdb(4) client support + +# Make an SMP-capable kernel by default +options SMP # Symmetric MultiProcessor Kernel +options EARLY_AP_STARTUP + +# CPU frequency control +device cpufreq + +# Bus support. +device acpi +options ACPI_DMAR +device pci +options PCI_HP # PCI-Express native HotPlug +options PCI_IOV # PCI SR-IOV support + +# Floppy drives +device fdc + +# ATA controllers +device ahci # AHCI-compatible SATA controllers +device ata # Legacy ATA/SATA controllers +device mvs # Marvell 88SX50XX/88SX60XX/88SX70XX/SoC SATA +device siis # SiliconImage SiI3124/SiI3132/SiI3531 SATA + +# SCSI Controllers +device ahc # AHA2940 and onboard AIC7xxx devices +device ahd # AHA39320/29320 and onboard AIC79xx devices +device esp # AMD Am53C974 (Tekram DC-390(T)) +device hptiop # Highpoint RocketRaid 3xxx series +device isp # Qlogic family +#device ispfw # Firmware for QLogic HBAs- normally a module +device mpt # LSI-Logic MPT-Fusion +device mps # LSI-Logic MPT-Fusion 2 +device mpr # LSI-Logic MPT-Fusion 3 +device sym # NCR/Symbios Logic +device isci # Intel C600 SAS controller +device ocs_fc # Emulex FC adapters +device pvscsi # VMware PVSCSI + +# ATA/SCSI peripherals +device scbus # SCSI bus (required for ATA/SCSI) +device ch # SCSI media changers +device da # Direct Access (disks) +device sa # Sequential Access (tape etc) +device cd # CD +device pass # Passthrough device (direct ATA/SCSI access) +device ses # Enclosure Services (SES and SAF-TE) +#device ctl # CAM Target Layer + +# RAID controllers interfaced to the SCSI subsystem +device amr # AMI MegaRAID +device arcmsr # Areca SATA II RAID +device ciss # Compaq Smart RAID 5* +device iir # Intel Integrated RAID +device ips # IBM (Adaptec) ServeRAID +device mly # Mylex AcceleRAID/eXtremeRAID +device twa # 3ware 9000 series PATA/SATA RAID +device smartpqi # Microsemi smartpqi driver +device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller + +# RAID controllers +device aac # Adaptec FSA RAID +device aacp # SCSI passthrough for aac (requires CAM) +device aacraid # Adaptec by PMC RAID +device ida # Compaq Smart RAID +device mfi # LSI MegaRAID SAS +device mlx # Mylex DAC960 family +device mrsas # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s +device pmspcv # PMC-Sierra SAS/SATA Controller driver +#XXX pointer/int warnings +#device pst # Promise Supertrak SX6000 +device twe # 3ware ATA RAID + +# NVM Express (NVMe) support +device nvme # base NVMe driver +device nvd # expose NVMe namespaces as disks, depends on nvme + +# Intel Volume Management Device (VMD) support +device vmd # base VMD device +device vmd_bus # bus for VMD children + +# atkbdc0 controls both the keyboard and the PS/2 mouse +device atkbdc # AT keyboard controller +device atkbd # AT keyboard +device psm # PS/2 mouse + +device kbdmux # keyboard multiplexer + +device vga # VGA video card driver +options VESA # Add support for VESA BIOS Extensions (VBE) + +device splash # Splash screen and screen saver support + +# syscons is the default console driver, resembling an SCO console +device sc +options SC_PIXEL_MODE # add support for the raster text mode + +# vt is the new video console driver +device vt +device vt_vga +device vt_efifb + +device agp # support several AGP chipsets + +# PCCARD (PCMCIA) support +# PCMCIA and cardbus bridge support +device cbb # cardbus (yenta) bridge +device pccard # PC Card (16-bit) bus +device cardbus # CardBus (32-bit) bus + +# Serial (COM) ports +device uart # Generic UART driver + +# Parallel port +device ppc +device ppbus # Parallel port bus (required) +device lpt # Printer +device ppi # Parallel port interface device +#device vpo # Requires scbus and da + +device puc # Multi I/O cards and multi-channel UARTs + +# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure +device iflib +device em # Intel PRO/1000 Gigabit Ethernet Family +device ix # Intel PRO/10GbE PCIE PF Ethernet +device ixv # Intel PRO/10GbE PCIE VF Ethernet +device ixl # Intel 700 Series Physical Function +device iavf # Intel Adaptive Virtual Function +device vmx # VMware VMXNET3 Ethernet + +# PCI Ethernet NICs. +device bxe # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE +device le # AMD Am7900 LANCE and Am79C9xx PCnet +device ti # Alteon Networks Tigon I/II gigabit Ethernet + +# PCI Ethernet NICs that use the common MII bus controller code. +# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! +device miibus # MII bus support +device ae # Attansic/Atheros L2 FastEthernet +device age # Attansic/Atheros L1 Gigabit Ethernet +device alc # Atheros AR8131/AR8132 Ethernet +device ale # Atheros AR8121/AR8113/AR8114 Ethernet +device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet +device bfe # Broadcom BCM440x 10/100 Ethernet +device bge # Broadcom BCM570xx Gigabit Ethernet +device cas # Sun Cassini/Cassini+ and NS DP83065 Saturn +device dc # DEC/Intel 21143 and various workalikes +device et # Agere ET1310 10/100/Gigabit Ethernet +device fxp # Intel EtherExpress PRO/100B (82557, 82558) +device gem # Sun GEM/Sun ERI/Apple GMAC +device hme # Sun HME (Happy Meal Ethernet) +device jme # JMicron JMC250 Gigabit/JMC260 Fast Ethernet +device lge # Level 1 LXT1001 gigabit Ethernet +device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet +device nfe # nVidia nForce MCP on-board Ethernet +device nge # NatSemi DP83820 gigabit Ethernet +device re # RealTek 8139C+/8169/8169S/8110S +device rl # RealTek 8129/8139 +device sge # Silicon Integrated Systems SiS190/191 +device sis # Silicon Integrated Systems SiS 900/SiS 7016 +device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet +device ste # Sundance ST201 (D-Link DFE-550TX) +device stge # Sundance/Tamarack TC9021 gigabit Ethernet +device vge # VIA VT612x gigabit Ethernet +device vr # VIA Rhine, Rhine II +device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') + +# Wireless NIC cards +device wlan # 802.11 support +options IEEE80211_DEBUG # enable debug msgs +options IEEE80211_SUPPORT_MESH # enable 802.11s draft support +device wlan_wep # 802.11 WEP support +device wlan_ccmp # 802.11 CCMP support +device wlan_tkip # 802.11 TKIP support +device wlan_amrr # AMRR transmit rate control algorithm +device an # Aironet 4500/4800 802.11 wireless NICs. +device ath # Atheros NICs +device ath_pci # Atheros pci/cardbus glue +device ath_hal # pci/cardbus chip support +options AH_AR5416_INTERRUPT_MITIGATION # AR5416 interrupt mitigation +options ATH_ENABLE_11N # Enable 802.11n support for AR5416 and later +device ath_rate_sample # SampleRate tx rate control for ath +#device bwi # Broadcom BCM430x/BCM431x wireless NICs. +#device bwn # Broadcom BCM43xx wireless NICs. +device ipw # Intel 2100 wireless NICs. +device iwi # Intel 2200BG/2225BG/2915ABG wireless NICs. +device iwn # Intel 4965/1000/5000/6000 wireless NICs. +device malo # Marvell Libertas wireless NICs. +device mwl # Marvell 88W8363 802.11n wireless NICs. +device ral # Ralink Technology RT2500 wireless NICs. +device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. +device wpi # Intel 3945ABG wireless NICs. + +# Pseudo devices. +device crypto # core crypto support +device loop # Network loopback +device padlock_rng # VIA Padlock RNG +device rdrand_rng # Intel Bull Mountain RNG +device ether # Ethernet support +device vlan # 802.1Q VLAN support +device tuntap # Packet tunnel. +device md # Memory "disks" +device gif # IPv6 and IPv4 tunneling +device firmware # firmware assist module + +# The `bpf' device enables the Berkeley Packet Filter. +# Be aware of the administrative consequences of enabling this! +# Note that 'bpf' is required for DHCP. +device bpf # Berkeley packet filter + +# USB support +options USB_DEBUG # enable debug msgs +device uhci # UHCI PCI->USB interface +device ohci # OHCI PCI->USB interface +device ehci # EHCI PCI->USB interface (USB 2.0) +device xhci # XHCI PCI->USB interface (USB 3.0) +device usb # USB Bus (required) +device ukbd # Keyboard +device umass # Disks/Mass storage - Requires scbus and da + +# Sound support +device sound # Generic sound driver (required) +device snd_cmi # CMedia CMI8338/CMI8738 +device snd_csa # Crystal Semiconductor CS461x/428x +device snd_emu10kx # Creative SoundBlaster Live! and Audigy +device snd_es137x # Ensoniq AudioPCI ES137x +device snd_hda # Intel High Definition Audio +device snd_ich # Intel, NVidia and other ICH AC'97 Audio +device snd_via8233 # VIA VT8233x Audio + +# MMC/SD +device mmc # MMC/SD bus +device mmcsd # MMC/SD memory card +device sdhci # Generic PCI SD Host Controller + +# VirtIO support +device virtio # Generic VirtIO bus (required) +device virtio_pci # VirtIO PCI device +device vtnet # VirtIO Ethernet device +device virtio_blk # VirtIO Block device +device virtio_scsi # VirtIO SCSI device +device virtio_balloon # VirtIO Memory Balloon device + +# HyperV drivers and enhancement support +device hyperv # HyperV drivers + +# Xen HVM Guest Optimizations +# NOTE: XENHVM depends on xenpci. They must be added or removed together. +# options XENHVM # Xen HVM kernel infrastructure +# device xenpci # Xen HVM Hypervisor services driver + +# Netmap provides direct access to TX/RX rings on supported NICs +device netmap # netmap(4) support + +# evdev interface +options EVDEV_SUPPORT # evdev support in legacy drivers +device evdev # input event device support +device uinput # install /dev/uinput cdev From owner-svn-src-projects@freebsd.org Sun May 3 01:05:24 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 69C3E2CAEC0 for ; Sun, 3 May 2020 01:05:24 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F78427r7z3xBS; Sun, 3 May 2020 01:05:24 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 43CF12A4A; Sun, 3 May 2020 01:05:24 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04315OXB029188; Sun, 3 May 2020 01:05:24 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04315Ou8029187; Sun, 3 May 2020 01:05:24 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030105.04315Ou8029187@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 01:05:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360586 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 360586 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 01:05:24 -0000 Author: rmacklem Date: Sun May 3 01:05:23 2020 New Revision: 360586 URL: https://svnweb.freebsd.org/changeset/base/360586 Log: Disable the code that copied received mbufs to ext_pgs mbufs. This code is only useful for testing the NFS ext_pgs mbuf handling code. Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc_vc.c Sun May 3 00:51:14 2020 (r360585) +++ projects/nfs-over-tls/sys/rpc/svc_vc.c Sun May 3 01:05:23 2020 (r360586) @@ -596,6 +596,7 @@ svc_vc_process_pending(SVCXPRT *xprt) struct socket *so = xprt->xp_socket; struct mbuf *m; +#ifdef notnow { struct mbuf *m1, *m2, *m3, *m4; int txxxx; m3 = cd->mpending; @@ -620,6 +621,7 @@ svc_vc_process_pending(SVCXPRT *xprt) } } } +#endif /* * If cd->resid is non-zero, we have part of the * record already, otherwise we are expecting a record @@ -876,6 +878,7 @@ svc_vc_backchannel_recv(SVCXPRT *xprt, struct rpc_msg sx_xunlock(&xprt->xp_lock); printf("recv backch m=%p\n", m); +#ifdef notnow { struct mbuf *m1, *m2; int txxxx; if (m != NULL) { @@ -887,6 +890,7 @@ m2 = m; m = m1; m_freem(m2); } } } +#endif xdrmbuf_create(&xdrs, m, XDR_DECODE); if (! xdr_callmsg(&xdrs, msg)) { printf("recv backch callmsg failed\n"); From owner-svn-src-projects@freebsd.org Sun May 3 01:20:39 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 679CB2CBDC1 for ; Sun, 3 May 2020 01:20:39 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F7Tg28Qhz3ySv; Sun, 3 May 2020 01:20:39 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 430DF2C95; Sun, 3 May 2020 01:20:39 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0431KduO036096; Sun, 3 May 2020 01:20:39 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0431KdWU036095; Sun, 3 May 2020 01:20:39 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030120.0431KdWU036095@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 01:20:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360587 - in projects/nfs-over-tls/usr.sbin: . ac accton acpi adduser amd ancontrol apm apmd arp audit auditd auditdistd auditreduce authpf autofs bhyve bhyvectl bhyveload binmiscctl bl... X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/usr.sbin: . ac accton acpi adduser amd ancontrol apm apmd arp audit auditd auditdistd auditreduce authpf autofs bhyve bhyvectl bhyveload binmiscctl blacklistctl blacklistd blu... X-SVN-Commit-Revision: 360587 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 01:20:39 -0000 Author: rmacklem Date: Sun May 3 01:20:38 2020 New Revision: 360587 URL: https://svnweb.freebsd.org/changeset/base/360587 Log: Clear out all the cruft I created under usr.sbin, so that only the three modified/added utilities are left under usr.sbin. Deleted: projects/nfs-over-tls/usr.sbin/Makefile projects/nfs-over-tls/usr.sbin/Makefile.amd64 projects/nfs-over-tls/usr.sbin/Makefile.arm projects/nfs-over-tls/usr.sbin/Makefile.arm64 projects/nfs-over-tls/usr.sbin/Makefile.i386 projects/nfs-over-tls/usr.sbin/Makefile.inc projects/nfs-over-tls/usr.sbin/Makefile.mips projects/nfs-over-tls/usr.sbin/Makefile.powerpc projects/nfs-over-tls/usr.sbin/Makefile.riscv projects/nfs-over-tls/usr.sbin/Makefile.sparc64 projects/nfs-over-tls/usr.sbin/ac/ projects/nfs-over-tls/usr.sbin/accton/ projects/nfs-over-tls/usr.sbin/acpi/ projects/nfs-over-tls/usr.sbin/adduser/ projects/nfs-over-tls/usr.sbin/amd/ projects/nfs-over-tls/usr.sbin/ancontrol/ projects/nfs-over-tls/usr.sbin/apm/ projects/nfs-over-tls/usr.sbin/apmd/ projects/nfs-over-tls/usr.sbin/arp/ projects/nfs-over-tls/usr.sbin/audit/ projects/nfs-over-tls/usr.sbin/auditd/ projects/nfs-over-tls/usr.sbin/auditdistd/ projects/nfs-over-tls/usr.sbin/auditreduce/ projects/nfs-over-tls/usr.sbin/authpf/ projects/nfs-over-tls/usr.sbin/autofs/ projects/nfs-over-tls/usr.sbin/bhyve/ projects/nfs-over-tls/usr.sbin/bhyvectl/ projects/nfs-over-tls/usr.sbin/bhyveload/ projects/nfs-over-tls/usr.sbin/binmiscctl/ projects/nfs-over-tls/usr.sbin/blacklistctl/ projects/nfs-over-tls/usr.sbin/blacklistd/ projects/nfs-over-tls/usr.sbin/bluetooth/ projects/nfs-over-tls/usr.sbin/boot0cfg/ projects/nfs-over-tls/usr.sbin/bootparamd/ projects/nfs-over-tls/usr.sbin/bsdconfig/ projects/nfs-over-tls/usr.sbin/bsdinstall/ projects/nfs-over-tls/usr.sbin/bsnmpd/ projects/nfs-over-tls/usr.sbin/btxld/ projects/nfs-over-tls/usr.sbin/camdd/ projects/nfs-over-tls/usr.sbin/cdcontrol/ projects/nfs-over-tls/usr.sbin/certctl/ projects/nfs-over-tls/usr.sbin/chkgrp/ projects/nfs-over-tls/usr.sbin/chown/ projects/nfs-over-tls/usr.sbin/chroot/ projects/nfs-over-tls/usr.sbin/ckdist/ projects/nfs-over-tls/usr.sbin/clear_locks/ projects/nfs-over-tls/usr.sbin/config/ projects/nfs-over-tls/usr.sbin/cpucontrol/ projects/nfs-over-tls/usr.sbin/crashinfo/ projects/nfs-over-tls/usr.sbin/cron/ projects/nfs-over-tls/usr.sbin/crunch/ projects/nfs-over-tls/usr.sbin/ctladm/ projects/nfs-over-tls/usr.sbin/ctld/ projects/nfs-over-tls/usr.sbin/cxgbetool/ projects/nfs-over-tls/usr.sbin/daemon/ projects/nfs-over-tls/usr.sbin/dconschat/ projects/nfs-over-tls/usr.sbin/devctl/ projects/nfs-over-tls/usr.sbin/devinfo/ projects/nfs-over-tls/usr.sbin/diskinfo/ projects/nfs-over-tls/usr.sbin/dumpcis/ projects/nfs-over-tls/usr.sbin/editmap/ projects/nfs-over-tls/usr.sbin/edquota/ projects/nfs-over-tls/usr.sbin/eeprom/ projects/nfs-over-tls/usr.sbin/efibootmgr/ projects/nfs-over-tls/usr.sbin/efidp/ projects/nfs-over-tls/usr.sbin/efivar/ projects/nfs-over-tls/usr.sbin/etcupdate/ projects/nfs-over-tls/usr.sbin/extattr/ projects/nfs-over-tls/usr.sbin/extattrctl/ projects/nfs-over-tls/usr.sbin/fdcontrol/ projects/nfs-over-tls/usr.sbin/fdformat/ projects/nfs-over-tls/usr.sbin/fdread/ projects/nfs-over-tls/usr.sbin/fdwrite/ projects/nfs-over-tls/usr.sbin/fifolog/ projects/nfs-over-tls/usr.sbin/flowctl/ projects/nfs-over-tls/usr.sbin/fmtree/ projects/nfs-over-tls/usr.sbin/freebsd-update/ projects/nfs-over-tls/usr.sbin/fstyp/ projects/nfs-over-tls/usr.sbin/ftp-proxy/ projects/nfs-over-tls/usr.sbin/fwcontrol/ projects/nfs-over-tls/usr.sbin/getfmac/ projects/nfs-over-tls/usr.sbin/getpmac/ projects/nfs-over-tls/usr.sbin/gpioctl/ projects/nfs-over-tls/usr.sbin/gssd/ projects/nfs-over-tls/usr.sbin/gstat/ projects/nfs-over-tls/usr.sbin/hyperv/ projects/nfs-over-tls/usr.sbin/i2c/ projects/nfs-over-tls/usr.sbin/ifmcstat/ projects/nfs-over-tls/usr.sbin/inetd/ projects/nfs-over-tls/usr.sbin/iostat/ projects/nfs-over-tls/usr.sbin/iovctl/ projects/nfs-over-tls/usr.sbin/ip6addrctl/ projects/nfs-over-tls/usr.sbin/ipfwpcap/ projects/nfs-over-tls/usr.sbin/iscsid/ projects/nfs-over-tls/usr.sbin/jail/ projects/nfs-over-tls/usr.sbin/jexec/ projects/nfs-over-tls/usr.sbin/jls/ projects/nfs-over-tls/usr.sbin/kbdcontrol/ projects/nfs-over-tls/usr.sbin/kbdmap/ projects/nfs-over-tls/usr.sbin/keyserv/ projects/nfs-over-tls/usr.sbin/kgmon/ projects/nfs-over-tls/usr.sbin/kldxref/ projects/nfs-over-tls/usr.sbin/lastlogin/ projects/nfs-over-tls/usr.sbin/lpr/ projects/nfs-over-tls/usr.sbin/lptcontrol/ projects/nfs-over-tls/usr.sbin/mailstats/ projects/nfs-over-tls/usr.sbin/mailwrapper/ projects/nfs-over-tls/usr.sbin/makefs/ projects/nfs-over-tls/usr.sbin/makemap/ projects/nfs-over-tls/usr.sbin/manctl/ projects/nfs-over-tls/usr.sbin/memcontrol/ projects/nfs-over-tls/usr.sbin/mergemaster/ projects/nfs-over-tls/usr.sbin/mfiutil/ projects/nfs-over-tls/usr.sbin/mixer/ projects/nfs-over-tls/usr.sbin/mld6query/ projects/nfs-over-tls/usr.sbin/mlx5tool/ projects/nfs-over-tls/usr.sbin/mlxcontrol/ projects/nfs-over-tls/usr.sbin/mount_smbfs/ projects/nfs-over-tls/usr.sbin/moused/ projects/nfs-over-tls/usr.sbin/mpsutil/ projects/nfs-over-tls/usr.sbin/mptable/ projects/nfs-over-tls/usr.sbin/mptutil/ projects/nfs-over-tls/usr.sbin/mtest/ projects/nfs-over-tls/usr.sbin/ndiscvt/ projects/nfs-over-tls/usr.sbin/ndp/ projects/nfs-over-tls/usr.sbin/newsyslog/ projects/nfs-over-tls/usr.sbin/nfscbd/ projects/nfs-over-tls/usr.sbin/nfsd/ projects/nfs-over-tls/usr.sbin/nfsdumpstate/ projects/nfs-over-tls/usr.sbin/nfsrevoke/ projects/nfs-over-tls/usr.sbin/nfsuserd/ projects/nfs-over-tls/usr.sbin/ngctl/ projects/nfs-over-tls/usr.sbin/nghook/ projects/nfs-over-tls/usr.sbin/nmtree/ projects/nfs-over-tls/usr.sbin/nologin/ projects/nfs-over-tls/usr.sbin/nscd/ projects/nfs-over-tls/usr.sbin/ntp/ projects/nfs-over-tls/usr.sbin/nvram/ projects/nfs-over-tls/usr.sbin/ofwdump/ projects/nfs-over-tls/usr.sbin/pciconf/ projects/nfs-over-tls/usr.sbin/periodic/ projects/nfs-over-tls/usr.sbin/pkg/ projects/nfs-over-tls/usr.sbin/pmc/ projects/nfs-over-tls/usr.sbin/pmcannotate/ projects/nfs-over-tls/usr.sbin/pmccontrol/ projects/nfs-over-tls/usr.sbin/pmcstat/ projects/nfs-over-tls/usr.sbin/pmcstudy/ projects/nfs-over-tls/usr.sbin/pnfsdscopymr/ projects/nfs-over-tls/usr.sbin/pnfsdsfile/ projects/nfs-over-tls/usr.sbin/pnfsdskill/ projects/nfs-over-tls/usr.sbin/pnpinfo/ projects/nfs-over-tls/usr.sbin/portsnap/ projects/nfs-over-tls/usr.sbin/powerd/ projects/nfs-over-tls/usr.sbin/ppp/ projects/nfs-over-tls/usr.sbin/pppctl/ projects/nfs-over-tls/usr.sbin/praliases/ projects/nfs-over-tls/usr.sbin/praudit/ projects/nfs-over-tls/usr.sbin/prometheus_sysctl_exporter/ projects/nfs-over-tls/usr.sbin/pstat/ projects/nfs-over-tls/usr.sbin/pw/ projects/nfs-over-tls/usr.sbin/pwd_mkdb/ projects/nfs-over-tls/usr.sbin/pwm/ projects/nfs-over-tls/usr.sbin/quot/ projects/nfs-over-tls/usr.sbin/quotaon/ projects/nfs-over-tls/usr.sbin/rarpd/ projects/nfs-over-tls/usr.sbin/repquota/ projects/nfs-over-tls/usr.sbin/rip6query/ projects/nfs-over-tls/usr.sbin/rmt/ projects/nfs-over-tls/usr.sbin/route6d/ projects/nfs-over-tls/usr.sbin/rpc.lockd/ projects/nfs-over-tls/usr.sbin/rpc.statd/ projects/nfs-over-tls/usr.sbin/rpc.umntall/ projects/nfs-over-tls/usr.sbin/rpc.yppasswdd/ projects/nfs-over-tls/usr.sbin/rpc.ypupdated/ projects/nfs-over-tls/usr.sbin/rpc.ypxfrd/ projects/nfs-over-tls/usr.sbin/rpcbind/ projects/nfs-over-tls/usr.sbin/rrenumd/ projects/nfs-over-tls/usr.sbin/rtadvctl/ projects/nfs-over-tls/usr.sbin/rtadvd/ projects/nfs-over-tls/usr.sbin/rtprio/ projects/nfs-over-tls/usr.sbin/rtsold/ projects/nfs-over-tls/usr.sbin/rwhod/ projects/nfs-over-tls/usr.sbin/sa/ projects/nfs-over-tls/usr.sbin/sendmail/ projects/nfs-over-tls/usr.sbin/service/ projects/nfs-over-tls/usr.sbin/services_mkdb/ projects/nfs-over-tls/usr.sbin/sesutil/ projects/nfs-over-tls/usr.sbin/setfib/ projects/nfs-over-tls/usr.sbin/setfmac/ projects/nfs-over-tls/usr.sbin/setpmac/ projects/nfs-over-tls/usr.sbin/simple_httpd/ projects/nfs-over-tls/usr.sbin/smbmsg/ projects/nfs-over-tls/usr.sbin/snapinfo/ projects/nfs-over-tls/usr.sbin/spi/ projects/nfs-over-tls/usr.sbin/spkrtest/ projects/nfs-over-tls/usr.sbin/spray/ projects/nfs-over-tls/usr.sbin/syslogd/ projects/nfs-over-tls/usr.sbin/sysrc/ projects/nfs-over-tls/usr.sbin/tcpdchk/ projects/nfs-over-tls/usr.sbin/tcpdmatch/ projects/nfs-over-tls/usr.sbin/tcpdrop/ projects/nfs-over-tls/usr.sbin/tcpdump/ projects/nfs-over-tls/usr.sbin/tests/ projects/nfs-over-tls/usr.sbin/traceroute/ projects/nfs-over-tls/usr.sbin/traceroute6/ projects/nfs-over-tls/usr.sbin/trim/ projects/nfs-over-tls/usr.sbin/trpt/ projects/nfs-over-tls/usr.sbin/tzsetup/ projects/nfs-over-tls/usr.sbin/uathload/ projects/nfs-over-tls/usr.sbin/uefisign/ projects/nfs-over-tls/usr.sbin/ugidfw/ projects/nfs-over-tls/usr.sbin/uhsoctl/ projects/nfs-over-tls/usr.sbin/unbound/ projects/nfs-over-tls/usr.sbin/usbconfig/ projects/nfs-over-tls/usr.sbin/usbdump/ projects/nfs-over-tls/usr.sbin/utx/ projects/nfs-over-tls/usr.sbin/valectl/ projects/nfs-over-tls/usr.sbin/vidcontrol/ projects/nfs-over-tls/usr.sbin/vigr/ projects/nfs-over-tls/usr.sbin/vipw/ projects/nfs-over-tls/usr.sbin/wake/ projects/nfs-over-tls/usr.sbin/watch/ projects/nfs-over-tls/usr.sbin/watchdogd/ projects/nfs-over-tls/usr.sbin/wlandebug/ projects/nfs-over-tls/usr.sbin/wpa/ projects/nfs-over-tls/usr.sbin/yp_mkdb/ projects/nfs-over-tls/usr.sbin/ypbind/ projects/nfs-over-tls/usr.sbin/ypldap/ projects/nfs-over-tls/usr.sbin/yppoll/ projects/nfs-over-tls/usr.sbin/yppush/ projects/nfs-over-tls/usr.sbin/ypserv/ projects/nfs-over-tls/usr.sbin/ypset/ projects/nfs-over-tls/usr.sbin/zic/ projects/nfs-over-tls/usr.sbin/zonectl/ projects/nfs-over-tls/usr.sbin/zzz/ From owner-svn-src-projects@freebsd.org Sun May 3 01:23:13 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C093A2CC1BD for ; Sun, 3 May 2020 01:23:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F7Xd4hpNz3yy9; Sun, 3 May 2020 01:23:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 83E172E63; Sun, 3 May 2020 01:23:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0431NDVZ041927; Sun, 3 May 2020 01:23:13 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0431NDe9041926; Sun, 3 May 2020 01:23:13 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030123.0431NDe9041926@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 01:23:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360588 - projects/nfs-over-tls/usr.sbin/mountd/mountd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/usr.sbin/mountd/mountd X-SVN-Commit-Revision: 360588 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 01:23:13 -0000 Author: rmacklem Date: Sun May 3 01:23:13 2020 New Revision: 360588 URL: https://svnweb.freebsd.org/changeset/base/360588 Log: Get rid of mountd subdirectory that I somehow created. Deleted: projects/nfs-over-tls/usr.sbin/mountd/mountd/ From owner-svn-src-projects@freebsd.org Sun May 3 02:23:41 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1D8812CF567 for ; Sun, 3 May 2020 02:23:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F8tN73R3z43h6; Sun, 3 May 2020 02:23:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D4B543CA2; Sun, 3 May 2020 02:23:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0432NeGd079149; Sun, 3 May 2020 02:23:40 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0432NeCO079148; Sun, 3 May 2020 02:23:40 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005030223.0432NeCO079148@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 02:23:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360589 - projects/nfs-over-tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls X-SVN-Commit-Revision: 360589 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 02:23:41 -0000 Author: rmacklem Date: Sun May 3 02:23:40 2020 New Revision: 360589 URL: https://svnweb.freebsd.org/changeset/base/360589 Log: Add the patch file for /usr/ktls/include/openssl. Added: projects/nfs-over-tls/openssl3.patch Added: projects/nfs-over-tls/openssl3.patch ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/nfs-over-tls/openssl3.patch Sun May 3 02:23:40 2020 (r360589) @@ -0,0 +1,95 @@ +--- conf.h.sav 2020-05-02 08:36:38.880690000 -0700 ++++ conf.h 2020-05-02 08:37:15.173793000 -0700 +@@ -34,6 +34,7 @@ typedef struct { + } CONF_VALUE; + + DEFINE_OR_DECLARE_STACK_OF(CONF_VALUE) ++typedef struct conf_module_st CONF_MODULE; + DEFINE_OR_DECLARE_STACK_OF(CONF_MODULE) + + DEFINE_LHASH_OF(CONF_VALUE); +@@ -58,7 +59,6 @@ struct conf_method_st { + /* Module definitions */ + + typedef struct conf_imodule_st CONF_IMODULE; +-typedef struct conf_module_st CONF_MODULE; + + STACK_OF(CONF_IMODULE); + +--- ssl.h.sav 2020-05-02 08:36:44.543950000 -0700 ++++ ssl.h 2020-05-02 08:37:15.179047000 -0700 +@@ -230,16 +230,15 @@ typedef struct tls_sigalgs_st TLS_SIGALGS; + typedef struct ssl_conf_ctx_st SSL_CONF_CTX; + typedef struct ssl_comp_st SSL_COMP; + +-DEFINE_OR_DECLARE_STACK_OF(SSL_CIPHER) +-DEFINE_OR_DECLARE_STACK_OF(SSL_COMP) +-DEFINE_OR_DECLARE_STACK_OF(SRTP_PROTECTION_PROFILE) +-DEFINE_OR_DECLARE_STACK_OF(SSL_COMP) +- + /* SRTP protection profiles for use with the use_srtp extension (RFC 5764)*/ + typedef struct srtp_protection_profile_st { + const char *name; + unsigned long id; + } SRTP_PROTECTION_PROFILE; ++ ++DEFINE_OR_DECLARE_STACK_OF(SSL_CIPHER) ++DEFINE_OR_DECLARE_STACK_OF(SSL_COMP) ++DEFINE_OR_DECLARE_STACK_OF(SRTP_PROTECTION_PROFILE) + + + typedef int (*tls_session_ticket_ext_cb_fn)(SSL *s, const unsigned char *data, +--- x509v3.h.sav 2020-05-02 08:36:55.891210000 -0700 ++++ x509v3.h 2020-05-02 08:37:15.179843000 -0700 +@@ -25,25 +25,6 @@ + extern "C" { + #endif + +-DEFINE_OR_DECLARE_STACK_OF(GENERAL_NAME) +-DEFINE_OR_DECLARE_STACK_OF(X509V3_EXT_METHOD) +-DEFINE_OR_DECLARE_STACK_OF(GENERAL_NAMES) +-DEFINE_OR_DECLARE_STACK_OF(ACCESS_DESCRIPTION) +-DEFINE_OR_DECLARE_STACK_OF(DIST_POINT) +-DEFINE_OR_DECLARE_STACK_OF(SXNETID) +-DEFINE_OR_DECLARE_STACK_OF(POLICYQUALINFO) +-DEFINE_OR_DECLARE_STACK_OF(POLICYINFO) +-DEFINE_OR_DECLARE_STACK_OF(POLICY_MAPPING) +-DEFINE_OR_DECLARE_STACK_OF(GENERAL_SUBTREE) +-DEFINE_OR_DECLARE_STACK_OF(X509_PURPOSE) +-DEFINE_OR_DECLARE_STACK_OF(X509_POLICY_NODE) +-DEFINE_OR_DECLARE_STACK_OF(ASIdOrRange) +-DEFINE_OR_DECLARE_STACK_OF(IPAddressOrRange) +-DEFINE_OR_DECLARE_STACK_OF(IPAddressFamily) +-DEFINE_OR_DECLARE_STACK_OF(ASN1_STRING) +-DEFINE_OR_DECLARE_STACK_OF(ADMISSIONS) +-DEFINE_OR_DECLARE_STACK_OF(PROFESSION_INFO) +- + /* Forward reference */ + struct v3_ext_method; + struct v3_ext_ctx; +@@ -892,6 +873,25 @@ DECLARE_ASN1_FUNCTIONS(PROFESSION_INFO) + DECLARE_ASN1_FUNCTIONS(ADMISSIONS) + DECLARE_ASN1_FUNCTIONS(ADMISSION_SYNTAX) + typedef STACK_OF(PROFESSION_INFO) PROFESSION_INFOS; ++ ++DEFINE_OR_DECLARE_STACK_OF(GENERAL_NAME) ++DEFINE_OR_DECLARE_STACK_OF(X509V3_EXT_METHOD) ++DEFINE_OR_DECLARE_STACK_OF(GENERAL_NAMES) ++DEFINE_OR_DECLARE_STACK_OF(ACCESS_DESCRIPTION) ++DEFINE_OR_DECLARE_STACK_OF(DIST_POINT) ++DEFINE_OR_DECLARE_STACK_OF(SXNETID) ++DEFINE_OR_DECLARE_STACK_OF(POLICYQUALINFO) ++DEFINE_OR_DECLARE_STACK_OF(POLICYINFO) ++DEFINE_OR_DECLARE_STACK_OF(POLICY_MAPPING) ++DEFINE_OR_DECLARE_STACK_OF(GENERAL_SUBTREE) ++DEFINE_OR_DECLARE_STACK_OF(X509_PURPOSE) ++DEFINE_OR_DECLARE_STACK_OF(X509_POLICY_NODE) ++DEFINE_OR_DECLARE_STACK_OF(ASIdOrRange) ++DEFINE_OR_DECLARE_STACK_OF(IPAddressOrRange) ++DEFINE_OR_DECLARE_STACK_OF(IPAddressFamily) ++DEFINE_OR_DECLARE_STACK_OF(ASN1_STRING) ++DEFINE_OR_DECLARE_STACK_OF(ADMISSIONS) ++DEFINE_OR_DECLARE_STACK_OF(PROFESSION_INFO) + + const ASN1_OBJECT *NAMING_AUTHORITY_get0_authorityId( + const NAMING_AUTHORITY *n); From owner-svn-src-projects@freebsd.org Sun May 3 21:59:41 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 499142BB2EF for ; Sun, 3 May 2020 21:59:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49FfzK15pJz4Mw5; Sun, 3 May 2020 21:59:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 02A6219DF5; Sun, 3 May 2020 21:59:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 043LxeID014905; Sun, 3 May 2020 21:59:40 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 043Lxe1L014904; Sun, 3 May 2020 21:59:40 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005032159.043Lxe1L014904@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 21:59:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360615 - in projects/nfs-over-tls/usr.sbin: rpctlscd rpctlssd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/usr.sbin: rpctlscd rpctlssd X-SVN-Commit-Revision: 360615 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 21:59:41 -0000 Author: rmacklem Date: Sun May 3 21:59:40 2020 New Revision: 360615 URL: https://svnweb.freebsd.org/changeset/base/360615 Log: Fix handling of ktls not enabled in the daemons. Also, fix a case in the client daemon where it did not obey the "-d" option properly. Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Sun May 3 18:46:03 2020 (r360614) +++ projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Sun May 3 21:59:40 2020 (r360615) @@ -579,10 +579,18 @@ rpctls_connect(SSL_CTX *ctx, int s) NULL, 0); cp2 = X509_NAME_oneline(X509_get_subject_name(cert), NULL, 0); - syslog(LOG_INFO | LOG_DAEMON, "rpctls_connect: client" - " IP %s issuerName=%s subjectName=%s verify " - "failed %s\n", hostnam, cp, cp2, - X509_verify_cert_error_string(ret)); + if (rpctls_debug_level == 0) + syslog(LOG_INFO | LOG_DAEMON, + "rpctls_connect: client IP %s " + "issuerName=%s subjectName=%s verify " + "failed %s\n", hostnam, cp, cp2, + X509_verify_cert_error_string(ret)); + else + fprintf(stderr, + "rpctls_connect: client IP %s " + "issuerName=%s subjectName=%s verify " + "failed %s\n", hostnam, cp, cp2, + X509_verify_cert_error_string(ret)); } SSL_free(ssl); return (NULL); @@ -595,12 +603,14 @@ rpctls_connect(SSL_CTX *ctx, int s) ret = BIO_get_ktls_recv(SSL_get_rbio(ssl)); rpctlscd_verbose_out("rpctls_connect: BIO_get_ktls_recv=%d\n", ret); } -#ifdef notnow if (ret == 0) { + if (rpctls_debug_level == 0) + syslog(LOG_ERR, "ktls not working\n"); + else + fprintf(stderr, "ktls not working\n"); SSL_free(ssl); return (NULL); } -#endif return (ssl); } Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Sun May 3 18:46:03 2020 (r360614) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Sun May 3 21:59:40 2020 (r360615) @@ -672,12 +672,17 @@ rpctlssd_verbose_out("%s\n", cp2); ret = BIO_get_ktls_recv(SSL_get_rbio(ssl)); rpctlssd_verbose_out("rpctls_server: BIO_get_ktls_recv=%d\n", ret); } -#ifdef notnow if (ret == 0) { - SSL_free(ssl); - return (NULL); + if (rpctls_debug_level == 0) + syslog(LOG_ERR, "ktls not working\n"); + else + fprintf(stderr, "ktls not working\n"); + /* + * The handshake has completed, so all that can be + * done is disable the connection. + */ + *flags |= RPCTLS_FLAGS_DISABLED; } -#endif return (ssl); } From owner-svn-src-projects@freebsd.org Sun May 3 23:15:17 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BBBFF2BCADB for ; Sun, 3 May 2020 23:15:17 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49FhfY3vTGz4Qv4; Sun, 3 May 2020 23:15:17 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 62DD91ACC9; Sun, 3 May 2020 23:15:17 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 043NFHcH065476; Sun, 3 May 2020 23:15:17 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 043NFFsp065458; Sun, 3 May 2020 23:15:15 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005032315.043NFFsp065458@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 3 May 2020 23:15:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360616 - in projects/nfs-over-tls/sys: fs/nfs fs/nfsclient fs/nfsserver kern rpc rpc/rpcsec_tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys: fs/nfs fs/nfsclient fs/nfsserver kern rpc rpc/rpcsec_tls X-SVN-Commit-Revision: 360616 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 May 2020 23:15:17 -0000 Author: rmacklem Date: Sun May 3 23:15:14 2020 New Revision: 360616 URL: https://svnweb.freebsd.org/changeset/base/360616 Log: Add a little function to acquire the ktls sysctls. Add rpctls_getinfo() to acquire the ktls sysctls that the krpc and nfs code needs to have. This avoids making the variables global. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c projects/nfs-over-tls/sys/kern/uipc_ktls.c projects/nfs-over-tls/sys/rpc/clnt_bck.c projects/nfs-over-tls/sys/rpc/clnt_vc.c projects/nfs-over-tls/sys/rpc/rpcsec_tls.h projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c projects/nfs-over-tls/sys/rpc/svc_vc.c Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sun May 3 23:15:14 2020 (r360616) @@ -51,6 +51,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include @@ -87,9 +88,6 @@ extern volatile int nfsrv_devidcnt; extern int nfscl_debuglevel; extern struct nfsdevicehead nfsrv_devidhead; extern struct nfsstatsv1 nfsstatsv1; -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif SYSCTL_DECL(_vfs_nfs); SYSCTL_INT(_vfs_nfs, OID_AUTO, enable_uidtostring, CTLFLAG_RW, @@ -340,6 +338,9 @@ nfscl_reqstart(struct nfsrv_descript *nd, int procnum, u_int32_t *tl; int opcnt; nfsattrbit_t attrbits; +#ifdef KERN_TLS + u_int maxlen; +#endif /* * First, fill in some of the fields of nd. @@ -371,8 +372,9 @@ nfscl_reqstart(struct nfsrv_descript *nd, int procnum, if (use_ext && PMAP_HAS_DMAP != 0) { nd->nd_flag |= ND_NOMAP; #ifdef KERN_TLS - nd->nd_maxextsiz = min(TLS_MAX_MSG_SIZE_V10_2, - ktls_maxlen); + if (rpctls_getinfo(&maxlen)) + nd->nd_maxextsiz = min(TLS_MAX_MSG_SIZE_V10_2, + maxlen); #endif } Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Sun May 3 23:15:14 2020 (r360616) @@ -57,9 +57,6 @@ extern u_long sb_max_adj; extern int nfs_numnfscbd; extern int nfscl_debuglevel; extern bool nfs_use_ext_pgs; -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif /* * NFS client system calls for handling callbacks. @@ -73,6 +70,9 @@ nfscb_program(struct svc_req *rqst, SVCXPRT *xprt) { struct nfsrv_descript nd; int cacherep, credflavor; +#ifdef KERN_TLS + u_int maxlen; +#endif printf("cbprogram proc=%d\n", rqst->rq_proc); memset(&nd, 0, sizeof(nd)); @@ -121,9 +121,10 @@ printf("cbreq nd_md=%p offs=%d\n", nd.nd_md, rqst->rq_ nd.nd_flag |= ND_NOMAP; nd.nd_maxextsiz = 16384; #ifdef KERN_TLS - if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0) + if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0 && + rpctls_getinfo(&maxlen)) nd.nd_maxextsiz = min(TLS_MAX_MSG_SIZE_V10_2, - ktls_maxlen); + maxlen); #endif } cacherep = nfs_cbproc(&nd, rqst->rq_xid); Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Sun May 3 23:15:14 2020 (r360616) @@ -53,6 +53,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include SYSCTL_DECL(_vfs_nfs); @@ -78,9 +79,6 @@ extern int nfs_pnfsiothreads; extern u_long sb_max_adj; extern int nfs_maxcopyrange; extern bool nfs_use_ext_pgs; -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif NFSCLSTATEMUTEX; int nfstest_outofseq = 0; int nfscl_assumeposixlocks = 1; @@ -5782,6 +5780,9 @@ nfscl_doiods(vnode_t vp, struct uio *uiop, int *iomode ssize_t resid = 0; int maxextsiz; bool doextpgs; +#ifdef KERN_TLS + u_int maxlen; +#endif if (!NFSHASPNFS(nmp) || nfscl_enablecallb == 0 || nfs_numnfscbd == 0 || (np->n_flag & NNOLAYOUT) != 0) @@ -5884,9 +5885,10 @@ nfscl_doiods(vnode_t vp, struct uio *uiop, int *iomode doextpgs = true; maxextsiz = 16384; #ifdef KERN_TLS - maxextsiz = min( - TLS_MAX_MSG_SIZE_V10_2, - ktls_maxlen); + if (rpctls_getinfo(&maxlen)) + maxextsiz = min( + TLS_MAX_MSG_SIZE_V10_2, + maxlen); #endif } m = nfsm_uiombuflist(doextpgs, Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c Sun May 3 23:15:14 2020 (r360616) @@ -111,9 +111,6 @@ extern time_t nfsdev_time; extern int nfsrv_writerpc[NFS_NPROCS]; extern volatile int nfsrv_devidcnt; extern struct nfsv4_opflag nfsv4_opflag[NFSV42_NOPS]; -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif /* * NFS server system calls @@ -125,6 +122,9 @@ nfssvc_program(struct svc_req *rqst, SVCXPRT *xprt) struct nfsrv_descript nd; struct nfsrvcache *rp = NULL; int cacherep, credflavor; +#ifdef KERN_TLS + u_int maxlen; +#endif memset(&nd, 0, sizeof(nd)); if (rqst->rq_vers == NFS_VER2) { @@ -282,9 +282,10 @@ nfssvc_program(struct svc_req *rqst, SVCXPRT *xprt) } #ifdef KERN_TLS - if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0) + if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0 && + rpctls_getinfo(&maxlen)) nd.nd_maxextsiz = min(TLS_MAX_MSG_SIZE_V10_2, - ktls_maxlen); + maxlen); #endif cacherep = nfs_proc(&nd, rqst->rq_xid, xprt, &rp); NFSLOCKV4ROOTMUTEX(); Modified: projects/nfs-over-tls/sys/kern/uipc_ktls.c ============================================================================== --- projects/nfs-over-tls/sys/kern/uipc_ktls.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/kern/uipc_ktls.c Sun May 3 23:15:14 2020 (r360616) @@ -109,7 +109,7 @@ SYSCTL_INT(_kern_ipc_tls, OID_AUTO, bind_threads, CTLF &ktls_bind_threads, 0, "Bind crypto threads to cores or domains at boot"); -u_int ktls_maxlen = 16384; +static u_int ktls_maxlen = 16384; SYSCTL_UINT(_kern_ipc_tls, OID_AUTO, maxlen, CTLFLAG_RWTUN, &ktls_maxlen, 0, "Maximum TLS record size"); Modified: projects/nfs-over-tls/sys/rpc/clnt_bck.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_bck.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/rpc/clnt_bck.c Sun May 3 23:15:14 2020 (r360616) @@ -89,9 +89,6 @@ __FBSDID("$FreeBSD$"); #include #include -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif struct cmessage { struct cmsghdr cmsg; @@ -213,6 +210,9 @@ clnt_bck_call( struct ct_request *cr; int error, maxextsiz; uint32_t junk; +#ifdef KERN_TLS + u_int maxlen; +#endif cr = malloc(sizeof(struct ct_request), M_RPC, M_WAITOK); @@ -313,7 +313,8 @@ call_again: */ maxextsiz = TLS_MAX_MSG_SIZE_V10_2; #ifdef KERN_TLS - maxextsiz = min(maxextsiz, ktls_maxlen); + if (rpctls_getinfo(&maxlen)) + maxextsiz = min(maxextsiz, maxlen); #endif mreq = _rpc_copym_into_ext_pgs(mreq, maxextsiz); } Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Sun May 3 23:15:14 2020 (r360616) @@ -86,9 +86,6 @@ __FBSDID("$FreeBSD$"); #include #include -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif struct cmessage { struct cmsghdr cmsg; @@ -312,6 +309,9 @@ clnt_vc_call( struct mbuf *mreq = NULL, *results; struct ct_request *cr; int error, maxextsiz, trycnt; +#ifdef KERN_TLS + u_int maxlen; +#endif cr = malloc(sizeof(struct ct_request), M_RPC, M_WAITOK); @@ -424,7 +424,8 @@ call_again: */ maxextsiz = TLS_MAX_MSG_SIZE_V10_2; #ifdef KERN_TLS - maxextsiz = min(maxextsiz, ktls_maxlen); + if (rpctls_getinfo(&maxlen)) + maxextsiz = min(maxextsiz, maxlen); #endif mreq = _rpc_copym_into_ext_pgs(mreq, maxextsiz); } Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Sun May 3 23:15:14 2020 (r360616) @@ -54,6 +54,9 @@ enum clnt_stat rpctls_srv_disconnect(uint64_t sec, uin /* Initialization function for rpcsec_tls. */ int rpctls_init(void); +/* Get TLS information function. */ +bool rpctls_getinfo(u_int *maxlen); + /* String for AUTH_TLS reply verifier. */ #define RPCTLS_START_STRING "STARTTLS" Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Sun May 3 23:15:14 2020 (r360616) @@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -124,6 +125,9 @@ sys_gssd_syscall(struct thread *td, struct gssd_syscal int fd = -1, error, retry_count = 5; CLIENT *cl, *oldcl; bool ssd; +#ifdef KERN_TLS + u_int maxlen; +#endif printf("in gssd syscall\n"); error = priv_check(td, PRIV_NFS_DAEMON); @@ -223,7 +227,8 @@ printf("cl=%p oldcl=%p\n", cl, oldcl); printf("In connect\n"); error = EINVAL; #ifdef KERN_TLS - if (PMAP_HAS_DMAP != 0) + if (PMAP_HAS_DMAP != 0 && mb_use_ext_pgs && + rpctls_getinfo(&maxlen)) error = 0; #endif if (error == 0) @@ -242,7 +247,8 @@ printf("returning=%d\n", fd); printf("In srvconnect\n"); error = EINVAL; #ifdef KERN_TLS - if (PMAP_HAS_DMAP != 0) + if (PMAP_HAS_DMAP != 0 && mb_use_ext_pgs && + rpctls_getinfo(&maxlen)) error = 0; #endif if (error == 0) @@ -574,5 +580,30 @@ printf("got uid=%d ngrps=%d gidp=%p\n", uid, ngrps, gi printf("authtls: aft handshake stat=%d\n", stat); return (RPCSEC_GSS_NODISPATCH); +} + +/* + * Get kern.ipc.tls.enable and kern.ipc.tls.maxlen. + */ +bool +rpctls_getinfo(u_int *maxlenp) +{ + u_int maxlen; + bool enable; + int error; + size_t siz; + + siz = sizeof(enable); + error = kernel_sysctlbyname(curthread, "kern.ipc.tls.enable", + &enable, &siz, NULL, 0, NULL, 0); + if (error != 0) + return (false); + siz = sizeof(maxlen); + error = kernel_sysctlbyname(curthread, "kern.ipc.tls.maxlen", + &maxlen, &siz, NULL, 0, NULL, 0); + if (error != 0) + return (false); + *maxlenp = maxlen; + return (enable); } Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc_vc.c Sun May 3 21:59:40 2020 (r360615) +++ projects/nfs-over-tls/sys/rpc/svc_vc.c Sun May 3 23:15:14 2020 (r360616) @@ -76,9 +76,6 @@ __FBSDID("$FreeBSD$"); #include -#ifdef KERN_TLS -extern u_int ktls_maxlen; -#endif static bool_t svc_vc_rendezvous_recv(SVCXPRT *, struct rpc_msg *, struct sockaddr **, struct mbuf **); @@ -916,6 +913,9 @@ svc_vc_reply(SVCXPRT *xprt, struct rpc_msg *msg, struct mbuf *mrep; bool_t stat = TRUE; int error, len, maxextsiz; +#ifdef KERN_TLS + u_int maxlen; +#endif /* * Leave space for record mark. @@ -954,7 +954,8 @@ svc_vc_reply(SVCXPRT *xprt, struct rpc_msg *msg, */ maxextsiz = TLS_MAX_MSG_SIZE_V10_2; #ifdef KERN_TLS - maxextsiz = min(maxextsiz, ktls_maxlen); + if (rpctls_getinfo(&maxlen)) + maxextsiz = min(maxextsiz, maxlen); #endif mrep = _rpc_copym_into_ext_pgs(mrep, maxextsiz); } @@ -989,6 +990,9 @@ svc_vc_backchannel_reply(SVCXPRT *xprt, struct rpc_msg struct mbuf *mrep; bool_t stat = TRUE; int error, maxextsiz; +#ifdef KERN_TLS + u_int maxlen; +#endif /* * Leave space for record mark. @@ -1027,7 +1031,8 @@ svc_vc_backchannel_reply(SVCXPRT *xprt, struct rpc_msg */ maxextsiz = TLS_MAX_MSG_SIZE_V10_2; #ifdef KERN_TLS - maxextsiz = min(maxextsiz, ktls_maxlen); + if (rpctls_getinfo(&maxlen)) + maxextsiz = min(maxextsiz, maxlen); #endif mrep = _rpc_copym_into_ext_pgs(mrep, maxextsiz); } From owner-svn-src-projects@freebsd.org Mon May 4 23:49:38 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1D8612C74D7 for ; Mon, 4 May 2020 23:49:38 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49GKMj6yplz40Gb; Mon, 4 May 2020 23:49:37 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E5F4E48C0; Mon, 4 May 2020 23:49:37 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 044NnbqW083689; Mon, 4 May 2020 23:49:37 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 044Nnbek083688; Mon, 4 May 2020 23:49:37 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005042349.044Nnbek083688@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 4 May 2020 23:49:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360646 - projects/nfs-over-tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls X-SVN-Commit-Revision: 360646 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 May 2020 23:49:38 -0000 Author: rmacklem Date: Mon May 4 23:49:37 2020 New Revision: 360646 URL: https://svnweb.freebsd.org/changeset/base/360646 Log: Replace openssl3.patch with a slightly cleaner one. Modified: projects/nfs-over-tls/openssl3.patch Modified: projects/nfs-over-tls/openssl3.patch ============================================================================== --- projects/nfs-over-tls/openssl3.patch Mon May 4 23:05:11 2020 (r360645) +++ projects/nfs-over-tls/openssl3.patch Mon May 4 23:49:37 2020 (r360646) @@ -1,18 +1,22 @@ --- conf.h.sav 2020-05-02 08:36:38.880690000 -0700 -+++ conf.h 2020-05-02 08:37:15.173793000 -0700 -@@ -34,6 +34,7 @@ typedef struct { ++++ conf.h 2020-04-03 13:13:12.140750000 -0700 +@@ -33,9 +33,6 @@ typedef struct { + char *value; } CONF_VALUE; - DEFINE_OR_DECLARE_STACK_OF(CONF_VALUE) -+typedef struct conf_module_st CONF_MODULE; - DEFINE_OR_DECLARE_STACK_OF(CONF_MODULE) - +-DEFINE_OR_DECLARE_STACK_OF(CONF_VALUE) +-DEFINE_OR_DECLARE_STACK_OF(CONF_MODULE) +- DEFINE_LHASH_OF(CONF_VALUE); -@@ -58,7 +59,6 @@ struct conf_method_st { - /* Module definitions */ + struct conf_st; +@@ -59,6 +56,9 @@ struct conf_method_st { + typedef struct conf_imodule_st CONF_IMODULE; --typedef struct conf_module_st CONF_MODULE; + typedef struct conf_module_st CONF_MODULE; ++ ++DEFINE_OR_DECLARE_STACK_OF(CONF_VALUE) ++DEFINE_OR_DECLARE_STACK_OF(CONF_MODULE) STACK_OF(CONF_IMODULE); From owner-svn-src-projects@freebsd.org Mon May 4 23:53:46 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A0C9F2C77D5 for ; Mon, 4 May 2020 23:53:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49GKSV3nM8z40m3; Mon, 4 May 2020 23:53:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7D2914A95; Mon, 4 May 2020 23:53:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 044NrkN5089429; Mon, 4 May 2020 23:53:46 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 044NrkAY089428; Mon, 4 May 2020 23:53:46 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005042353.044NrkAY089428@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 4 May 2020 23:53:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360647 - projects/nfs-over-tls/sys/kern X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/kern X-SVN-Commit-Revision: 360647 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 May 2020 23:53:46 -0000 Author: rmacklem Date: Mon May 4 23:53:46 2020 New Revision: 360647 URL: https://svnweb.freebsd.org/changeset/base/360647 Log: Fix a leak in ktls_decrypt(). ktls_decrypt() allocates an iov, but never free's it. This one line change fixes it. It might make more sense to make "iov" and "iov_cap" static instead, but I'll let jhb@ decide. Modified: projects/nfs-over-tls/sys/kern/uipc_ktls.c Modified: projects/nfs-over-tls/sys/kern/uipc_ktls.c ============================================================================== --- projects/nfs-over-tls/sys/kern/uipc_ktls.c Mon May 4 23:49:37 2020 (r360646) +++ projects/nfs-over-tls/sys/kern/uipc_ktls.c Mon May 4 23:53:46 2020 (r360647) @@ -1865,6 +1865,7 @@ ktls_decrypt(struct socket *so) sorwakeup_locked(so); deref: + free(iov, M_KTLS); SOCKBUF_UNLOCK_ASSERT(sb); CURVNET_SET(so->so_vnet); From owner-svn-src-projects@freebsd.org Tue May 5 00:08:42 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 148DD2C8019 for ; Tue, 5 May 2020 00:08:42 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49GKnk05MWz41TZ; Tue, 5 May 2020 00:08:42 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EDC1C4C79; Tue, 5 May 2020 00:08:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04508fdE095637; Tue, 5 May 2020 00:08:41 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04508fb8095636; Tue, 5 May 2020 00:08:41 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005050008.04508fb8095636@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 5 May 2020 00:08:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360649 - projects/nfs-over-tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls X-SVN-Commit-Revision: 360649 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 May 2020 00:08:42 -0000 Author: rmacklem Date: Tue May 5 00:08:41 2020 New Revision: 360649 URL: https://svnweb.freebsd.org/changeset/base/360649 Log: Put a copy of nfs-over-tls-setup.txt here. It is already on freefall so it can be looked at by a web browser, but keeping a copy here makes sense. (I just have to remember to update both.;-) Added: projects/nfs-over-tls/nfs-over-tls-setup.txt (contents, props changed) Added: projects/nfs-over-tls/nfs-over-tls-setup.txt ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/nfs-over-tls/nfs-over-tls-setup.txt Tue May 5 00:08:41 2020 (r360649) @@ -0,0 +1,348 @@ +The first part of this document covers how to set up a test environment. +This will keep changing as things get merged into FreeBSD's head. +Note that, for now, the NFS over TLS uses TLS1.2 and not TLS1.3. +This will change once the kernel tls knows how to do 1.3. + +Setup for testing. For now, setting up test machine(s) is a bit +awkward, but hopefully this lists all the steps. +- Install a recent FreeBSD current system on amd64 system(s). + There are many ways to do this, but downloading and installing a + snapshot is probably the easiest. + Go anonymous ftp onto ftp.freebsd.org + cd pub/FreeBSD/snapshots/ISO-IMAGES/13.0 + - You want to get an install image with "amd64" in it (thats 64bit x86). + The higher the 6digit number, the newer the snapshot. + An iso with "disc1" in the name is a full install image that can be + burned onto a DVD. (I haven't done other types of installs, but there + are several others.) + - Burn the iso onto a DVD (or whatever). + - Install it on a 64bit x86 system. When it asks, include installation + of src. + +(For here on, you might as well login as root.) +Now, you will need a subversion client on some machine. If you do not +already have one, log into the newly installed FreeBSD system and +# pkg install subversion +- It will probably ask you to install "pkg" first and then it should work. + +If you don't yet have /usr/src, you can use "svn" to get it: +# cd /usr/src +# svn checkout http://svn.freebsd.org/base/head . + +Use "svn" to download the modified sources into /usr/nfs-over-tls. +# cd /usr +# mkdir nfs-over-tls +# cd nfs-over-tls +# svn checkout http://svn.freebsd.org/base/projects/nfs-over-tls . + +Now, you can build/install the modified kernel. Here's the "cheater" way +I do it. +# cd /usr/src +# mv sys sys.old +# ln -s /usr/nfs-over-tls/sys sys +# cd /usr/src/sys/amd64/conf +# config MYTEST +# cd ../compile/MYTEST +# make LD=ld.lld SRCTOP=/usr/src cleandepend +# make LD=ld.lld SRCTOP=/usr/src depend +# make LD=ld.lld SRCTOP=/usr/src +# make LD=ld.lld KERNEL=nkernel install +- This puts the new kernel in /boot/nkernel. It can be booted by selecting + '3' at the boot menu and then typing "boot nkernel". + +Fix the include files. Here's my "cheat" way of doing this. +# cd /usr/include +# mv sys sys.old +# ln -s /usr/nfs-over-tls/sys/sys sys +# mv fs fs.old +# ln -s /usr/nfs-over-tls/sys/fs fs +# mv netinet netinet.old +# ln -s /usr/nfs-over-tls/sys/netinet netinet +# cd rpc +# cp /usr/nfs-over-tls/sys/rpc/rpcsec_tls.h . +- This should be enough to allow userspace building. + +Now, you need jhb@'s patched openssl3 source tree, so you can build it. +- If you don't already have one, get a github account. + (If you don't have git anywhere, I think "pkg install git" will get it + installed.) + - You will need perl5. +# pkg install perl +# cd /usr +# mkdir openssl +# cd openssl +# git clone https://github.com/bsdjhb/openssl.git +# cd openssl (or not, I can't remember if you end up with another openssl dir?) +# git checkout ktls_rx +# mkdir obj +# cd obj +# ../config --prefix=/usr/ktls --openssldir=/usr/ktls enable-ktls +# make +# make install +- This installs the patched openssl3 under /usr/ktls. I only use this + stuff for linking the daemons and use the regular openssl1.1.1 otherwise. + +Now, you need to patch the include files in /usr/ktls/include/openssl. +(clang doesn't like the DEFINE_OR_DECLARE_STACK_OF(XX) before the typedef for XX.) +# cd /usr/ktls/include/openssl +# patch -p0 < /usr/nfs-over-tls/openssl3.patch + +And now you should be able to build/install the utilities. +(You'll get warnings about SSL_CTX_load_verify_locations() deprecated. + Thats ok for now.) +# cd /usr/nfs-over-tls/usr.sbin/rpctlssd +# make SRCTOP=/usr/nfs-over-tls +# cp rpctlssd /usr/sbin +# cp rpctlssd.8.gz /usr/share/man/man8 +# cd ../rpctlscd +# make SRCTOP=/usr/nfs-over-tls +# cp rpctlscd /usr/sbin +# cp rpctlscd.8.gz /usr/share/man/man8 +# cd .. +# ln -s /usr/src/usr.sbin/mount mount +# cd mountd +# make SRCTOP=/usr/nfs-over-tls +# cp mountd /usr/sbin +# cp exports.5.gz /usr/share/man/man5 + +Almost done. Here's a few more things you need to do: +# cd /etc +- edit sysctl.conf and add these two lines +kern.ipc.tls.enable=1 +kern.ipc.mb_use_ext_pgs=1 +- edit rc.conf and add this line (if you already have a line for ldconfig_paths, + add /usr/ktls/lib to the line) +ldconfig_paths="/usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/ktls/lib" +Then reboot the system with nkernel. + +Every time you boot and are going to be doing testing, login as root and type +# kldload ktls_ocf +You should now be finally ready to configure and run a TLS mount. +(If you have problems or I forgot any steps, feel free to email me + at rmacklem@uoguelph.ca.) + +First, a bit of background. NFS-over-TLS uses the KERNEL_TLS and +will only work on architectures that support a direct map, such +as amd64 (not i386). +Then daemons must be running on the NFS server(s) and NFS client(s) +for NFS-over-TLS to work. rpctlssd(8) for the server(s) and rpctlscd(8) +for the client(s). +Then you will have to create x509 certificate for at least the NFS +server(s) and, optionally, some or all of the NFS clients. + +There are many ways to create signed certificates for TLS, but here +is the simple method I've used for a site local CA, using only the +openssl command that is in FreeBSD. +I am far from an expert in this area, so there are almost certainly +better ways to do this. (Some have mentioned easyrsa in the openVPN +software package, but I haven't tried it.) + +I have only applied this one simple patch to /etc/ssl/openssl.cnf: +--- openssl.cnf.sav 2020-03-31 07:41:12.545812000 -0700 ++++ openssl.cnf 2020-04-01 10:55:31.682616000 -0700 +@@ -66,7 +66,7 @@ name_opt = ca_default # Subject Name options + cert_opt = ca_default # Certificate field options + + # Extension copying option: use with caution. +-# copy_extensions = copy ++copy_extensions = copy + + # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs + # so this is commented out by default to leave a V1 CRL. + +You will probably want to make additional changes to /etc/ssl/openssl.cnf. + +First, I create a simple subtree of files under a top level directory +(I use /root) with the following commands: +# mkdir demoCA +# cd demoCA +# cat /dev/null > index.txt +# echo 01 > serial +# echo 01 > crlnumber +# mkdir private +# chmod 700 private +# mkdir newcerts + +Now, I use the following commands to create certifcates and a +Certificate Revocation List (CRL). +(In the directory above "demoCA".) +1 - Create the Certificate Authority (CA) root key. +# openssl genrsa -out demoCA/private/cakey.pem + +2 - Create a CA root certificate. +# openssl req -new -x509 -key demoCA/private/cakey.pem -out demoCA/cacert.pem +--> There will be several questions to answer. As far as I know, you can + put just about anything in these fields. + +Now, you should be ready to create/sign certificates for the NFS server/client(s). +3 - Create a key for the certificate. +# openssl genrsa -out key.pem +(If this certificate is for a client laptop, you might want to use the "-aes256" + option, so the key.pem file is encrypted using a passphrase. + This implies that the passphrase will need to be entered when the + rpctlscd(8) daemon is started on the client, but that the key cannot + be used without the passphrase, if it is compromised.) + +4 - Create a Certificate Signing Request (CSR). +# openssl req -new -key key.pem -addext "subjectAltName=" -out req.pem +--> Similar to #2, you will need to answer the questions. + The CN must be something different than used for #2. + Typically it would be the FQDN of the machine, if it has one. + The other fields can be the same as #2 and some (maybe all?) + must be the same. + is the messy part. The subjectAltName should have all + the FQDN name(s) for the machine, if it has any. + These are specified via DNS: and are separated by ','s if + there are multiple entries. + If you are using the "-u" option on the server and this + certificate is for a client where you wish all RPCs to be done + as a specific "user" on the NFS server, you also need to put an + entry in of the form otherName:1.X;UTF8:. + +Here are a few examples: +For a server with a FQDN of nfsv4-server.uoguelph.ca: +# openssl req -new -key key.pem -addext "subjectAltName=DNS:nfsv4-server.uoguelph.ca" -out req.pem + +For a server with two network interfaces, where one reverse DNS resolves +to nfsv4-server.uoguelph.ca and the other to nfsv4-server2.uoguelph.ca +# openssl req -new -key key.pem -addext "subjectAltName=DNS:nfsv4-server.uoguelph.ca,DNS:nfsv4-server2.uoguelph.ca" -out req.pem + +For a client where you wish all RPCs to be done as the user rmacklem +on the above server: +# openssl req -new -key key.pem -addext "subjectAltName=otherName:1.2.3.4.6.9;UTF8:rmacklem@uoguelph.ca" -out req.pem + +For a client similar to the above, but has a FQDN of nfsv4-client.uoguelph.ca: +# openssl req -new -key key.pem -addext "subjectAltName=DNS:nfsv4-client.uoguelph.ca,othername:1.2.3.4.6.9;UTF8:rmacklem@uoguelph.ca" -out req.pem + +If you want to look at the CSR: +# openssl req -in req.pem -noout -text + +5 - Sign the CSR with the CA. +# openssl ca -in req.pem -out cert.pem +(A copy of cert.pem will be kept in demoCA/newcerts with the + name .pem.) + +You can now copy key.pem and cert.pem to the directory +/etc/rpctlssd on the server(s) or /etc/rpctlscd on the client(s). + +If you want to look at any certificate, you can use the command... +# openssl x509 -in cert.pem -noout -text + +For a minimal setup where you do not want certificates in the client(s), +all you need to do is #3-5 for each of the NFS servers. +If you want certificates for clients, just repeat #3-5 for each of them. + +If you have created certificates for any of your NFS client(s), you +probably want to create a Certificate Revocation List (CRL) as well. +The initial file will not have any revocations in it, but can be +provided to either/both of the rpctlssd(8) and rpctlscd(8) daemons, +then it can be reloaded by posting a SIGHUP to the daemon(s) when +updated. This avoids restarting the daemon(s), which is not a good +thing to do while there are NFS-over-TLS mount(s) to the NFS server. + +6 - Create a new/empty CRL. +# openssl ca -gencrl -out demoCA/crl.pem + +You can look at this CRL with the command... +# openssl crl -in crl.pem -noout -text + +If you need to revoke a certifcate at some time. +# openssl ca -revoke where is the + certificate being revoked +Then do #6 again and copy it to the NFS server(s) and NFS client(s) +that are using it. (More on where it is used further down it this doc.) + +Now, when will you want certificates for some/all of the client(s)? + +One case might be laptops. They do not have fixed IP addresses/DNS names, +so exports(5) cannot be applied to them easily. You might find that the +client having a certificate signed by your CA is sufficient assurance +of its identity to allow it to do an NFS mount from any IP address. +For this case, you can also set the otherName field of the +subjectAltName to "user@dns_domain" so that all RPCs will be +performed on the server as "user", if you specify the "-u" +command line option for the rpctlssd(8) daemon on the NFS server. +(If you do not want this feature simply do not set the otherName + field of subjectAltName or do not set "-u" on the rpctlssd(8) daemon.) + +Another case might be where you do not trust the client to use the +correct IP address when mounting the NFS server, although the client +has a well known fixed IP address/DNS name. +For this case, you can also set the DNS field of subjectAltName to +the FQDN of the client and enable the server to check this via +the "-h" option. +The FQDN in the client's certificate may have a wildcard "*" in +it, depending on what command line options are specified for +the server's rpctlssd. + +For client(s) where you find controlling mount access via the +client's IP address using the exports(5) file is sufficient +and you are not using the "-u" command line option on the +server's rpctlssd, the client does not need to have a certificate. +You can still allow/require the client to use TLS so that +the RPC traffic is encrypted on the wire. + +Once you have key(s) and certificate(s) in the /etc/rpctlssd directory +on the NFS server(s) and in the /etc/rpctlscd directory on +the NFS client(s), you need to set the appropriate command +line option(s) for the daemons. + +The man pages for rpctlscd(8) and rpctlssd(8) cover the command line +options, but here are a few examples. + +For an NFS server: +- An NFS server where no clients have certificates. +# rpctlssd + +- An NFS server where some/all clients have certificates and you wish + to verify them against your site local CA created above. +# rpctlssd -m -l /root/demoCA/cacert.pem -r /root/demoCA/crl.pem + +- An NFS server where all clients have certificates and + FQDN names that are in the certificates (in either the subjectAltName DNS + field or subjectName CN field) and you want to check the client's IP + address reverse DNS maps to the FQDN. The FQDN in the client's + certificate cannot have a wildcard "*" in it. +# rpctlssd -m -h -l /root/demoCA/cacert.pem -r /root/demoCA/crl.pem + +- Similar to the above, but the FQDN in the client's + certificate may have a wildcard "*" in it, which will only match a + single component of the client's reverse DNS name. + For example, an FQDN set to "*.uoguelph.ca" will match "laptop21.uoguelph.ca", + but not "laptop3.cis.uoguelph.ca". +# rpctlssd -m -h -w -l /root/demoCA/cacert.pem -r /root/demoCA/crl.pem + +- Similar to the above, except that a wildcard "*" in the FQDN can match + multiple fields. For example, if the FQDN is set to "*.uoguelph.ca", + it would match "laptop3.cis.uoguelph.ca" as well as "laptop21.uoguelph.ca". +# rpctlssd -m -h -W -l /root/demoCA/cacert.pem -r /root/demoCA/crl.pem + +- An NFS server where some client(s) have certificates with the otherName field + of the subjectAltName set to "user@dns_domain" and you want those clients + to use the for "user" in the password database for + all RPCs on the connection, ignoring the credentials in the RPC header. +# rpctlssd -m -u -l /root/demoCA/cacert.pem -r /root/demoCA/crl.pem + +For an NFS client: +- An NFS client without a certificate. +# rpctlscd + +- An NFS client with a certificate and key in /etc/rpctlscd on the client + created by the site local CA above that the server can use for verification. +# rpctlscd -m + +- An NFS client which wants to verify the NFS server's certificate. + (This requires that the cacert.pem and crl.pem + be copied onto the client from the CA site.) + The FQDN in the server's certificate must match the reverse DNS name for + the server's IP address and there cannot be a wildcard in the FQDN. +# rpctlscd -l -r + +- An NFS client that has a certificate and key in /etc/rpctlscd and + also wishes to verify the NFS server's certificate as above. +# rpctlscd -m -l -r + +If you use either the "-m" and/or "-v" options, you probably want to +modify your /etc/syslog.conf so that "LOG_INFO | LOG_DAEMON" goes somewhere. +For "-m" it will log failed certificate verifications there and for "-v" +it will log a lot of other stuff, as well. From owner-svn-src-projects@freebsd.org Wed May 6 01:47:46 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5B7C42DBE90 for ; Wed, 6 May 2020 01:47:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49GzxZ1XFsz3Kg0; Wed, 6 May 2020 01:47:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2AFDA1F908; Wed, 6 May 2020 01:47:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0461ljRO066727; Wed, 6 May 2020 01:47:45 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0461lj07066726; Wed, 6 May 2020 01:47:45 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005060147.0461lj07066726@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 6 May 2020 01:47:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360680 - projects/nfs-over-tls/usr.sbin/rpctlssd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/usr.sbin/rpctlssd X-SVN-Commit-Revision: 360680 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2020 01:47:46 -0000 Author: rmacklem Date: Wed May 6 01:47:45 2020 New Revision: 360680 URL: https://svnweb.freebsd.org/changeset/base/360680 Log: Do not do SSL_shutdown() for now, since it can cause crashes. Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Wed May 6 01:08:19 2020 (r360679) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Wed May 6 01:47:45 2020 (r360680) @@ -431,7 +431,6 @@ rpctlssd_disconnect_1_svc(struct rpctlssd_disconnect_a rpctlssd_verbose_out("rpctlssd_disconnect fd=%d closed\n", slp->s); LIST_REMOVE(slp, next); - SSL_shutdown(slp->ssl); SSL_free(slp->ssl); /* * For RPC-over-TLS, this upcall is expected From owner-svn-src-projects@freebsd.org Wed May 6 22:52:20 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D67382E1CFC for ; Wed, 6 May 2020 22:52:20 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49HX0h44H7z4KMc; Wed, 6 May 2020 22:52:20 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 86ED46FA7; Wed, 6 May 2020 22:52:20 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 046MqKvW060279; Wed, 6 May 2020 22:52:20 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 046MqKrX060278; Wed, 6 May 2020 22:52:20 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005062252.046MqKrX060278@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 6 May 2020 22:52:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360719 - projects/nfs-over-tls/sys/amd64/conf X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/amd64/conf X-SVN-Commit-Revision: 360719 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2020 22:52:20 -0000 Author: rmacklem Date: Wed May 6 22:52:20 2020 New Revision: 360719 URL: https://svnweb.freebsd.org/changeset/base/360719 Log: Get rid of MYTEST and modify GENERIC, so that builds are easier for testers. Deleted: projects/nfs-over-tls/sys/amd64/conf/MYTEST Modified: projects/nfs-over-tls/sys/amd64/conf/GENERIC Modified: projects/nfs-over-tls/sys/amd64/conf/GENERIC ============================================================================== --- projects/nfs-over-tls/sys/amd64/conf/GENERIC Wed May 6 22:49:21 2020 (r360718) +++ projects/nfs-over-tls/sys/amd64/conf/GENERIC Wed May 6 22:52:20 2020 (r360719) @@ -84,6 +84,7 @@ options INCLUDE_CONFIG_FILE # Include this file in ke options RACCT # Resource accounting framework options RACCT_DEFAULT_TO_DISABLED # Set kern.racct.enable=0 by default options RCTL # Resource limits +options KERN_TLS # Debugging support. Always need this: options KDB # Enable kernel debugger support. @@ -365,8 +366,8 @@ device hyperv # HyperV drivers # Xen HVM Guest Optimizations # NOTE: XENHVM depends on xenpci. They must be added or removed together. -options XENHVM # Xen HVM kernel infrastructure -device xenpci # Xen HVM Hypervisor services driver +# options XENHVM # Xen HVM kernel infrastructure +# device xenpci # Xen HVM Hypervisor services driver # Netmap provides direct access to TX/RX rings on supported NICs device netmap # netmap(4) support From owner-svn-src-projects@freebsd.org Wed May 6 23:03:40 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E2D692E211D for ; Wed, 6 May 2020 23:03:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49HXFm5JYYz4Kth; Wed, 6 May 2020 23:03:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B0E517199; Wed, 6 May 2020 23:03:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 046N3ec5066414; Wed, 6 May 2020 23:03:40 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 046N3e84066413; Wed, 6 May 2020 23:03:40 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005062303.046N3e84066413@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 6 May 2020 23:03:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360720 - projects/nfs-over-tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls X-SVN-Commit-Revision: 360720 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2020 23:03:40 -0000 Author: rmacklem Date: Wed May 6 23:03:40 2020 New Revision: 360720 URL: https://svnweb.freebsd.org/changeset/base/360720 Log: Update nfs-over-tls-setup.txt. Modified: projects/nfs-over-tls/nfs-over-tls-setup.txt Modified: projects/nfs-over-tls/nfs-over-tls-setup.txt ============================================================================== --- projects/nfs-over-tls/nfs-over-tls-setup.txt Wed May 6 22:52:20 2020 (r360719) +++ projects/nfs-over-tls/nfs-over-tls-setup.txt Wed May 6 23:03:40 2020 (r360720) @@ -40,15 +40,8 @@ I do it. # cd /usr/src # mv sys sys.old # ln -s /usr/nfs-over-tls/sys sys -# cd /usr/src/sys/amd64/conf -# config MYTEST -# cd ../compile/MYTEST -# make LD=ld.lld SRCTOP=/usr/src cleandepend -# make LD=ld.lld SRCTOP=/usr/src depend -# make LD=ld.lld SRCTOP=/usr/src -# make LD=ld.lld KERNEL=nkernel install -- This puts the new kernel in /boot/nkernel. It can be booted by selecting - '3' at the boot menu and then typing "boot nkernel". +# make buildkernel +# make installkernel Fix the include files. Here's my "cheat" way of doing this. # cd /usr/include From owner-svn-src-projects@freebsd.org Sat May 9 01:18:43 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 50EAB2D27E1 for ; Sat, 9 May 2020 01:18:43 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Jq8g1Wqlz3yrV; Sat, 9 May 2020 01:18:43 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2FA1DB8D7; Sat, 9 May 2020 01:18:43 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0491IhSd099558; Sat, 9 May 2020 01:18:43 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0491Ih2W099557; Sat, 9 May 2020 01:18:43 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005090118.0491Ih2W099557@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 9 May 2020 01:18:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360830 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 360830 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2020 01:18:43 -0000 Author: rmacklem Date: Sat May 9 01:18:42 2020 New Revision: 360830 URL: https://svnweb.freebsd.org/changeset/base/360830 Log: Rewrite clnt_vc_soupcall() so that it does soreceive() for an entire TLS record (or all data available for TCP without TLS), similar to what soreceive() in svc_vc.c does. This should avoid problems if/when an NFS RPC message record mark straddles TLS records and reduces the number of soreceive() calls done by the client. It also makes adding code to handle the TLS control records simpler. This is next on my todo list. Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Fri May 8 23:00:02 2020 (r360829) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Sat May 9 01:18:42 2020 (r360830) @@ -274,6 +274,7 @@ clnt_vc_create( soupcall_set(ct->ct_socket, SO_RCV, clnt_vc_soupcall, ct); SOCKBUF_UNLOCK(&ct->ct_socket->so_rcv); + ct->ct_raw = NULL; ct->ct_record = NULL; ct->ct_record_resid = 0; TAILQ_INIT(&ct->ct_pending); @@ -921,9 +922,9 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai struct ct_request *cr; int error, rcvflag, foundreq; uint32_t xid_plus_direction[2], header; - bool_t do_read; SVCXPRT *xprt; struct cf_conn *cd; + u_int rawlen; CTASSERT(sizeof(xid_plus_direction) == 2 * sizeof(uint32_t)); @@ -935,117 +936,125 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai } mtx_unlock(&ct->ct_lock); + /* + * If another thread is already here, it must be in + * soreceive(), so just return. + * ct_upcallrefs is protected by the SOCKBUF_LOCK(), + * which is held in this function, except when + * soreceive() is called. + */ + if (ct->ct_upcallrefs > 0) + return (SU_OK); ct->ct_upcallrefs++; - uio.uio_td = curthread; - do { - /* - * If ct_record_resid is zero, we are waiting for a - * record mark. - */ - if (ct->ct_record_resid == 0) { + /* + * Read as much as possible off the socket and link it + * onto ct_raw. + */ + for (;;) { + uio.uio_resid = 1000000000; + uio.uio_td = curthread; + m = NULL; + rcvflag = MSG_DONTWAIT | MSG_SOCALLBCK; + SOCKBUF_UNLOCK(&so->so_rcv); + error = soreceive(so, NULL, &uio, &m, NULL, &rcvflag); + SOCKBUF_LOCK(&so->so_rcv); + + if (error == EWOULDBLOCK) { /* - * Make sure there is either a whole record - * mark in the buffer or there is some other - * error condition + * We must re-test for readability after + * taking the lock to protect us in the case + * where a new packet arrives on the socket + * after our call to soreceive fails with + * EWOULDBLOCK. */ - do_read = FALSE; - if (sbavail(&so->so_rcv) >= sizeof(uint32_t) - || (so->so_rcv.sb_state & SBS_CANTRCVMORE) - || so->so_error) - do_read = TRUE; - - if (!do_read) + if (!soreadable(so)) break; - - SOCKBUF_UNLOCK(&so->so_rcv); - uio.uio_resid = sizeof(uint32_t); - m = NULL; - rcvflag = MSG_DONTWAIT | MSG_SOCALLBCK; - error = soreceive(so, NULL, &uio, &m, NULL, &rcvflag); - SOCKBUF_LOCK(&so->so_rcv); - - if (error == EWOULDBLOCK) - break; - + continue; + } + if (error == 0 && m == NULL) { /* - * If there was an error, wake up all pending - * requests. + * We must have got EOF trying + * to read from the stream. */ - if (error || uio.uio_resid > 0) { - wakeup_all: - mtx_lock(&ct->ct_lock); - if (!error) { - /* - * We must have got EOF trying - * to read from the stream. - */ - error = ECONNRESET; - } - ct->ct_error.re_status = RPC_CANTRECV; - ct->ct_error.re_errno = error; - TAILQ_FOREACH(cr, &ct->ct_pending, cr_link) { - cr->cr_error = error; - wakeup(cr); - } - mtx_unlock(&ct->ct_lock); - break; + error = ECONNRESET; + } + if (error != 0) { + mtx_lock(&ct->ct_lock); + ct->ct_error.re_status = RPC_CANTRECV; + ct->ct_error.re_errno = error; + TAILQ_FOREACH(cr, &ct->ct_pending, cr_link) { + cr->cr_error = error; + wakeup(cr); } - m_copydata(m, 0, sizeof(uint32_t), (char *)&header); + mtx_unlock(&ct->ct_lock); + goto out; + } + + if (ct->ct_raw != NULL) + m_last(ct->ct_raw)->m_next = m; + else + ct->ct_raw = m; + } + rawlen = m_length(ct->ct_raw, NULL); + + /* Now, process as much of ct_raw as possible. */ + for (;;) { + /* + * If ct_record_resid is zero, we are waiting for a + * record mark. + */ + if (ct->ct_record_resid == 0) { + if (rawlen < sizeof(uint32_t)) + break; + m_copydata(ct->ct_raw, 0, sizeof(uint32_t), + (char *)&header); header = ntohl(header); ct->ct_record = NULL; ct->ct_record_resid = header & 0x7fffffff; ct->ct_record_eor = ((header & 0x80000000) != 0); - m_freem(m); +if (ct->ct_record_resid < 20 || ct->ct_record_resid > 70000 || !ct->ct_record_eor) +printf("EEK!! recres=%zd eor=%d\n", ct->ct_record_resid, ct->ct_record_eor); + m_adj(ct->ct_raw, sizeof(uint32_t)); + rawlen -= sizeof(uint32_t); } else { /* - * Wait until the socket has the whole record - * buffered. + * Move as much of the record as possible to + * ct_record. */ - do_read = FALSE; - if (sbavail(&so->so_rcv) >= ct->ct_record_resid - || (so->so_rcv.sb_state & SBS_CANTRCVMORE) - || so->so_error) - do_read = TRUE; - - if (!do_read) + if (rawlen == 0) break; - - /* - * We have the record mark. Read as much as - * the socket has buffered up to the end of - * this record. - */ - SOCKBUF_UNLOCK(&so->so_rcv); - uio.uio_resid = ct->ct_record_resid; - m = NULL; - rcvflag = MSG_DONTWAIT | MSG_SOCALLBCK; - error = soreceive(so, NULL, &uio, &m, NULL, &rcvflag); - SOCKBUF_LOCK(&so->so_rcv); - - if (error == EWOULDBLOCK) + if (rawlen <= ct->ct_record_resid) { + if (ct->ct_record != NULL) + m_last(ct->ct_record)->m_next = + ct->ct_raw; + else + ct->ct_record = ct->ct_raw; + ct->ct_raw = NULL; + ct->ct_record_resid -= rawlen; + rawlen = 0; + } else { + m = m_split(ct->ct_raw, ct->ct_record_resid, + M_NOWAIT); + if (m == NULL) + break; + if (ct->ct_record != NULL) + m_last(ct->ct_record)->m_next = + ct->ct_raw; + else + ct->ct_record = ct->ct_raw; + rawlen -= ct->ct_record_resid; + ct->ct_record_resid = 0; + ct->ct_raw = m; + } + if (ct->ct_record_resid > 0) break; - if (error || uio.uio_resid == ct->ct_record_resid) - goto wakeup_all; - /* - * If we have part of the record already, - * chain this bit onto the end. - */ - if (ct->ct_record) - m_last(ct->ct_record)->m_next = m; - else - ct->ct_record = m; - - ct->ct_record_resid = uio.uio_resid; - - /* * If we have the entire record, see if we can * match it to a request. */ - if (ct->ct_record_resid == 0 - && ct->ct_record_eor) { + if (ct->ct_record_eor) { /* * The XID is in the first uint32_t of * the reply and the message direction @@ -1067,11 +1076,9 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai ntohl(xid_plus_direction[1]); /* Check message direction. */ if (xid_plus_direction[1] == CALL) { -printf("Got backchannel callback\n"); /* This is a backchannel request. */ mtx_lock(&ct->ct_lock); xprt = ct->ct_backchannelxprt; -printf("backxprt=%p\n", xprt); if (xprt == NULL) { mtx_unlock(&ct->ct_lock); /* Just throw it away. */ @@ -1135,7 +1142,8 @@ printf("backxprt=%p\n", xprt); } } } - } while (m); + } +out: ct->ct_upcallrefs--; if (ct->ct_upcallrefs < 0) panic("rpcvc upcall refcnt"); From owner-svn-src-projects@freebsd.org Sat May 9 01:34:44 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8BEF42D3348 for ; Sat, 9 May 2020 01:34:44 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49JqW839WHz40vJ; Sat, 9 May 2020 01:34:44 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6834ABE89; Sat, 9 May 2020 01:34:44 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0491YiBO011493; Sat, 9 May 2020 01:34:44 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0491Yi0D011492; Sat, 9 May 2020 01:34:44 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005090134.0491Yi0D011492@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 9 May 2020 01:34:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360831 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 360831 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2020 01:34:44 -0000 Author: rmacklem Date: Sat May 9 01:34:43 2020 New Revision: 360831 URL: https://svnweb.freebsd.org/changeset/base/360831 Log: Update krpc.h as needed for the new clnt_vc_soupcall() just committed. Modified: projects/nfs-over-tls/sys/rpc/krpc.h Modified: projects/nfs-over-tls/sys/rpc/krpc.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/krpc.h Sat May 9 01:18:42 2020 (r360830) +++ projects/nfs-over-tls/sys/rpc/krpc.h Sat May 9 01:34:43 2020 (r360831) @@ -107,6 +107,7 @@ struct ct_data { uint64_t ct_sslusec; uint64_t ct_sslrefno; bool_t ct_dontrcv; /* TRUE to block receiving */ + struct mbuf *ct_raw; /* Raw mbufs recv'd */ }; struct cf_conn { /* kept in xprt->xp_p1 for actual connection */ From owner-svn-src-projects@freebsd.org Sat May 9 01:48:09 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 03BD22D3FD7 for ; Sat, 9 May 2020 01:48:09 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Jqpc5s8zz41kH; Sat, 9 May 2020 01:48:08 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C44F7C0D3; Sat, 9 May 2020 01:48:08 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0491m8Xr017611; Sat, 9 May 2020 01:48:08 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0491m88i017610; Sat, 9 May 2020 01:48:08 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005090148.0491m88i017610@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 9 May 2020 01:48:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360832 - in projects/nfs-over-tls/usr.sbin: rpctlscd rpctlssd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/usr.sbin: rpctlscd rpctlssd X-SVN-Commit-Revision: 360832 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2020 01:48:09 -0000 Author: rmacklem Date: Sat May 9 01:48:08 2020 New Revision: 360832 URL: https://svnweb.freebsd.org/changeset/base/360832 Log: Make both rpctlssd and rpctlscd load the ktls_ocf module, as required. Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Sat May 9 01:34:43 2020 (r360831) +++ projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Sat May 9 01:48:08 2020 (r360832) @@ -141,6 +141,12 @@ main(int argc, char **argv) warn("cannot open or create pidfile"); } + if (modfind("ktls_ocf") < 0) { + /* Not present in kernel, try loading it */ + if (kldload("ktls_ocf") < 0 || modfind("ktls_ocf") < 0) + errx(1, "Cannot load ktls_ocf"); + } + /* Get the time when this daemon is started. */ gettimeofday(&tm, &tz); rpctls_ssl_sec = tm.tv_sec; Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Sat May 9 01:34:43 2020 (r360831) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Sat May 9 01:48:08 2020 (r360832) @@ -153,6 +153,12 @@ main(int argc, char **argv) warn("cannot open or create pidfile"); } + if (modfind("ktls_ocf") < 0) { + /* Not present in kernel, try loading it */ + if (kldload("ktls_ocf") < 0 || modfind("ktls_ocf") < 0) + errx(1, "Cannot load ktls_ocf"); + } + /* Get the time when this daemon is started. */ gettimeofday(&tm, &tz); rpctls_ssl_sec = tm.tv_sec; From owner-svn-src-projects@freebsd.org Sat May 9 23:56:13 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F3B132D2465 for ; Sat, 9 May 2020 23:56:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49KPH16F9Mz4T6P; Sat, 9 May 2020 23:56:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D0E242406C; Sat, 9 May 2020 23:56:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 049NuDsY046729; Sat, 9 May 2020 23:56:13 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 049NuDVe046728; Sat, 9 May 2020 23:56:13 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005092356.049NuDVe046728@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 9 May 2020 23:56:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360858 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 360858 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2020 23:56:14 -0000 Author: rmacklem Date: Sat May 9 23:56:13 2020 New Revision: 360858 URL: https://svnweb.freebsd.org/changeset/base/360858 Log: Add some very basic handling of TLS_GET_RECORD control mbufs for the client side krpc. For now, it just tosses away any non-application data records. This will probably need to change, but not until TLS1.3, I think? I also added code that shuts down the connection for a couple of cases that should rarely, if ever, happen: - If m_split(..,M_NOWAIT) returns NULL, the system is basically out of mbufs and I figured shutting down the TCP connection and then trying to create a new one is about as good as any other recovery. At least it doesn't end up hung waiting for more record data, which may never come. - If the RPC over TCP record mark is bogus, the connection is obviously no longer usable. This should never happen, but I have seen it recently during testing of the nfs-over-tls and I am not sure if it might still happen. Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Sat May 9 21:34:50 2020 (r360857) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Sat May 9 23:56:13 2020 (r360858) @@ -894,6 +894,8 @@ clnt_vc_destroy(CLIENT *cl) soclose(so); } } + m_freem(ct->ct_record); + m_freem(ct->ct_raw); mem_free(ct, sizeof(struct ct_data)); if (cl->cl_netid && cl->cl_netid[0]) mem_free(cl->cl_netid, strlen(cl->cl_netid) +1); @@ -925,6 +927,8 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai SVCXPRT *xprt; struct cf_conn *cd; u_int rawlen; + struct cmsghdr *cmsg; + struct tls_get_record tgr; CTASSERT(sizeof(xid_plus_direction) == 2 * sizeof(uint32_t)); @@ -944,7 +948,9 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai * soreceive() is called. */ if (ct->ct_upcallrefs > 0) +{ printf("soup another\n"); return (SU_OK); +} ct->ct_upcallrefs++; /* @@ -954,10 +960,10 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai for (;;) { uio.uio_resid = 1000000000; uio.uio_td = curthread; - m = NULL; + m2 = m = NULL; rcvflag = MSG_DONTWAIT | MSG_SOCALLBCK; SOCKBUF_UNLOCK(&so->so_rcv); - error = soreceive(so, NULL, &uio, &m, NULL, &rcvflag); + error = soreceive(so, NULL, &uio, &m, &m2, &rcvflag); SOCKBUF_LOCK(&so->so_rcv); if (error == EWOULDBLOCK) { @@ -980,6 +986,7 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai error = ECONNRESET; } if (error != 0) { + wakeup_all: mtx_lock(&ct->ct_lock); ct->ct_error.re_status = RPC_CANTRECV; ct->ct_error.re_errno = error; @@ -991,6 +998,30 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai goto out; } + /* Process any record header(s). */ + if (m2 != NULL) { +if (m2->m_next != NULL) printf("EEK! list of controls\n"); + cmsg = mtod(m2, struct cmsghdr *); + if (cmsg->cmsg_type == TLS_GET_RECORD && + cmsg->cmsg_len == CMSG_LEN(sizeof(tgr))) { + memcpy(&tgr, CMSG_DATA(cmsg), sizeof(tgr)); + /* + * For now, just toss non-application + * data records. + * In the future, there may need to be + * an upcall done to the daemon, but + * it cannot be done here. + */ + if (tgr.tls_type != TLS_RLTYPE_APP) { +printf("Got weird type=%d\n", tgr.tls_type); + m_freem(m); + m_free(m2); + continue; + } + } + m_free(m2); + } + if (ct->ct_raw != NULL) m_last(ct->ct_raw)->m_next = m; else @@ -1013,8 +1044,22 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai ct->ct_record = NULL; ct->ct_record_resid = header & 0x7fffffff; ct->ct_record_eor = ((header & 0x80000000) != 0); -if (ct->ct_record_resid < 20 || ct->ct_record_resid > 70000 || !ct->ct_record_eor) -printf("EEK!! recres=%zd eor=%d\n", ct->ct_record_resid, ct->ct_record_eor); + if (ct->ct_record_resid < 20 || + ct->ct_record_resid > 150000 || + !ct->ct_record_eor) { + printf("clnt_vc_soupcall: bogus record " + "mark recres=%zd eor=%d\n", + ct->ct_record_resid, ct->ct_record_eor); + /* + * Connection is messed up. All we can + * do now is shut it down and let + * clnt_reconnect_XXX establish a new + * connection. + * This should never happen, but?? + */ + error = ECONNRESET; + goto wakeup_all; + } m_adj(ct->ct_raw, sizeof(uint32_t)); rawlen -= sizeof(uint32_t); } else { @@ -1036,8 +1081,22 @@ printf("EEK!! recres=%zd eor=%d\n", ct->ct_record_resi } else { m = m_split(ct->ct_raw, ct->ct_record_resid, M_NOWAIT); - if (m == NULL) - break; + if (m == NULL) { +printf("soup m_split returned NULL\n"); + /* + * What to do now? + * The system is out of mbufs. + * I think it best to close this + * connection and allow + * clnt_reconnect_XXX() to try + * and establish a new one. + * If we just return and there are + * no more data received, the + * connection will be hung. + */ + error = ECONNRESET; + goto wakeup_all; + } if (ct->ct_record != NULL) m_last(ct->ct_record)->m_next = ct->ct_raw; @@ -1076,9 +1135,11 @@ printf("EEK!! recres=%zd eor=%d\n", ct->ct_record_resi ntohl(xid_plus_direction[1]); /* Check message direction. */ if (xid_plus_direction[1] == CALL) { +printf("Got backchannel callback\n"); /* This is a backchannel request. */ mtx_lock(&ct->ct_lock); xprt = ct->ct_backchannelxprt; +printf("backxprt=%p\n", xprt); if (xprt == NULL) { mtx_unlock(&ct->ct_lock); /* Just throw it away. */