From owner-svn-src-projects@freebsd.org Sun May 10 00:17:39 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A7C162D2EC1 for ; Sun, 10 May 2020 00:17:39 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49KPll3yx1z4Tq5; Sun, 10 May 2020 00:17:39 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8351D24427; Sun, 10 May 2020 00:17:39 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04A0HdY5058864; Sun, 10 May 2020 00:17:39 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04A0Hd7I058863; Sun, 10 May 2020 00:17:39 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005100017.04A0Hd7I058863@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 10 May 2020 00:17:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360859 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 360859 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2020 00:17:39 -0000 Author: rmacklem Date: Sun May 10 00:17:39 2020 New Revision: 360859 URL: https://svnweb.freebsd.org/changeset/base/360859 Log: Add some very basic handling of TLS_GET_RECORD control mbufs. For now, it just throws away any that are non-application data. In the future, this will need to change, but not until TLS1.3, I think? Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc_vc.c Sat May 9 23:56:13 2020 (r360858) +++ projects/nfs-over-tls/sys/rpc/svc_vc.c Sun May 10 00:17:39 2020 (r360859) @@ -698,11 +698,13 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, { struct cf_conn *cd = (struct cf_conn *) xprt->xp_p1; struct uio uio; - struct mbuf *m; + struct mbuf *m, *ctrl; struct socket* so = xprt->xp_socket; XDR xdrs; int error, rcvflag; uint32_t xid_plus_direction[3], junk; + struct cmsghdr *cmsg; + struct tls_get_record tgr; /* * Serialise access to the socket and our own record parsing @@ -782,6 +784,7 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, * If receiving is disabled so that a TLS handshake can be * done by the rpctlssd daemon, return FALSE here. */ +tryagain: if (xprt->xp_dontrcv) { sx_xunlock(&xprt->xp_lock); return (FALSE); @@ -797,9 +800,9 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, */ uio.uio_resid = 1000000000; uio.uio_td = curthread; - m = NULL; + ctrl = m = NULL; rcvflag = MSG_DONTWAIT; - error = soreceive(so, NULL, &uio, &m, NULL, &rcvflag); + error = soreceive(so, NULL, &uio, &m, &ctrl, &rcvflag); if (error == EWOULDBLOCK) { /* @@ -838,6 +841,29 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, cd->strm_stat = XPRT_DIED; sx_xunlock(&xprt->xp_lock); return (FALSE); + } + + /* Process any record header(s). */ + if (ctrl != NULL) { +if (ctrl->m_next != NULL) printf("EEK! svc list of controls\n"); + cmsg = mtod(ctrl, struct cmsghdr *); + if (cmsg->cmsg_type == TLS_GET_RECORD && + cmsg->cmsg_len == CMSG_LEN(sizeof(tgr))) { + memcpy(&tgr, CMSG_DATA(cmsg), sizeof(tgr)); + /* + * For now, just toss non-application + * data records. + * In the future, there may need to be + * an upcall done to the daemon. + */ + if (tgr.tls_type != TLS_RLTYPE_APP) { +printf("Got weird type=%d\n", tgr.tls_type); + m_freem(m); + m_free(ctrl); + goto tryagain; + } + } + m_free(ctrl); } if (cd->mpending) From owner-svn-src-projects@freebsd.org Sun May 10 00:24:41 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 424002D3057 for ; Sun, 10 May 2020 00:24:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49KPvs11qcz4V9K; Sun, 10 May 2020 00:24:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1DCE424612; Sun, 10 May 2020 00:24:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04A0OfEe065001; Sun, 10 May 2020 00:24:41 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04A0Oe5D065000; Sun, 10 May 2020 00:24:41 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005100024.04A0Oe5D065000@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 10 May 2020 00:24:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360860 - projects/nfs-over-tls/sys/kern X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/kern X-SVN-Commit-Revision: 360860 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2020 00:24:41 -0000 Author: rmacklem Date: Sun May 10 00:24:40 2020 New Revision: 360860 URL: https://svnweb.freebsd.org/changeset/base/360860 Log: Modify ktls_decrypt() slightly, so that a small iov is allocated on the stack. As such, it only needs to malloc larger ones. Since most NFS RPC messages only need an iov of size 1 or 2, it seemed to be overkill to malloc/free for those cases. This is not needed for correct behaviour and it will be up to jhb@ whether or not this patch gets applied to his KTLS RX patch. Modified: projects/nfs-over-tls/sys/kern/uipc_ktls.c Modified: projects/nfs-over-tls/sys/kern/uipc_ktls.c ============================================================================== --- projects/nfs-over-tls/sys/kern/uipc_ktls.c Sun May 10 00:17:39 2020 (r360859) +++ projects/nfs-over-tls/sys/kern/uipc_ktls.c Sun May 10 00:24:40 2020 (r360860) @@ -1665,6 +1665,7 @@ m_segments(struct mbuf *m, int skip) return (count); } +#define KTLS_SMALLIOVEC 2 static void ktls_decrypt(struct socket *so) { @@ -1672,7 +1673,7 @@ ktls_decrypt(struct socket *so) struct ktls_session *tls; struct sockbuf *sb; struct tls_record_layer *hdr; - struct iovec *iov; + struct iovec *iov, iv[KTLS_SMALLIOVEC]; struct tls_get_record tgr; struct mbuf *control, *data, *m; uint64_t seqno; @@ -1687,8 +1688,8 @@ ktls_decrypt(struct socket *so) tls = sb->sb_tls_info; MPASS(tls != NULL); - iov = NULL; - iov_cap = 0; + iov = iv; + iov_cap = KTLS_SMALLIOVEC; for (;;) { /* Is there enough queued for a TLS header? */ if (sb->sb_tlscc < tls->params.tls_hlen) @@ -1746,7 +1747,8 @@ ktls_decrypt(struct socket *so) */ iov_count = m_segments(data, tls->params.tls_hlen); if (iov_count > iov_cap) { - free(iov, M_KTLS); + if (iov_cap > KTLS_SMALLIOVEC) + free(iov, M_KTLS); iov = malloc(sizeof(*iov) * iov_count, M_KTLS, M_WAITOK); iov_cap = iov_count; @@ -1865,7 +1867,8 @@ ktls_decrypt(struct socket *so) sorwakeup_locked(so); deref: - free(iov, M_KTLS); + if (iov_cap > KTLS_SMALLIOVEC) + free(iov, M_KTLS); SOCKBUF_UNLOCK_ASSERT(sb); CURVNET_SET(so->so_vnet); From owner-svn-src-projects@freebsd.org Sun May 10 00:28:44 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1AAF22D310F for ; Sun, 10 May 2020 00:28:44 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49KQ0W6r4Hz4VJ3; Sun, 10 May 2020 00:28:43 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E5EC82461B; Sun, 10 May 2020 00:28:43 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04A0ShHr065228; Sun, 10 May 2020 00:28:43 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04A0Sh6W065227; Sun, 10 May 2020 00:28:43 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005100028.04A0Sh6W065227@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 10 May 2020 00:28:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360861 - projects/nfs-over-tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls X-SVN-Commit-Revision: 360861 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.32 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2020 00:28:44 -0000 Author: rmacklem Date: Sun May 10 00:28:43 2020 New Revision: 360861 URL: https://svnweb.freebsd.org/changeset/base/360861 Log: Update the nfs-over-tls-setup.txt document slightly. I think this project is now ready for third party testing, although the setup is rather awkward for now. As pieces of the code make it into head, this should get simpler. However, I think the document covers the basics of getting it going. Thanks go to jhb@ for his KERN_TLS work and for providing the patch for ktls rx and patched openssl3 needed to make it work. Modified: projects/nfs-over-tls/nfs-over-tls-setup.txt Modified: projects/nfs-over-tls/nfs-over-tls-setup.txt ============================================================================== --- projects/nfs-over-tls/nfs-over-tls-setup.txt Sun May 10 00:24:40 2020 (r360860) +++ projects/nfs-over-tls/nfs-over-tls-setup.txt Sun May 10 00:28:43 2020 (r360861) @@ -106,10 +106,8 @@ kern.ipc.mb_use_ext_pgs=1 - edit rc.conf and add this line (if you already have a line for ldconfig_paths, add /usr/ktls/lib to the line) ldconfig_paths="/usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/ktls/lib" -Then reboot the system with nkernel. +Then reboot the system. -Every time you boot and are going to be doing testing, login as root and type -# kldload ktls_ocf You should now be finally ready to configure and run a TLS mount. (If you have problems or I forgot any steps, feel free to email me at rmacklem@uoguelph.ca.) From owner-svn-src-projects@freebsd.org Sun May 10 20:44:45 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2BEA22F34DD for ; Sun, 10 May 2020 20:44:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Kwzd0lTpz4db1; Sun, 10 May 2020 20:44:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1275FB253; Sun, 10 May 2020 20:44:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04AKij7Q019223; Sun, 10 May 2020 20:44:45 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04AKiiC5019215; Sun, 10 May 2020 20:44:44 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005102044.04AKiiC5019215@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 10 May 2020 20:44:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360883 - in projects/nfs-over-tls/sys/fs: nfs nfsserver X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/fs: nfs nfsserver X-SVN-Commit-Revision: 360883 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2020 20:44:45 -0000 Author: rmacklem Date: Sun May 10 20:44:43 2020 New Revision: 360883 URL: https://svnweb.freebsd.org/changeset/base/360883 Log: Revert some of the code for dissecting the ext_pgs mbufs, since it is not currently needed. The code that dissects ext_pgs mbufs for NFS involves a lot of churn and removing will simplify merging to head. I will keep these patches around, in case they are needed someday. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c projects/nfs-over-tls/sys/fs/nfs/nfs_var.h projects/nfs-over-tls/sys/fs/nfs/nfsm_subs.h projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sun May 10 20:28:38 2020 (r360882) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sun May 10 20:44:43 2020 (r360883) @@ -237,11 +237,6 @@ static void nfsrv_removeuser(struct nfsusrgrp *usrp, i static int nfsrv_getrefstr(struct nfsrv_descript *, u_char **, u_char **, int *, int *); static void nfsrv_refstrbigenough(int, u_char **, u_char **, int *); -static int nfsm_copyfrommbuf(struct nfsrv_descript *, char *, enum uio_seg, - int); -static int nfsm_copyfrommbuf_extpgs(struct nfsrv_descript *, char *, - enum uio_seg, int); -static struct mbuf *nfsm_splitatpgno(struct mbuf *, int, int); static struct { int op; @@ -641,11 +636,15 @@ nfscl_fillsattr(struct nfsrv_descript *nd, struct vatt int nfsm_mbufuio(struct nfsrv_descript *nd, struct uio *uiop, int siz) { - char *uiocp; + char *mbufcp, *uiocp; int xfer, left, len; + struct mbuf *mp; long uiosiz, rem; int error = 0; + mp = nd->nd_md; + mbufcp = nd->nd_dpos; + len = mtod(mp, caddr_t) + mp->m_len - mbufcp; rem = NFSM_RNDUP(siz) - siz; while (siz > 0) { if (uiop->uio_iovcnt <= 0 || uiop->uio_iov == NULL) { @@ -658,16 +657,35 @@ nfsm_mbufuio(struct nfsrv_descript *nd, struct uio *ui left = siz; uiosiz = left; while (left > 0) { - xfer = nfsm_copyfrommbuf(nd, uiocp, uiop->uio_segflg, - left); + while (len == 0) { + mp = mp->m_next; + if (mp == NULL) { + error = EBADRPC; + goto out; + } + mbufcp = mtod(mp, caddr_t); + len = mp->m_len; + KASSERT(len >= 0, + ("len %d, corrupted mbuf?", len)); + } + xfer = (left > len) ? len : left; +#ifdef notdef + /* Not Yet.. */ + if (uiop->uio_iov->iov_op != NULL) + (*(uiop->uio_iov->iov_op)) + (mbufcp, uiocp, xfer); + else +#endif + if (uiop->uio_segflg == UIO_SYSSPACE) + NFSBCOPY(mbufcp, uiocp, xfer); + else + copyout(mbufcp, uiocp, xfer); left -= xfer; + len -= xfer; + mbufcp += xfer; uiocp += xfer; uiop->uio_offset += xfer; uiop->uio_resid -= xfer; - if (left > 0 && !nfsm_shiftnext(nd, &len)) { - error = EBADRPC; - goto out; - } } if (uiop->uio_iov->iov_len <= siz) { uiop->uio_iovcnt--; @@ -679,8 +697,14 @@ nfsm_mbufuio(struct nfsrv_descript *nd, struct uio *ui } siz -= uiosiz; } - if (rem > 0) - error = nfsm_advance(nd, rem, -1); + nd->nd_dpos = mbufcp; + nd->nd_md = mp; + if (rem > 0) { + if (len < rem) + error = nfsm_advance(nd, rem, len); + else + nd->nd_dpos += rem; + } out: NFSEXITCODE2(error, nd); @@ -698,83 +722,58 @@ APPLESTATIC void * nfsm_dissct(struct nfsrv_descript *nd, int siz, int how) { struct mbuf *mp2; - struct mbuf_ext_pgs *pgs; int siz2, xfer; caddr_t p; int left; caddr_t retp; retp = NULL; - if ((nd->nd_md->m_flags & M_NOMAP) != 0) - left = nd->nd_dextpgsiz; - else - left = mtod(nd->nd_md, char *) + nd->nd_md->m_len - - nd->nd_dpos; + left = mtod(nd->nd_md, caddr_t) + nd->nd_md->m_len - nd->nd_dpos; while (left == 0) { - if ((nd->nd_md->m_flags & M_NOMAP) != 0 && - nd->nd_dextpg < - nd->nd_md->m_ext_pgs.npgs - 1) { - pgs = &nd->nd_md->m_ext_pgs; - nd->nd_dextpg++; - nd->nd_dpos = (char *)(void *) - PHYS_TO_DMAP(nd->nd_md->m_epg_pa[nd->nd_dextpg]); - left = nd->nd_dextpgsiz = mbuf_ext_pg_len(pgs, - nd->nd_dextpg, 0); - } else if (!nfsm_shiftnext(nd, &left)) - return (NULL); + nd->nd_md = nd->nd_md->m_next; + if (nd->nd_md == NULL) + return (retp); + left = nd->nd_md->m_len; + nd->nd_dpos = mtod(nd->nd_md, caddr_t); } if (left >= siz) { retp = nd->nd_dpos; nd->nd_dpos += siz; - if ((nd->nd_md->m_flags & M_NOMAP) != 0) - nd->nd_dextpgsiz -= siz; + } else if (nd->nd_md->m_next == NULL) { + return (retp); } else if (siz > ncl_mbuf_mhlen) { panic("nfs S too big"); } else { - /* Make sure an ext_pgs mbuf is at the last page. */ - if ((nd->nd_md->m_flags & M_NOMAP) != 0) { - if (nd->nd_dextpg < - nd->nd_md->m_ext_pgs.npgs - 1) { - mp2 = nfsm_splitatpgno(nd->nd_md, - nd->nd_dextpg, how); - if (mp2 == NULL) - return (NULL); - } - nd->nd_md->m_ext_pgs.last_pg_len -= left; - } - if (nd->nd_md->m_next == NULL) - return (NULL); - - /* Allocate a new mbuf for the "siz" bytes of data. */ MGET(mp2, MT_DATA, how); if (mp2 == NULL) return (NULL); - - /* - * Link the new mp2 mbuf into the list then copy left - * bytes from the mbuf before it and siz - left bytes - * from the mbuf after it. - */ mp2->m_next = nd->nd_md->m_next; nd->nd_md->m_next = mp2; nd->nd_md->m_len -= left; - retp = p = mtod(mp2, char *); - memcpy(p, nd->nd_dpos, left); /* Copy what was left */ + nd->nd_md = mp2; + retp = p = mtod(mp2, caddr_t); + NFSBCOPY(nd->nd_dpos, p, left); /* Copy what was left */ siz2 = siz - left; p += left; - mp2->m_len = siz; - nd->nd_md = mp2->m_next; + mp2 = mp2->m_next; /* Loop around copying up the siz2 bytes */ while (siz2 > 0) { - if (nd->nd_md == NULL) + if (mp2 == NULL) return (NULL); - nfsm_set(nd, 0, false); - xfer = nfsm_copyfrommbuf(nd, p, UIO_SYSSPACE, siz2); - p += xfer; - siz2 -= xfer; + xfer = (siz2 > mp2->m_len) ? mp2->m_len : siz2; + if (xfer > 0) { + NFSBCOPY(mtod(mp2, caddr_t), p, xfer); + mp2->m_data += xfer; + mp2->m_len -= xfer; + p += xfer; + siz2 -= xfer; + } if (siz2 > 0) - nd->nd_md = nd->nd_md->m_next; + mp2 = mp2->m_next; } + nd->nd_md->m_len = siz; + nd->nd_md = mp2; + nd->nd_dpos = mtod(mp2, caddr_t); } return (retp); } @@ -788,7 +787,7 @@ nfsm_dissct(struct nfsrv_descript *nd, int siz, int ho APPLESTATIC int nfsm_advance(struct nfsrv_descript *nd, int offs, int left) { - int error = 0, xfer; + int error = 0; if (offs == 0) goto out; @@ -805,39 +804,24 @@ nfsm_advance(struct nfsrv_descript *nd, int offs, int /* * If left == -1, calculate it here. */ - if (left == -1) { - if ((nd->nd_md->m_flags & M_NOMAP) != 0) - left = nd->nd_dextpgsiz; - else - left = mtod(nd->nd_md, char *) + - nd->nd_md->m_len - nd->nd_dpos; - } + if (left == -1) + left = mtod(nd->nd_md, caddr_t) + nd->nd_md->m_len - + nd->nd_dpos; /* * Loop around, advancing over the mbuf data. */ while (offs > left) { - if ((nd->nd_md->m_flags & M_NOMAP) != 0 && - nd->nd_dextpg < - nd->nd_md->m_ext_pgs.npgs - 1) { - xfer = nfsm_copyfrommbuf_extpgs(nd, NULL, - UIO_SYSSPACE, offs); - offs -= xfer; - } else - offs -= left; - left = 0; - if (offs > 0 && !nfsm_shiftnext(nd, &left)) { + offs -= left; + nd->nd_md = nd->nd_md->m_next; + if (nd->nd_md == NULL) { error = EBADRPC; goto out; } + left = nd->nd_md->m_len; + nd->nd_dpos = mtod(nd->nd_md, caddr_t); } - if (offs > 0) { - if ((nd->nd_md->m_flags & M_NOMAP) != 0) - nfsm_copyfrommbuf_extpgs(nd, NULL, - UIO_SYSSPACE, offs); - else - nd->nd_dpos += offs; - } + nd->nd_dpos += offs; out: NFSEXITCODE(error); @@ -2468,21 +2452,45 @@ nfsv4_wanted(struct nfsv4lock *lp) APPLESTATIC int nfsrv_mtostr(struct nfsrv_descript *nd, char *str, int siz) { - int rem, error = 0, xfer; + char *cp; + int xfer, len; + struct mbuf *mp; + int rem, error = 0; + mp = nd->nd_md; + cp = nd->nd_dpos; + len = mtod(mp, caddr_t) + mp->m_len - cp; rem = NFSM_RNDUP(siz) - siz; while (siz > 0) { - xfer = nfsm_copyfrommbuf(nd, str, UIO_SYSSPACE, siz); + if (len > siz) + xfer = siz; + else + xfer = len; + NFSBCOPY(cp, str, xfer); str += xfer; siz -= xfer; - if (siz > 0 && !nfsm_shiftnext(nd, &xfer)) { - error = EBADRPC; - goto out; + if (siz > 0) { + mp = mp->m_next; + if (mp == NULL) { + error = EBADRPC; + goto out; + } + cp = mtod(mp, caddr_t); + len = mp->m_len; + } else { + cp += xfer; + len -= xfer; } } *str = '\0'; - if (rem > 0) - error = nfsm_advance(nd, rem, -1); + nd->nd_dpos = cp; + nd->nd_md = mp; + if (rem > 0) { + if (len < rem) + error = nfsm_advance(nd, rem, len); + else + nd->nd_dpos += rem; + } out: NFSEXITCODE2(error, nd); @@ -4945,149 +4953,6 @@ nfsm_set(struct nfsrv_descript *nd, u_int offs, bool b } /* - * Copy up to "len" bytes from the mbuf into "cp" and adjust the - * mbuf accordingly. - * If cp == NULL, do not do the actual copy, but adjust the mbuf. - * Return the number of bytes actually copied. - * Adjust m_data and m_len so that a future calculation of what - * is left using mtod() will work correctly. - */ -static int -nfsm_copyfrommbuf(struct nfsrv_descript *nd, char *cp, enum uio_seg segflg, - int len) -{ - struct mbuf *m; - int xfer; - - m = nd->nd_md; - if ((m->m_flags & M_NOMAP) != 0) { - xfer = nfsm_copyfrommbuf_extpgs(nd, cp, segflg, len); - return (xfer); - } - xfer = mtod(m, char *) + m->m_len - nd->nd_dpos; - xfer = min(xfer, len); - if (xfer > 0) { - if (cp != NULL) { - if (segflg == UIO_SYSSPACE) - memcpy(cp, nd->nd_dpos, xfer); - else - copyout(nd->nd_dpos, cp, xfer); - } - nd->nd_dpos += xfer; - m->m_data += xfer; - m->m_len -= xfer; - } - return (xfer); -} - -/* - * Copy up to "len" bytes from the mbuf into "cp" and adjust the - * mbuf accordingly. - * If cp == NULL, do not do the actual copy, but adjust the mbuf. - * Return the number of bytes actually copied. - * Same as above, but for an ext_pgs mbuf. - */ -static int -nfsm_copyfrommbuf_extpgs(struct nfsrv_descript *nd, char *cp, - enum uio_seg segflg, int len) -{ - struct mbuf_ext_pgs *pgs; - int tlen, xfer; - - pgs = &nd->nd_md->m_ext_pgs; - tlen = 0; - /* Copy from the page(s) into cp. */ - do { - xfer = nd->nd_dextpgsiz; - xfer = min(xfer, len); - if (cp != NULL && xfer > 0) { - if (segflg == UIO_SYSSPACE) - memcpy(cp, nd->nd_dpos, xfer); - else - copyout(nd->nd_dpos, cp, xfer); - cp += xfer; - } - tlen += xfer; - len -= xfer; - nd->nd_dextpgsiz -= xfer; - nd->nd_dpos += xfer; - if (nd->nd_dextpgsiz == 0 && len > 0 && - nd->nd_dextpg < pgs->npgs - 1) { - nd->nd_dextpg++; - nd->nd_dpos = (char *)(void *) - PHYS_TO_DMAP(nd->nd_md->m_epg_pa[nd->nd_dextpg]); - nd->nd_dextpgsiz = mbuf_ext_pg_len(pgs, - nd->nd_dextpg, 0); - } - } while (len > 0 && nd->nd_dextpgsiz > 0); - return (tlen); -} - -/* - * Split an ext_pgs mbuf into two ext_pgs mbufs on a page boundary. - */ -static struct mbuf * -nfsm_splitatpgno(struct mbuf *mp, int pgno, int how) -{ - struct mbuf *m; - struct mbuf_ext_pgs *pgs, *pgs0; - int i, j, tlen; - - KASSERT((mp->m_flags & (M_EXT | M_NOMAP)) == - (M_EXT | M_NOMAP), ("nfsm_splitatpgno: mp not ext_pgs")); - pgs = &mp->m_ext_pgs; - KASSERT(pgno < pgs->npgs - 1, ("nfsm_splitatpgno:" - " at the last page")); - m = mb_alloc_ext_pgs(how, mb_free_mext_pgs); - if (m == NULL) - return (m); - pgs0 = &m->m_ext_pgs; - pgs0->flags |= MBUF_PEXT_FLAG_ANON; - - /* Move the pages beyond pgno to the new mbuf. */ - for (i = pgno + 1, j = 0; i < pgs->npgs; i++, j++) - m->m_epg_pa[j] = mp->m_epg_pa[i]; - pgs0->npgs = j; - pgs0->last_pg_len = pgs->last_pg_len; - pgs->npgs = pgno + 1; - pgs->last_pg_len = PAGE_SIZE; - if (pgno == 0) - pgs->last_pg_len -= pgs->first_pg_off; - - /* Now set m_len for both mbufs. */ - tlen = mbuf_ext_pg_len(pgs, 0, pgs->first_pg_off); - for (i = 1; i < pgs->npgs; i++) - tlen += mbuf_ext_pg_len(pgs, i, 0); - mp->m_len = tlen; - - /* The new mbuf has first_pg_off == 0. */ - tlen = 0; - for (i = 0; i < pgs0->npgs; i++) - tlen += mbuf_ext_pg_len(pgs0, i, 0); - m->m_len = tlen; - - /* Link the new mbuf after mp. */ - m->m_next = mp->m_next; - mp->m_next = m; - return (mp); -} - -/* - * Shift to the next mbuf in the list list and update the nd fields. - * Return true if successful, false otherwise. - */ -bool -nfsm_shiftnext(struct nfsrv_descript *nd, int *leftp) -{ - - nd->nd_md = nd->nd_md->m_next; - if (nd->nd_md == NULL) - return (false); - *leftp = nfsm_set(nd, 0, false); - return (true); -} - -/* * Grow a ext_pgs mbuf list. Either allocate another page or add * an mbuf to the list. */ @@ -5119,31 +4984,4 @@ nfsm_add_ext_pgs(struct mbuf *m, int maxextsiz, int *b mp = m; } return (mp); -} - -/* - * Calculate the data offset of m for dextpg and dextpgsiz. - */ -int -nfsm_extpgs_calc_offs(struct mbuf *m, int dextpg, int dextpgsiz) -{ - struct mbuf_ext_pgs *pgs; - int cnt, offs; - - offs = 0; - pgs = &m->m_ext_pgs; - for (cnt = 0; cnt < dextpg; cnt++) { - if (cnt == 0) - offs += mbuf_ext_pg_len(pgs, 0, - pgs->first_pg_off); - else - offs += mbuf_ext_pg_len(pgs, cnt, 0); - } - if (dextpg == 0) - cnt = mbuf_ext_pg_len(pgs, 0, - pgs->first_pg_off); - else - cnt = mbuf_ext_pg_len(pgs, dextpg, 0); - offs += cnt - dextpgsiz; - return (offs); } Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_var.h ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Sun May 10 20:28:38 2020 (r360882) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Sun May 10 20:44:43 2020 (r360883) @@ -362,8 +362,6 @@ void nfsv4_freeslot(struct nfsclsession *, int); struct ucred *nfsrv_getgrpscred(struct ucred *); struct nfsdevice *nfsv4_findmirror(struct nfsmount *); int nfsm_set(struct nfsrv_descript *, u_int, bool); -bool nfsm_shiftnext(struct nfsrv_descript *, int *); -int nfsm_extpgs_calc_offs(struct mbuf *, int, int); /* nfs_clcomsubs.c */ void nfsm_uiombuf(struct nfsrv_descript *, struct uio *, int); @@ -686,8 +684,8 @@ int nfsvno_readlink(vnode_t, struct ucred *, int, NFSP struct mbuf **, int *); int nfsvno_read(vnode_t, off_t, int, struct ucred *, int, NFSPROC_T *, struct mbuf **, struct mbuf **); -int nfsvno_write(vnode_t, off_t, int, int *, struct nfsrv_descript *, - NFSPROC_T *); +int nfsvno_write(vnode_t, off_t, int, int *, struct mbuf *, char *, + struct ucred *, NFSPROC_T *); int nfsvno_createsub(struct nfsrv_descript *, struct nameidata *, vnode_t *, struct nfsvattr *, int *, int32_t *, NFSDEV_T, struct nfsexstuff *); Modified: projects/nfs-over-tls/sys/fs/nfs/nfsm_subs.h ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfsm_subs.h Sun May 10 20:28:38 2020 (r360882) +++ projects/nfs-over-tls/sys/fs/nfs/nfsm_subs.h Sun May 10 20:44:43 2020 (r360883) @@ -57,11 +57,6 @@ * Replace most of the macro with an inline function, to minimize * the machine code. The inline functions in lower case can be called * directly, bypassing the macro. - * For ND_NOMAP, if there is not enough contiguous space left in - * the mbuf page, allocate a regular mbuf. The data in these regular - * mbufs will need to be copied into pages later, since the data must - * be filled pages. This should only happen after a write request or - * read reply has been filled into the mbuf list. */ static __inline void * nfsm_build(struct nfsrv_descript *nd, int siz) @@ -106,22 +101,12 @@ nfsm_dissect(struct nfsrv_descript *nd, int siz) int tt1; void *retp; - if ((nd->nd_md->m_flags & (M_EXT | M_NOMAP)) == - (M_EXT | M_NOMAP)) { - if (nd->nd_dextpgsiz >= siz) { - retp = (void *)nd->nd_dpos; - nd->nd_dpos += siz; - nd->nd_dextpgsiz -= siz; - } else - retp = nfsm_dissct(nd, siz, M_WAITOK); - } else { - tt1 = mtod(nd->nd_md, char *) + nd->nd_md->m_len - - nd->nd_dpos; - if (tt1 >= siz) { - retp = (void *)nd->nd_dpos; - nd->nd_dpos += siz; - } else - retp = nfsm_dissct(nd, siz, M_WAITOK); + tt1 = mtod(nd->nd_md, caddr_t) + nd->nd_md->m_len - nd->nd_dpos; + if (tt1 >= siz) { + retp = (void *)nd->nd_dpos; + nd->nd_dpos += siz; + } else { + retp = nfsm_dissct(nd, siz, M_WAITOK); } return (retp); } @@ -132,22 +117,12 @@ nfsm_dissect_nonblock(struct nfsrv_descript *nd, int s int tt1; void *retp; - if ((nd->nd_md->m_flags & (M_EXT | M_NOMAP)) == - (M_EXT | M_NOMAP)) { - if (nd->nd_dextpgsiz >= siz) { - retp = (void *)nd->nd_dpos; - nd->nd_dpos += siz; - nd->nd_dextpgsiz -= siz; - } else - retp = nfsm_dissct(nd, siz, M_NOWAIT); - } else { - tt1 = mtod(nd->nd_md, char *) + nd->nd_md->m_len - - nd->nd_dpos; - if (tt1 >= siz) { - retp = (void *)nd->nd_dpos; - nd->nd_dpos += siz; - } else - retp = nfsm_dissct(nd, siz, M_NOWAIT); + tt1 = mtod(nd->nd_md, caddr_t) + nd->nd_md->m_len - nd->nd_dpos; + if (tt1 >= siz) { + retp = (void *)nd->nd_dpos; + nd->nd_dpos += siz; + } else { + retp = nfsm_dissct(nd, siz, M_NOWAIT); } return (retp); } Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Sun May 10 20:28:38 2020 (r360882) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Sun May 10 20:44:43 2020 (r360883) @@ -123,14 +123,14 @@ static void nfsrv_pnfsremovesetup(struct vnode *, NFSP static void nfsrv_pnfsremove(struct vnode **, int, char *, fhandle_t *, NFSPROC_T *); static int nfsrv_proxyds(struct vnode *, off_t, int, struct ucred *, - struct thread *, int, struct mbuf **, struct nfsrv_descript *, + struct thread *, int, struct mbuf **, char *, struct mbuf **, struct nfsvattr *, struct acl *, off_t *, int, bool *); static int nfsrv_setextattr(struct vnode *, struct nfsvattr *, NFSPROC_T *); static int nfsrv_readdsrpc(fhandle_t *, off_t, int, struct ucred *, NFSPROC_T *, struct nfsmount *, struct mbuf **, struct mbuf **); static int nfsrv_writedsrpc(fhandle_t *, off_t, int, struct ucred *, NFSPROC_T *, struct vnode *, struct nfsmount **, int, struct mbuf **, - struct nfsrv_descript *, int *); + char *, int *); static int nfsrv_allocatedsrpc(fhandle_t *, off_t, off_t, struct ucred *, NFSPROC_T *, struct vnode *, struct nfsmount **, int, int *); static int nfsrv_setacldsrpc(fhandle_t *, struct ucred *, NFSPROC_T *, @@ -1122,7 +1122,7 @@ nfsrv_createiovecw_extpgs(int retlen, struct mbuf *m, */ int nfsvno_write(struct vnode *vp, off_t off, int retlen, int *stable, - struct nfsrv_descript *nd, struct thread *p) + struct mbuf *mp, char *cp, struct ucred *cred, struct thread *p) { struct iovec *iv; int cnt, ioflags, error; @@ -1133,25 +1133,19 @@ nfsvno_write(struct vnode *vp, off_t off, int retlen, * Attempt to write to a DS file. A return of ENOENT implies * there is no DS file to write. */ - error = nfsrv_proxyds(vp, off, retlen, nd->nd_cred, p, - NFSPROC_WRITEDS, &nd->nd_md, nd, NULL, NULL, NULL, - NULL, 0, NULL); + error = nfsrv_proxyds(vp, off, retlen, cred, p, NFSPROC_WRITEDS, + &mp, cp, NULL, NULL, NULL, NULL, 0, NULL); if (error != ENOENT) { *stable = NFSWRITE_FILESYNC; return (error); } + if (*stable == NFSWRITE_UNSTABLE) ioflags = IO_NODELOCKED; else ioflags = (IO_SYNC | IO_NODELOCKED); - if ((nd->nd_md->m_flags & M_NOMAP) != 0) - error = nfsrv_createiovecw_extpgs(retlen, nd->nd_md, - nd->nd_dpos, nd->nd_dextpg, nd->nd_dextpgsiz, - &iv, &cnt); - else - error = nfsrv_createiovecw(retlen, nd->nd_md, - nd->nd_dpos, &iv, &cnt); + error = nfsrv_createiovecw(retlen, mp, cp, &iv, &cnt); if (error != 0) return (error); uiop->uio_iov = iv; @@ -1165,7 +1159,7 @@ nfsvno_write(struct vnode *vp, off_t off, int retlen, ioflags |= nh->nh_seqcount << IO_SEQSHIFT; /* XXX KDM make this more systematic? */ nfsstatsv1.srvbytes[NFSV4OP_WRITE] += uiop->uio_resid; - error = VOP_WRITE(vp, uiop, ioflags, nd->nd_cred); + error = VOP_WRITE(vp, uiop, ioflags, cred); if (error == 0) nh->nh_nextoff = uiop->uio_offset; free(iv, M_TEMP); @@ -4670,7 +4664,7 @@ nfsrv_dssetacl(struct vnode *vp, struct acl *aclp, str static int nfsrv_proxyds(struct vnode *vp, off_t off, int cnt, struct ucred *cred, - struct thread *p, int ioproc, struct mbuf **mpp, struct nfsrv_descript *nd, + struct thread *p, int ioproc, struct mbuf **mpp, char *cp, struct mbuf **mpp2, struct nfsvattr *nap, struct acl *aclp, off_t *offp, int content, bool *eofp) { @@ -4802,7 +4796,7 @@ tryagain: } } else if (ioproc == NFSPROC_WRITEDS) error = nfsrv_writedsrpc(fh, off, cnt, cred, p, vp, - &nmp[0], mirrorcnt, mpp, nd, &failpos); + &nmp[0], mirrorcnt, mpp, cp, &failpos); else if (ioproc == NFSPROC_SETATTR) error = nfsrv_setattrdsrpc(fh, cred, p, vp, &nmp[0], mirrorcnt, nap, &failpos); @@ -5180,54 +5174,37 @@ nfsrv_readdsrpc(fhandle_t *fhp, off_t off, int len, st } /* - * Now, get rid of mbuf data that preceeds the - * current position. For a regular mbuf, adjust - * m_data, m_len and then find the end of the read - * data and trim off any mbuf(s) after that. - * For an ext_pgs mbuf, split it and free the first - * and third mbuf chains. + * Now, adjust first mbuf so that any XDR before the + * read data is skipped over. */ + trimlen = nd->nd_dpos - mtod(m, char *); + if (trimlen > 0) { + m->m_len -= trimlen; + NFSM_DATAP(m, trimlen); + } + + /* + * Truncate the mbuf chain at retlen bytes of data, + * plus XDR padding that brings the length up to a + * multiple of 4. + */ tlen = NFSM_RNDUP(retlen); - if ((m->m_flags & M_NOMAP) != 0) { - trimlen = nfsm_extpgs_calc_offs(m, - nd->nd_dextpg, nd->nd_dextpgsiz); - nd->nd_mrep = mb_splitatpos_ext(m, trimlen, - M_WAITOK); - m_freem(m); - m = mb_splitatpos_ext(nd->nd_mrep, tlen, - M_WAITOK); - m_freem(m); - m = m_last(nd->nd_mrep); - } else { - trimlen = nd->nd_dpos - mtod(m, char *); - if (trimlen > 0) { - m->m_len -= trimlen; - m->m_data += trimlen; + do { + if (m->m_len >= tlen) { + m->m_len = tlen; + tlen = 0; + m2 = m->m_next; + m->m_next = NULL; + m_freem(m2); + break; } - - /* - * Truncate the mbuf chain at retlen bytes of - * data, plus XDR padding that brings the - * length up to a multiple of 4. - */ - do { - if (m->m_len >= tlen) { - m->m_len = tlen; - tlen = 0; - m2 = m->m_next; - m->m_next = NULL; - m_freem(m2); - break; - } - tlen -= m->m_len; - m = m->m_next; - } while (m != NULL); - if (tlen > 0) { - printf("nfsrv_readdsrpc: busted mbuf " - "list\n"); - error = ENOENT; - goto nfsmout; - } + tlen -= m->m_len; + m = m->m_next; + } while (m != NULL); + if (tlen > 0) { + printf("nfsrv_readdsrpc: busted mbuf list\n"); + error = ENOENT; + goto nfsmout; } *mpp = nd->nd_mrep; *mpendp = m; @@ -5391,13 +5368,12 @@ start_writedsdorpc(void *arg, int pending) static int nfsrv_writedsrpc(fhandle_t *fhp, off_t off, int len, struct ucred *cred, NFSPROC_T *p, struct vnode *vp, struct nfsmount **nmpp, int mirrorcnt, - struct mbuf **mpp, struct nfsrv_descript *nd, int *failposp) + struct mbuf **mpp, char *cp, int *failposp) { struct nfsrvwritedsdorpc *drpc, *tdrpc = NULL; struct nfsvattr na; - struct mbuf *m, *m1, *m2; + struct mbuf *m; int error, i, offs, ret, timo; - bool gotnomap; NFSD_DEBUG(4, "in nfsrv_writedsrpc\n"); KASSERT(*mpp != NULL, ("nfsrv_writedsrpc: NULL mbuf chain")); @@ -5406,27 +5382,11 @@ nfsrv_writedsrpc(fhandle_t *fhp, off_t off, int len, s tdrpc = drpc = malloc(sizeof(*drpc) * (mirrorcnt - 1), M_TEMP, M_WAITOK); - NFSD_DEBUG(4, "nfsrv_writedsrpc: mcopy len=%d\n", len); + /* Calculate offset in mbuf chain that data starts. */ + offs = cp - mtod(*mpp, char *); + NFSD_DEBUG(4, "nfsrv_writedsrpc: mcopy offs=%d len=%d\n", offs, len); /* - * For M_NOMAP mbufs, the mbuf chain needs to be split into 3 chains - * so that m_copym() can be done with offs == 0 and M_COPYALL. - * *mpp - Everything that preceeds the data to be written. - * m1 - The data to be written. - * m2 - Everything that follows the data to be written. - */ - m1 = *mpp; - gotnomap = false; - if ((m1->m_flags & M_NOMAP) != 0) { - gotnomap = true; - offs = nfsm_extpgs_calc_offs(nd->nd_md, nd->nd_dextpg, - nd->nd_dextpgsiz); - m1 = mb_splitatpos_ext(m1, offs, M_WAITOK); - m2 = mb_splitatpos_ext(m1, NFSM_RNDUP(len), M_WAITOK); - } else - offs = nd->nd_dpos - mtod(m1, char *); - - /* * Do the write RPC for every DS, using a separate kernel process * for every DS except the last one. */ @@ -5441,11 +5401,7 @@ nfsrv_writedsrpc(fhandle_t *fhp, off_t off, int len, s tdrpc->p = p; tdrpc->inprog = 0; tdrpc->err = 0; - if (gotnomap) - tdrpc->m = m_copym(m1, 0, M_COPYALL, M_WAITOK); - else - tdrpc->m = m_copym(m1, offs, NFSM_RNDUP(len), - M_WAITOK); + tdrpc->m = m_copym(*mpp, offs, NFSM_RNDUP(len), M_WAITOK); ret = EIO; if (nfs_pnfsiothreads != 0) { ret = nfs_pnfsio(start_writedsdorpc, tdrpc); @@ -5463,10 +5419,7 @@ nfsrv_writedsrpc(fhandle_t *fhp, off_t off, int len, s nmpp++; fhp++; } - if (gotnomap) - m = m_copym(m1, 0, M_COPYALL, M_WAITOK); - else - m = m_copym(m1, offs, NFSM_RNDUP(len), M_WAITOK); + m = m_copym(*mpp, offs, NFSM_RNDUP(len), M_WAITOK); ret = nfsrv_writedsdorpc(*nmpp, fhp, off, len, &na, m, cred, p); if (nfsds_failerr(ret) && *failposp == -1 && mirrorcnt > 1) *failposp = mirrorcnt - 1; @@ -5487,14 +5440,6 @@ nfsrv_writedsrpc(fhandle_t *fhp, off_t off, int len, s *failposp = i; else if (error == 0 && tdrpc->err != 0) error = tdrpc->err; - } - - /* For gotnomap, chain the lists back to-gether. */ - if (gotnomap) { - m_last(*mpp)->m_next = m1; - m_last(m1)->m_next = m2; - nd->nd_md = m1; - nfsm_set(nd, 0, false); } free(drpc, M_TEMP); return (error); Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Sun May 10 20:28:38 2020 (r360882) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Sun May 10 20:44:43 2020 (r360883) @@ -1059,8 +1059,8 @@ nfsrvd_write(struct nfsrv_descript *nd, __unused int i * which is to return ok so long as there are no permission problems. */ if (retlen > 0) { - nd->nd_repstat = nfsvno_write(vp, off, retlen, &stable, nd, - p); + nd->nd_repstat = nfsvno_write(vp, off, retlen, &stable, + nd->nd_md, nd->nd_dpos, nd->nd_cred, p); error = nfsm_advance(nd, NFSM_RNDUP(retlen), -1); if (error) goto nfsmout; Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c Sun May 10 20:28:38 2020 (r360882) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c Sun May 10 20:44:43 2020 (r360883) @@ -1870,9 +1870,8 @@ APPLESTATIC int nfsrv_parsename(struct nfsrv_descript *nd, char *bufp, u_long *hashp, NFSPATHLEN_T *outlenp) { - struct mbuf_ext_pgs *pgs; - vm_page_t pg; char *fromcp, *tocp, val = '\0'; + struct mbuf *md; int i; int rem, len, error = 0, pubtype = 0, outlen = 0, percent = 0; char digit; @@ -1887,196 +1886,177 @@ nfsrv_parsename(struct nfsrv_descript *nd, char *bufp, * Otherwise, get the component name. */ if ((nd->nd_flag & ND_NFSV4) && nd->nd_procnum == NFSV4OP_LOOKUPP) { - *tocp++ = '.'; - hash += ((u_char)'.'); - *tocp++ = '.'; - hash += ((u_char)'.'); - outlen = 2; + *tocp++ = '.'; + hash += ((u_char)'.'); + *tocp++ = '.'; + hash += ((u_char)'.'); + outlen = 2; } else { - /* - * First, get the name length. - */ - NFSM_DISSECT(tl, u_int32_t *, NFSX_UNSIGNED); - len = fxdr_unsigned(int, *tl); - if (len > NFS_MAXNAMLEN) { - nd->nd_repstat = NFSERR_NAMETOL; + /* + * First, get the name length. + */ + NFSM_DISSECT(tl, u_int32_t *, NFSX_UNSIGNED); + len = fxdr_unsigned(int, *tl); + if (len > NFS_MAXNAMLEN) { + nd->nd_repstat = NFSERR_NAMETOL; + error = 0; + goto nfsmout; + } else if (len <= 0) { + nd->nd_repstat = NFSERR_INVAL; + error = 0; + goto nfsmout; + } + + /* + * Now, copy the component name into the buffer. + */ + fromcp = nd->nd_dpos; + md = nd->nd_md; + rem = mtod(md, caddr_t) + md->m_len - fromcp; + for (i = 0; i < len; i++) { + while (rem == 0) { + md = md->m_next; + if (md == NULL) { + error = EBADRPC; + goto nfsmout; + } + fromcp = mtod(md, caddr_t); + rem = md->m_len; + } + if (*fromcp == '\0') { + nd->nd_repstat = EACCES; error = 0; goto nfsmout; - } else if (len <= 0) { - nd->nd_repstat = NFSERR_INVAL; - error = 0; - goto nfsmout; } - /* - * Now, copy the component name into the buffer. + * For lookups on the public filehandle, do some special + * processing on the name. (The public file handle is the + * root of the public file system for this server.) */ - fromcp = nd->nd_dpos; - if ((nd->nd_md->m_flags & M_NOMAP) != 0) - rem = nd->nd_dextpgsiz; - else - rem = mtod(nd->nd_md, char *) + nd->nd_md->m_len - - fromcp; - for (i = 0; i < len; i++) { - while (rem == 0) { - if ((nd->nd_md->m_flags & M_NOMAP) != 0 && - nd->nd_dextpg < - nd->nd_md->m_ext_pgs.npgs - 1) { - pgs = &nd->nd_md->m_ext_pgs; - pg = PHYS_TO_VM_PAGE( - nd->nd_md->m_epg_pa[nd->nd_dextpg]); - vm_page_unwire_noq(pg); - vm_page_free(pg); - for (i = nd->nd_bextpg; - i < pgs->npgs - 1; i++) - nd->nd_md->m_epg_pa[i] = - nd->nd_md->m_epg_pa[i + 1]; - pgs->npgs--; - if (nd->nd_dextpg == 0) - pgs->first_pg_off = 0; - fromcp = nd->nd_dpos = (char *)(void *) - PHYS_TO_DMAP( - nd->nd_md->m_epg_pa[nd->nd_dextpg]); - rem = nd->nd_dextpgsiz = - mbuf_ext_pg_len(pgs, nd->nd_dextpg, - 0); + if (nd->nd_flag & ND_PUBLOOKUP) { + /* + * If the first char is ASCII, it is a canonical + * path, otherwise it is a native path. (RFC2054 + * doesn't actually state what it is if the first + * char isn't ASCII or 0x80, so I assume native.) + * pubtype == 1 -> native path + * pubtype == 2 -> canonical path + */ + if (i == 0) { + if (*fromcp & 0x80) { + /* + * Since RFC2054 doesn't indicate + * that a native path of just 0x80 + * isn't allowed, I'll replace the + * 0x80 with '/' instead of just + * throwing it away. + */ + *fromcp = '/'; + pubtype = 1; } else { - if (!nfsm_shiftnext(nd, &rem)) { - error = EBADRPC; - goto nfsmout; - } - fromcp = nd->nd_dpos; + pubtype = 2; } } - if (*fromcp == '\0') { + /* + * '/' only allowed in a native path + */ + if (*fromcp == '/' && pubtype != 1) { nd->nd_repstat = EACCES; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-projects@freebsd.org Sun May 10 21:37:41 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A230D2F4D0B for ; Sun, 10 May 2020 21:37:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Ky8j3r1dz3DBG; Sun, 10 May 2020 21:37:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7EC9BBEB9; Sun, 10 May 2020 21:37:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04ALbfSs050325; Sun, 10 May 2020 21:37:41 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04ALbelj050315; Sun, 10 May 2020 21:37:40 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005102137.04ALbelj050315@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 10 May 2020 21:37:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360884 - in projects/nfs-over-tls/sys/fs: nfs nfsclient nfsserver X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/fs: nfs nfsclient nfsserver X-SVN-Commit-Revision: 360884 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 May 2020 21:37:41 -0000 Author: rmacklem Date: Sun May 10 21:37:39 2020 New Revision: 360884 URL: https://svnweb.freebsd.org/changeset/base/360884 Log: Simplify nfsm_set() so that it does not do the dissect case. This will simplify integration with head. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonkrpc.c projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c projects/nfs-over-tls/sys/fs/nfs/nfs_var.h projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_fha_new.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonkrpc.c Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonkrpc.c Sun May 10 21:37:39 2020 (r360884) @@ -895,7 +895,7 @@ tryagain: */ newnfs_realign(&nd->nd_mrep, M_WAITOK); nd->nd_md = nd->nd_mrep; - nfsm_set(nd, ext.rc_mbufoffs, false); + nd->nd_dpos = mtod(nd->nd_md, char *); nd->nd_repstat = 0; if (nd->nd_procnum != NFSPROC_NULL && nd->nd_procnum != NFSV4PROC_CBNULL) { Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sun May 10 21:37:39 2020 (r360884) @@ -380,7 +380,7 @@ nfscl_reqstart(struct nfsrv_descript *nd, int procnum, mb = mb_alloc_ext_plus_pages(PAGE_SIZE, M_WAITOK, mb_free_mext_pgs); nd->nd_mreq = nd->nd_mb = mb; - nfsm_set(nd, 0, true); + nfsm_set(nd, 0); } else { if (nfs_bigrequest[procnum]) NFSMCLGET(mb, M_WAITOK); @@ -4873,81 +4873,49 @@ nfsv4_findmirror(struct nfsmount *nmp) * The build argument is true for build and false for dissect. */ int -nfsm_set(struct nfsrv_descript *nd, u_int offs, bool build) +nfsm_set(struct nfsrv_descript *nd, u_int offs) { struct mbuf *m; struct mbuf_ext_pgs *pgs; int rlen; - if (build) - m = nd->nd_mb; - else - m = nd->nd_md; + m = nd->nd_mb; if ((m->m_flags & M_NOMAP) != 0) { - if (build) { - pgs = &m->m_ext_pgs; - nd->nd_bextpg = 0; - while (offs > 0) { - if (nd->nd_bextpg == 0) - rlen = mbuf_ext_pg_len(pgs, 0, - pgs->first_pg_off); - else - rlen = mbuf_ext_pg_len(pgs, - nd->nd_bextpg, 0); - if (offs <= rlen) - break; - offs -= rlen; - nd->nd_bextpg++; - if (nd->nd_bextpg == pgs->npgs) { - printf("nfsm_set: build offs " - "out of range\n"); - nd->nd_bextpg--; - break; - } - } - nd->nd_bpos = (char *)(void *) - PHYS_TO_DMAP(m->m_epg_pa[nd->nd_bextpg]); + pgs = &m->m_ext_pgs; + nd->nd_bextpg = 0; + while (offs > 0) { if (nd->nd_bextpg == 0) - nd->nd_bpos += pgs->first_pg_off; - if (offs > 0) { - nd->nd_bpos += offs; - rlen = nd->nd_bextpgsiz = rlen - offs; - } else if (nd->nd_bextpg == 0) - rlen = nd->nd_bextpgsiz = PAGE_SIZE - - pgs->first_pg_off; + rlen = mbuf_ext_pg_len(pgs, 0, + pgs->first_pg_off); else - rlen = nd->nd_bextpgsiz = PAGE_SIZE; - } else { - pgs = &m->m_ext_pgs; - nd->nd_dextpg = 0; - do { - nd->nd_dpos = (char *)(void *) - PHYS_TO_DMAP(m->m_epg_pa[nd->nd_dextpg]); - if (nd->nd_dextpg == 0) { - nd->nd_dpos += pgs->first_pg_off; - rlen = nd->nd_dextpgsiz = - mbuf_ext_pg_len(pgs, 0, - pgs->first_pg_off); - } else - rlen = nd->nd_dextpgsiz = - mbuf_ext_pg_len(pgs, - nd->nd_dextpg, 0); - if (offs > rlen) { - nd->nd_dextpg++; - offs -= rlen; - } else if (offs > 0) { - nd->nd_dpos += offs; - nd->nd_dextpgsiz -= offs; - offs = 0; - } - } while (offs > 0); + rlen = mbuf_ext_pg_len(pgs, + nd->nd_bextpg, 0); + if (offs <= rlen) + break; + offs -= rlen; + nd->nd_bextpg++; + if (nd->nd_bextpg == pgs->npgs) { + printf("nfsm_set: build offs " + "out of range\n"); + nd->nd_bextpg--; + break; + } } - } else if (build) { + nd->nd_bpos = (char *)(void *) + PHYS_TO_DMAP(m->m_epg_pa[nd->nd_bextpg]); + if (nd->nd_bextpg == 0) + nd->nd_bpos += pgs->first_pg_off; + if (offs > 0) { + nd->nd_bpos += offs; + rlen = nd->nd_bextpgsiz = rlen - offs; + } else if (nd->nd_bextpg == 0) + rlen = nd->nd_bextpgsiz = PAGE_SIZE - + pgs->first_pg_off; + else + rlen = nd->nd_bextpgsiz = PAGE_SIZE; + } else { nd->nd_bpos = mtod(m, char *) + offs; rlen = m->m_len - offs; - } else { - nd->nd_dpos = mtod(m, char *); - rlen = m->m_len; } return (rlen); } Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_var.h ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Sun May 10 21:37:39 2020 (r360884) @@ -361,7 +361,7 @@ int nfsv4_sequencelookup(struct nfsmount *, struct nfs void nfsv4_freeslot(struct nfsclsession *, int); struct ucred *nfsrv_getgrpscred(struct ucred *); struct nfsdevice *nfsv4_findmirror(struct nfsmount *); -int nfsm_set(struct nfsrv_descript *, u_int, bool); +int nfsm_set(struct nfsrv_descript *, u_int); /* nfs_clcomsubs.c */ void nfsm_uiombuf(struct nfsrv_descript *, struct uio *, int); Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Sun May 10 21:37:39 2020 (r360884) @@ -95,7 +95,7 @@ printf("cbprogram proc=%d\n", rqst->rq_proc); newnfs_realign(&nd.nd_mrep, M_WAITOK); nd.nd_md = nd.nd_mrep; printf("cbreq nd_md=%p offs=%d\n", nd.nd_md, rqst->rq_xprt->xp_mbufoffs); - nfsm_set(&nd, rqst->rq_xprt->xp_mbufoffs, false); + nd.nd_dpos = mtod(nd.nd_md, char *); nd.nd_nam = svc_getrpccaller(rqst); nd.nd_nam2 = rqst->rq_addr; nd.nd_mreq = NULL; Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_fha_new.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_fha_new.c Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_fha_new.c Sun May 10 21:37:39 2020 (r360884) @@ -338,7 +338,7 @@ fha_extract_info(struct svc_req *req, struct fha_info if (error) goto out; nd->nd_md = req->rq_args; - nfsm_set(nd, req->rq_xprt->xp_mbufoffs, false); + nd->nd_dpos = mtod(nd->nd_md, char *); /* Grab the filehandle. */ error = fhanew_get_fh(&i->fh, v3, nd); Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c Sun May 10 21:37:39 2020 (r360884) @@ -164,7 +164,7 @@ nfssvc_program(struct svc_req *rqst, SVCXPRT *xprt) rqst->rq_args = NULL; newnfs_realign(&nd.nd_mrep, M_WAITOK); nd.nd_md = nd.nd_mrep; - nfsm_set(&nd, rqst->rq_xprt->xp_mbufoffs, false); + nd.nd_dpos = mtod(nd.nd_md, char *); nd.nd_nam = svc_getrpccaller(rqst); nd.nd_nam2 = rqst->rq_addr; nd.nd_mreq = NULL; Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Sun May 10 20:44:43 2020 (r360883) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Sun May 10 21:37:39 2020 (r360884) @@ -5285,7 +5285,7 @@ nfsrv_writedsdorpc(struct nfsmount *nmp, fhandle_t *fh while (m->m_next != NULL) m = m->m_next; nd->nd_mb = m; - nfsm_set(nd, m->m_len, true); + nfsm_set(nd, m->m_len); NFSD_DEBUG(4, "nfsrv_writedsdorpc: lastmb len=%d\n", m->m_len); /* Do a Getattr for the attributes that change upon writing. */ From owner-svn-src-projects@freebsd.org Mon May 11 20:22:07 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3668B2EA686 for ; Mon, 11 May 2020 20:22:07 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49LXR30kV3z42td; Mon, 11 May 2020 20:22:07 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-164.local (unknown [IPv6:2601:648:8203:2990:e804:1989:c6ed:be50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id AD1221710E; Mon, 11 May 2020 20:22:06 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc To: Rick Macklem , src-committers@freebsd.org, svn-src-projects@freebsd.org References: <202005100017.04A0Hd7I058863@repo.freebsd.org> From: John Baldwin Autocrypt: addr=jhb@FreeBSD.org; keydata= mQGiBETQ+XcRBADMFybiq69u+fJRy/0wzqTNS8jFfWaBTs5/OfcV7wWezVmf9sgwn8TW0Dk0 c9MBl0pz+H01dA2ZSGZ5fXlmFIsee1WEzqeJzpiwd/pejPgSzXB9ijbLHZ2/E0jhGBcVy5Yo /Tw5+U/+laeYKu2xb0XPvM0zMNls1ah5OnP9a6Ql6wCgupaoMySb7DXm2LHD1Z9jTsHcAQMD /1jzh2BoHriy/Q2s4KzzjVp/mQO5DSm2z14BvbQRcXU48oAosHA1u3Wrov6LfPY+0U1tG47X 1BGfnQH+rNAaH0livoSBQ0IPI/8WfIW7ub4qV6HYwWKVqkDkqwcpmGNDbz3gfaDht6nsie5Z pcuCcul4M9CW7Md6zzyvktjnbz61BADGDCopfZC4of0Z3Ka0u8Wik6UJOuqShBt1WcFS8ya1 oB4rc4tXfSHyMF63aPUBMxHR5DXeH+EO2edoSwViDMqWk1jTnYza51rbGY+pebLQOVOxAY7k do5Ordl3wklBPMVEPWoZ61SdbcjhHVwaC5zfiskcxj5wwXd2E9qYlBqRg7QeSm9obiBCYWxk d2luIDxqaGJARnJlZUJTRC5vcmc+iGAEExECACAFAkTQ+awCGwMGCwkIBwMCBBUCCAMEFgID AQIeAQIXgAAKCRBy3lIGd+N/BI6RAJ9S97fvbME+3hxzE3JUyUZ6vTewDACdE1stFuSfqMvM jomvZdYxIYyTUpC5Ag0ERND5ghAIAPwsO0B7BL+bz8sLlLoQktGxXwXQfS5cInvL17Dsgnr3 1AKa94j9EnXQyPEj7u0d+LmEe6CGEGDh1OcGFTMVrof2ZzkSy4+FkZwMKJpTiqeaShMh+Goj XlwIMDxyADYvBIg3eN5YdFKaPQpfgSqhT+7El7w+wSZZD8pPQuLAnie5iz9C8iKy4/cMSOrH YUK/tO+Nhw8Jjlw94Ik0T80iEhI2t+XBVjwdfjbq3HrJ0ehqdBwukyeJRYKmbn298KOFQVHO EVbHA4rF/37jzaMadK43FgJ0SAhPPF5l4l89z5oPu0b/+5e2inA3b8J3iGZxywjM+Csq1tqz hltEc7Q+E08AAwUIAL+15XH8bPbjNJdVyg2CMl10JNW2wWg2Q6qdljeaRqeR6zFus7EZTwtX sNzs5bP8y51PSUDJbeiy2RNCNKWFMndM22TZnk3GNG45nQd4OwYK0RZVrikalmJY5Q6m7Z16 4yrZgIXFdKj2t8F+x613/SJW1lIr9/bDp4U9tw0V1g3l2dFtD3p3ZrQ3hpoDtoK70ioIAjjH aIXIAcm3FGZFXy503DOA0KaTWwvOVdYCFLm3zWuSOmrX/GsEc7ovasOWwjPn878qVjbUKWwx Q4QkF4OhUV9zPtf9tDSAZ3x7QSwoKbCoRCZ/xbyTUPyQ1VvNy/mYrBcYlzHodsaqUDjHuW+I SQQYEQIACQUCRND5ggIbDAAKCRBy3lIGd+N/BCO8AJ9j1dWVQWxw/YdTbEyrRKOY8YZNwwCf afMAg8QvmOWnHx3wl8WslCaXaE8= Message-ID: <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> Date: Mon, 11 May 2020 13:22:05 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: <202005100017.04A0Hd7I058863@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2020 20:22:07 -0000 On 5/9/20 5:17 PM, Rick Macklem wrote: > Author: rmacklem > Date: Sun May 10 00:17:39 2020 > New Revision: 360859 > URL: https://svnweb.freebsd.org/changeset/base/360859 > > Log: > Add some very basic handling of TLS_GET_RECORD control mbufs. > > For now, it just throws away any that are non-application data. > In the future, this will need to change, but not until TLS1.3, I think? Ideally you'd keep an nfsd thread in userland that you could pass these records onto. One possible option is the thread just keeps calling SSL_read() but you do create a new flag on the socket buffer that causes soreceive() to only pass non-application data datagrams to userland reads() and have the in-kernel read requests block if they see a non-application data record as the next record until the user thread wakes up and reads it (or EAGAIN or whatever you need it to do). -- John Baldwin From owner-svn-src-projects@freebsd.org Tue May 12 00:36:38 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5CDD12DC381 for ; Tue, 12 May 2020 00:36:38 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Lf4k1r9vz4RwQ; Tue, 12 May 2020 00:36:38 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3A627272E1; Tue, 12 May 2020 00:36:38 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04C0ab8K061401; Tue, 12 May 2020 00:36:37 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04C0abmp061398; Tue, 12 May 2020 00:36:37 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005120036.04C0abmp061398@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 12 May 2020 00:36:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360945 - in projects/nfs-over-tls/sys/fs: nfs nfsserver X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/fs: nfs nfsserver X-SVN-Commit-Revision: 360945 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 00:36:38 -0000 Author: rmacklem Date: Tue May 12 00:36:37 2020 New Revision: 360945 URL: https://svnweb.freebsd.org/changeset/base/360945 Log: Get rid of some more of the dissect code for ext_pgs mbufs. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonport.c projects/nfs-over-tls/sys/fs/nfs/nfs_var.h projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonport.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonport.c Mon May 11 22:57:21 2020 (r360944) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonport.c Tue May 12 00:36:37 2020 (r360945) @@ -183,34 +183,8 @@ newnfs_realign(struct mbuf **pm, int how) { struct mbuf *m, *n; int off, space; - bool copyit; ++nfs_realign_test; - - /* - * For ext_pgs mbufs, just copy the entire chain if there is an - * alignment problem. - */ - copyit = false; - m = *pm; - while ((m->m_flags & M_NOMAP) != 0) { - if ((m->m_len & 0x3) != 0 || - (m->m_ext_pgs.first_pg_off & 0x3) != 0) { - copyit = true; - break; - } - m = m->m_next; - if (m == NULL) - return (0); - } - if (copyit) { - m = mb_unmapped_to_ext(*pm); - if (m == NULL) - return (ENOMEM); - *pm = m; - return (0); - } - while ((m = *pm) != NULL) { if ((m->m_len & 0x3) || (mtod(m, intptr_t) & 0x3)) { /* Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_var.h ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Mon May 11 22:57:21 2020 (r360944) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Tue May 12 00:36:37 2020 (r360945) @@ -750,7 +750,7 @@ int nfsvno_allocate(struct vnode *, off_t, off_t, stru int nfsvno_getxattr(struct vnode *, char *, uint32_t, struct ucred *, uint64_t, int, struct thread *, struct mbuf **, struct mbuf **, int *); int nfsvno_setxattr(struct vnode *, char *, int, struct mbuf *, char *, - int, int, struct ucred *, struct thread *); + struct ucred *, struct thread *); int nfsvno_rmxattr(struct nfsrv_descript *, struct vnode *, char *, struct ucred *, struct thread *); int nfsvno_listxattr(struct vnode *, uint64_t, struct ucred *, struct thread *, Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Mon May 11 22:57:21 2020 (r360944) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Tue May 12 00:36:37 2020 (r360945) @@ -114,8 +114,6 @@ static int nfsrv_createiovec_extpgs(int, int, struct m struct mbuf **, struct iovec **); static int nfsrv_createiovecw(int, struct mbuf *, char *, struct iovec **, int *); -static int nfsrv_createiovecw_extpgs(int, struct mbuf *, char *, int, - int, struct iovec **, int *); static void nfsrv_pnfscreate(struct vnode *, struct vattr *, struct ucred *, NFSPROC_T *); static void nfsrv_pnfsremovesetup(struct vnode *, NFSPROC_T *, struct vnode **, @@ -1032,92 +1030,6 @@ nfsrv_createiovecw(int retlen, struct mbuf *m, char *c } /* - * Create the iovec for the mbuf chain passed in as an argument. - * The "cp" argument is where the data starts within the first mbuf in - * the chain. It returns the iovec and the iovcnt. - * Same as above, but for ext_pgs mbufs. - */ -static int -nfsrv_createiovecw_extpgs(int retlen, struct mbuf *m, char *cp, int dextpg, - int dextpgsiz, struct iovec **ivpp, int *iovcntp) -{ - struct mbuf *mp; - struct mbuf_ext_pgs *pgs; - struct iovec *ivp; - int cnt, i, len, pgno; - - /* - * Loop through the mbuf chain, counting how many pages are - * part of this write oepration, so the iovec size is known. - */ - cnt = 0; - len = retlen; - mp = m; - pgs = &mp->m_ext_pgs; - i = dextpgsiz; - pgno = dextpg; - while (len > 0) { - if (i > 0) { - len -= i; - cnt++; - } - if (len > 0) { - if (pgno == pgs->npgs - 1) { - mp = mp->m_next; - if (mp == NULL) - return (EBADRPC); - pgno = 0; - pgs = &mp->m_ext_pgs; - } else - pgno++; - if (pgno == 0) - i = mbuf_ext_pg_len(pgs, 0, - pgs->first_pg_off); - else - i = mbuf_ext_pg_len(pgs, pgno, 0); - } - } - - /* Now, create the iovec. */ - mp = m; - *ivpp = ivp = malloc(cnt * sizeof (struct iovec), M_TEMP, - M_WAITOK); - *iovcntp = cnt; - len = retlen; - pgs = &mp->m_ext_pgs; - i = dextpgsiz; - pgno = dextpg; - while (len > 0) { - if (i > 0) { - i = min(i, len); - ivp->iov_base = cp; - ivp->iov_len = i; - ivp++; - len -= i; - } - if (len > 0) { - if (pgno == pgs->npgs - 1) { - mp = mp->m_next; - if (mp == NULL) - return (EBADRPC); - pgno = 0; - pgs = &mp->m_ext_pgs; - } else - pgno++; - cp = (char *)(void *) - PHYS_TO_DMAP(mp->m_epg_pa[pgno]); - if (pgno == 0) { - cp += pgs->first_pg_off; - i = mbuf_ext_pg_len(pgs, 0, - pgs->first_pg_off); - } else - i = mbuf_ext_pg_len(pgs, pgno, 0); - } - } - return (0); -} - -/* * Write vnode op from an mbuf list. */ int @@ -6435,8 +6347,7 @@ out: */ int nfsvno_setxattr(struct vnode *vp, char *name, int len, struct mbuf *m, - char *cp, int dextpg, int dextpgsiz, struct ucred *cred, - struct thread *p) + char *cp, struct ucred *cred, struct thread *p) { struct iovec *iv; struct uio uio, *uiop = &uio; @@ -6455,11 +6366,7 @@ nfsvno_setxattr(struct vnode *vp, char *name, int len, uiop->uio_td = p; uiop->uio_offset = 0; uiop->uio_resid = len; - if ((m->m_flags & M_NOMAP) != 0) - error = nfsrv_createiovecw_extpgs(len, m, cp, dextpg, - dextpgsiz, &iv, &cnt); - else - error = nfsrv_createiovecw(len, m, cp, &iv, &cnt); + error = nfsrv_createiovecw(len, m, cp, &iv, &cnt); uiop->uio_iov = iv; uiop->uio_iovcnt = cnt; if (error == 0) { Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Mon May 11 22:57:21 2020 (r360944) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Tue May 12 00:36:37 2020 (r360945) @@ -5708,8 +5708,7 @@ nfsrvd_setxattr(struct nfsrv_descript *nd, __unused in nd->nd_repstat = nfsvno_getattr(vp, &ova, nd, p, 1, &attrbits); if (nd->nd_repstat == 0) { nd->nd_repstat = nfsvno_setxattr(vp, name, len, nd->nd_md, - nd->nd_dpos, nd->nd_dextpg, nd->nd_dextpgsiz, nd->nd_cred, - p); + nd->nd_dpos, nd->nd_cred, p); if (nd->nd_repstat == ENXIO) nd->nd_repstat = NFSERR_XATTR2BIG; } From owner-svn-src-projects@freebsd.org Tue May 12 01:23:06 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4FA652DD95B for ; Tue, 12 May 2020 01:23:06 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Lg6L1Qjbz4VRx; Tue, 12 May 2020 01:23:06 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2BFDB27C77; Tue, 12 May 2020 01:23:06 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04C1N6Ex091372; Tue, 12 May 2020 01:23:06 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04C1N60l091371; Tue, 12 May 2020 01:23:06 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005120123.04C1N60l091371@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 12 May 2020 01:23:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360949 - projects/nfs-over-tls/sys/fs/nfsclient X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/fs/nfsclient X-SVN-Commit-Revision: 360949 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 01:23:06 -0000 Author: rmacklem Date: Tue May 12 01:23:05 2020 New Revision: 360949 URL: https://svnweb.freebsd.org/changeset/base/360949 Log: Revert one more change that was done to handle dissection of ext_pgs mbufs. I believe that all of the dissection of ext_pgs mbufs is now teased out of the NFS code. I still need to do the same for the rpc and xdr sources. Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Tue May 12 01:04:50 2020 (r360948) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Tue May 12 01:23:05 2020 (r360949) @@ -163,6 +163,7 @@ static int nfscl_dofflayoutio(vnode_t, struct uio *, i nfsv4stateid_t *, int, struct nfscldevinfo *, struct nfscllayout *, struct nfsclflayout *, uint64_t, uint64_t, int, int, struct mbuf *, struct nfsclwritedsdorpc *, struct ucred *, NFSPROC_T *); +static struct mbuf *nfsm_copym(struct mbuf *, int, int); static int nfsrpc_readds(vnode_t, struct uio *, nfsv4stateid_t *, int *, struct nfsclds *, uint64_t, int, struct nfsfh *, int, int, int, struct ucred *, NFSPROC_T *); @@ -6000,6 +6001,38 @@ nfscl_doiods(vnode_t vp, struct uio *uiop, int *iomode } /* + * Make a copy of the mbuf chain and add an mbuf for null padding, as required. + */ +static struct mbuf * +nfsm_copym(struct mbuf *m, int off, int xfer) +{ + struct mbuf *m2, *m3, *m4; + uint32_t *tl; + int rem; + + m2 = m_copym(m, off, xfer, M_WAITOK); + rem = NFSM_RNDUP(xfer) - xfer; + if (rem > 0) { + /* + * The zero padding to a multiple of 4 bytes is required by + * the XDR. So that the mbufs copied by reference aren't + * modified, add an mbuf with the zero'd bytes to the list. + * rem will be a maximum of 3, so one zero'd uint32_t is + * sufficient. + */ + m3 = m2; + while (m3->m_next != NULL) + m3 = m3->m_next; + NFSMGET(m4); + tl = mtod(m4, uint32_t *); + *tl = 0; + m4->m_len = rem; + m3->m_next = m4; + } + return (m2); +} + +/* * Find a file layout that will handle the first bytes of the requested * range and return the information from it needed to the I/O operation. */ @@ -6250,18 +6283,7 @@ nfscl_dofflayoutio(vnode_t vp, struct uio *uiop, int * NFSUNLOCKCLSTATE(); } } else { - /* - * Split off the first xfer bytes of the mbuf - * chain. - */ - m = mp; - if (xfer < len) { - if ((m->m_flags & M_NOMAP) != 0) - mp = mb_splitatpos_ext(m, xfer, - M_WAITOK); - else - mp = m_split(m, xfer, M_WAITOK); - } + m = nfsm_copym(mp, rel_off, xfer); NFSCL_DEBUG(4, "mcopy reloff=%d xfer=%jd\n", rel_off, (uintmax_t)xfer); /* @@ -6280,8 +6302,6 @@ nfscl_dofflayoutio(vnode_t vp, struct uio *uiop, int * xfer, fhp, m, dp->nfsdi_vers, dp->nfsdi_minorvers, tcred, p); NFSCL_DEBUG(4, "nfsio_writedsmir=%d\n", error); - if (xfer == len) - mp = NULL; if (error != 0 && error != EACCES && error != ESTALE) { NFSCL_DEBUG(4, @@ -6300,7 +6320,6 @@ nfscl_dofflayoutio(vnode_t vp, struct uio *uiop, int * if ((dp->nfsdi_flags & NFSDI_TIGHTCOUPLED) == 0) NFSFREECRED(tcred); } - m_freem(mp); /* In case errors occurred. */ NFSCL_DEBUG(4, "eo nfscl_dofflayoutio=%d\n", error); return (error); } From owner-svn-src-projects@freebsd.org Tue May 12 02:31:41 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DA2AB2E0053 for ; Tue, 12 May 2020 02:31:41 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670053.outbound.protection.outlook.com [40.107.67.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49LhdT4WKXz4ZSY; Tue, 12 May 2020 02:31:41 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Bvecpvbnl4oxB556f15alwkYDezKUnpnpFkD7KosCKVF+TOKEmfw+Co1a9oIZpkR2btcP5UQ8N4lrR9sYMLmSuQtdElRsHpYp/bWMDpEd0rV/udmXzypayyh7UIGyA1f2N1Tt51jaOrUGrMJ/wdZiq0CkSwrogkGs1ZictBJNABEfDfmFgTcqgDjHH7utawcET4f39/N4/2b0HGVgeggQtvJ2OiIwt8+BjGavd85k4GAYdBmvuC4WPY9ElanGdO3GUSXFZ0Amyiyrsfkq1IemV4kFyfLAgrTfYEYLPafbVnRDKo8Gb6ShGyAB33XdCSLLC16YXlItXCVdZs4fSpKTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J4o5drjSMmYGnapNYPV9BN+a37GtLeIRz//QU0h8MyU=; b=fMiOiJdrvT70Zw/w3Lt3r95x3qCb6pFYTihbm5VXoc/zo8EIpXpit50cJPJqYQBVeyDXK2gG/+b6a0jebTmav96DqBFy3Z26O4BaPGD5FskkRWmXIunXYOWbT95keewT3721nIe9Rnf2v0UDTMQUm2tSy9rhL9+quNfEj92iZic8QYa1QMOGbOQijdHYVEakOoTA1mNuSbSXdav/xzFtj4UG4LbV+8I122Yuc3jawErdW8l6ad/4pA4oJiBGIQA0y/HEle8eRUDi8eARmGE4j5N1uFrptHkRickCQh4mPA3uZcbe2Sy4unMHt2DtGhZZK2cNCin5oA8f6HSsGaWDQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM (52.132.86.26) by QB1PR01MB3505.CANPRD01.PROD.OUTLOOK.COM (52.132.85.213) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.35; Tue, 12 May 2020 02:31:39 +0000 Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2]) by QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 02:31:39 +0000 From: Rick Macklem To: John Baldwin , Rick Macklem , "src-committers@freebsd.org" , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Topic: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Index: AQHWJ9HZ1BQ89BXo/ki0EXCO1Y+2cKijtTOs Date: Tue, 12 May 2020 02:31:39 +0000 Message-ID: References: <202005100017.04A0Hd7I058863@repo.freebsd.org>, <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> In-Reply-To: <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6d2d51e2-de03-41d9-156a-08d7f61c99a0 x-ms-traffictypediagnostic: QB1PR01MB3505: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6Tt2lPRFuNpHv8sTzYdD1ApAOVKLxWoHonSYJd60zSK34khLysuVibpcvDEBPkrxE6pkVT4DSYlkoWIwgNXaMqPgx450LnGlusAPboguOBjulCAWOWnLYdI/gVZIvBjornL9o0Y2gTMww5yBPuIRaZmzugJQIZIcUcyma50XJ4v58gwMO8EK+jDEuxbBeoLo2+yln3gifIQtMvBWqOWRcalq6TI1nve04Bf/nSzNX58ebGKhbg+iSAQIRl80qf0lx6AD+p/QjrNnuLnlZS/J1Rnj3HaT1faj7We7ivl3dA/7h9jukWwtUjI2cxia2nw7Wcf3uLQL+iJ5mzLMr3cLML57EuV7qKfA7DrqjzTK6SQzoJKa+gV+XUMR12mJ6yCodMSp8WfvaTovQPVkjAm5eb2N6BKRHY0JQdijxwwo9pHfUuq/eusKE+5iL1ceN2HLliastiHffzKQ47vqNAwl96XhVgWtq5XcEtWJsfodcZ91RrNRBIaVjtXOKOqfTA6mt3NJlRcE4NhnQeeYWxYesO1OIQkVWCGMEY49XGBTkb1E60PVUnuWdR7QSut3Daccz/xINwajJaKdUD7Phf9TbECucxvHv+tpg65DBwcEqGc= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(366004)(346002)(396003)(39860400002)(376002)(136003)(33430700001)(8936002)(478600001)(5660300002)(186003)(33440700001)(110136005)(64756008)(66556008)(86362001)(52536014)(9686003)(66946007)(66446008)(6506007)(8676002)(76116006)(66476007)(55016002)(316002)(786003)(450100002)(33656002)(966005)(7696005)(71200400001)(2906002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: PPSp/n8AVnuvrRiO+8MerV8FYFkX2Lt8w59EMB1gQZCTfIjGnh87YNCbDs71BVCB52GfvZM+sdgRIDWWr/1sR99dTqqc6R70YvG6MfMQF3s2FZRitml8BZGjI0ks6oC9sxYNDTc2Oh0tyHA/KEcJnukyOWNuNIoOl3H3NCuN72XzE01A2ulMqNMYeKziE79+o1YM0d02kat9uCBheFXDhwNqctMytLCSY0Sn3ouj8FghP8CwajUJJ3PbHDXvHzTV8otdQro+HsMPpVsOhAv7cb8t0A1hN2qnuuFDOIoiJFTSmEhkgXH91A6AIEy0sJyhU1IwxlrAbCp7VLuzseW6A9HKjhukl/IAjF5alEaqWGCPCa+dnM/oMTXZdljNLRRACIeKUJr0UrzLopgG8aTJViVWmsgqGG/6u0l9pXPoCPAy6vqI+gMN6edig04d8CGxHdFzL4f/KY6erexyjGaD25W2yo5q0lqTD7jpRJjPeNnfHnE58xCx3CeuMSyUkHULRDVxcwTn1KfN+U4aLMFAmsw8cyePdY+Eky8bGSGyZN3V9cxKzU1jrZ1gAjUE5i3n x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 6d2d51e2-de03-41d9-156a-08d7f61c99a0 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 02:31:39.1827 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ypgr/X9i1t0OxoqXs3BMzuoHzcSzRhDvgGpe/0qW7XMcAwy0k1kcWiDgMXLsnkcQOF6J2b1CfREj5XKA4huLkQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB3505 X-Rspamd-Queue-Id: 49LhdT4WKXz4ZSY X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 02:31:41 -0000 John Baldwin wrote:=0A= >On 5/9/20 5:17 PM, Rick Macklem wrote:=0A= >> Author: rmacklem=0A= >> Date: Sun May 10 00:17:39 2020=0A= >> New Revision: 360859=0A= >> URL: https://svnweb.freebsd.org/changeset/base/360859=0A= >>=0A= >> Log:=0A= >> Add some very basic handling of TLS_GET_RECORD control mbufs.=0A= >>=0A= >> For now, it just throws away any that are non-application data.=0A= >> In the future, this will need to change, but not until TLS1.3, I think= ?=0A= >=0A= >Ideally you'd keep an nfsd thread in userland that you could pass=0A= >these records onto. One possible option is the thread just keeps=0A= >calling SSL_read() but you do create a new flag on the socket buffer=0A= >that causes soreceive() to only pass non-application data datagrams=0A= >to userland reads() and have the in-kernel read requests block if they=0A= >see a non-application data record as the next record until the user=0A= >thread wakes up and reads it (or EAGAIN or whatever you need it to=0A= >do).=0A= Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for= =0A= upcalls from the kernel and do the SSL stuff (mainly the handshake right no= w).=0A= (You can guess from the names which one is RPC client vs server.;-)=0A= I can easily do an upcall for a non-application data record if/when I need = to do so.=0A= (The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to what= =0A= the gssd does.)=0A= =0A= For me, the mystery is what to do with it once the daemon gets it.=0A= >From what you said, I'll need to "trick" SSL_read into reading it.=0A= Maybe I can push it back on the socket buffer receive queue in the kernel= =0A= and then the daemon can do a SSL_read() to read it off the socket and handl= e=0A= it?=0A= (I wouldn't want to MSG_PEEK for every record, since these will be rare.)= =0A= I also do already have code that blocks kernel reception when the upcall=0A= to do the handshake is done, so the same could be used in this case.=0A= =0A= There is the slight trick that the client krpc code is in a socket upcall t= hat can't sleep,=0A= so I'll have to hand it off to some other thread that can sleep when I need= to do it.=0A= =0A= Thanks for the hints, rick=0A= =0A= --=0A= John Baldwin=0A= From owner-svn-src-projects@freebsd.org Tue May 12 02:52:05 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 567EA2E066C for ; Tue, 12 May 2020 02:52:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660044.outbound.protection.outlook.com [40.107.66.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Lj5004m6z4bK8; Tue, 12 May 2020 02:52:03 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ms4U9yQBqPZ0CFEk/QJB7IkjVQT7GL+vMEVBSExfNj1ihAq4ktF3YSYnoIg732ycUy+bMJ2nQ0T7W6GKpKA+WsFOag3aJ0EZB6p1qYv4hAjPlXVPcPoWxlWQ5K60dggS41Dn2JPBlob3JIRKxXcAdOXHFgo4L5SLiSQW0Rsv2zGIpWaaHDnX03f+U1EHDPOCHdsr9Yva4pqlibbALL44LA2VGbPsvZvJPqewJlj6cTqFf2Xecfmajn87TKtZTl6c0uFiT38VS3m3DD8w5iMRHcCb092pyE5PzGHZ6RVZ5zFgdY/aztsaAOERCZoOfmCdezZZLhcugNCcPJBlXDAM2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=STZXZS+l9i9Kmn2PlDSUVYmwkMrzrlzmFDmum9kTFAk=; b=ZaiIzy/ctqwS9aLy4mkIRvq+9wk2ghJstfFELWO1nh2/nVXYoLphSqKjOD4JKz9H+5bEudpQ9So4u6qH9YNF6fe6nZH8PX8zdUGLUbeaxYbkqnyKrHq8mnSCEraQCNGLcEGgYkERP3T0KaEe+uEnjlFLO6sBSPWszfFBdgy56j5PvpdYbnpY8+hydYykpP2B2CtJG2qVkMeYgWzNMZeZVZI9XRXHxuSXnSojsrSAnKfaLXGlIOlc++bcFStCW2VX0B43JKe65B1hqaOcaphqJ7mthwfa9kU3w0MXFWr8YoCeXTsBhdk05f9wnBVJT/rGRLkfd6gGHybwOTed5luQYA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM (52.132.86.26) by QB1PR01MB4003.CANPRD01.PROD.OUTLOOK.COM (52.132.89.80) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.28; Tue, 12 May 2020 02:52:01 +0000 Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2]) by QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 02:52:01 +0000 From: Rick Macklem To: John Baldwin , Rick Macklem , "src-committers@freebsd.org" , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Topic: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Index: AQHWJ9HZ1BQ89BXo/ki0EXCO1Y+2cKijtTOsgAAIx/8= Date: Tue, 12 May 2020 02:52:01 +0000 Message-ID: References: <202005100017.04A0Hd7I058863@repo.freebsd.org>, <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f83a53f9-378d-4e62-2920-08d7f61f7263 x-ms-traffictypediagnostic: QB1PR01MB4003: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: N9rPKbEx5+sdkggQmEfO0tdjjqTOYzIDraCJszbmC9eTZKOTYukX6Kjj6ViWdWlOPBiiOdUAHp8AIsRUpddHgEwq08ofXYZWAi5oNYyhGoxyn8yLvpTsy47NG128+pIY4029ibThYjNlx3/FG2hLXjgyegfnxxugrq1N5s6qC3a0pIxlhjvAeHECyCsmwsPSWB8qwHEGrpV36DzcNIvi8nbR0h+a8qS1HSQMD4TDE5XpX0C+zO1QCuF0Iq2xCpQdMPC/jjEmkIWbBxyN1Dv9yvlm2VqBw29uuc12F0owbQfqAcmCh0ATxBNzZxQ9NN7lMTPOCGhxLzPO4/zKaQWGl6+xXNEcPdcGYn8FibxytUkyttVb+WttI7lIO7GmO13bs2LekBuzcBgMXAa5z+L5mRPqopLnNkVc9FNKt/+m5AOkozThXbSq7bfy5Ig+mXn2cVNinmfWkog0D38T5P72XDpq7SjGBAKVeUgk6INadI1RVvCE8ntKSDEOcw8sr/tqKvdycOIjbPZZmd78C3mkB98oaMZaLt1ajvFJX9Y1Gkxa3C+g29Pcn/NeYt3oyU5vSMIYGmRoHkcoe7wMkJ4yq0Lw6pNKeDfd4JNNFY8OpTY= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(366004)(39860400002)(376002)(346002)(136003)(396003)(33430700001)(52536014)(66556008)(450100002)(2940100002)(86362001)(66446008)(786003)(110136005)(316002)(66476007)(64756008)(7696005)(33440700001)(66946007)(55016002)(966005)(76116006)(2906002)(8936002)(8676002)(6506007)(478600001)(5660300002)(71200400001)(186003)(33656002)(9686003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: J4zovoQR8TqDcwXCUT93ku2Vrb5h1GV7xQF3BsZV4UuQwNKVv6JQX7nt1pDjHSlkd5X7YCuq2IBGshaTi3d1UEZ8C23ifg89/eslZcCSP65KTgt8TrkfcKR5yeXZ/SdbNAqZnkxG+4FF9AATU7L/HoUxvntC00i5/MXlPWRN051cgTVCOFvTaxdL7BWwb+yiJBFlGw3w4CvBXzpu1AzrGLqSzcjdOCnnCCzF8CmExbQeksFbtmwYdysgyawrvQEGpk53XxcEFTt4qPSyW39LVlEg/+L/N0CLqTinHkH1jrlI4P8lGH+j3JOxCzDz6vIfL/vLPOk9KHrQduFV67rUncF83J3FYMCCjcBpeSTJ5Xr6ba+kBfovyiK7JAW8P8Dk7dfrEZFdO8z7mdfMSG+gLOodSz+CpRRK5NR/1wkx2kwouUq9F679iL3je8t2KRtOuuzr5cb7BXBG+d9/OeaBHR0+re/ai2ljKVGNAfGxUCybmAh51rNpu3zC5ijYK7A0mgqQCmb3vP2npuHhGKvDygEaMrZuA++D+qGu8/toN2F2uvtabg3rdiVcA43dgF03 x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: f83a53f9-378d-4e62-2920-08d7f61f7263 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 02:52:01.9195 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nGnAuiNR1zkCye70Kns9BSd1+HbQtLTOze/78x9WFq+nUgW7zDRd0rdC69OqrcCtfNhyM6ODQEuS+j4dPYgLCw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB4003 X-Rspamd-Queue-Id: 49Lj5004m6z4bK8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.66.44 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.70 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[uoguelph.ca]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[44.66.107.40.list.dnswl.org : 127.0.3.0]; IP_SCORE(-1.40)[ipnet: 40.64.0.0/10(-3.73), asn: 8075(-3.24), country: US(-0.05)]; RWL_MAILSPIKE_POSSIBLE(0.00)[44.66.107.40.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; ARC_ALLOW(-1.00)[i=1]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 02:52:05 -0000 Rick Macklem wrote:=0A= >John Baldwin wrote:=0A= >>On 5/9/20 5:17 PM, Rick Macklem wrote:=0A= >>> Author: rmacklem=0A= >>> Date: Sun May 10 00:17:39 2020=0A= >>> New Revision: 360859=0A= >>> URL: https://svnweb.freebsd.org/changeset/base/360859=0A= >>>=0A= >>> Log:=0A= >>> Add some very basic handling of TLS_GET_RECORD control mbufs.=0A= >>>=0A= >>> For now, it just throws away any that are non-application data.=0A= >>> In the future, this will need to change, but not until TLS1.3, I thin= k?=0A= >>=0A= >>Ideally you'd keep an nfsd thread in userland that you could pass=0A= >>these records onto. One possible option is the thread just keeps=0A= >>calling SSL_read() but you do create a new flag on the socket buffer=0A= >>that causes soreceive() to only pass non-application data datagrams=0A= >>to userland reads() and have the in-kernel read requests block if they=0A= >>see a non-application data record as the next record until the user=0A= >>thread wakes up and reads it (or EAGAIN or whatever you need it to=0A= >>do).=0A= >Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for= =0A= >upcalls from the kernel and do the SSL stuff (mainly the handshake right n= ow).=0A= >(You can guess from the names which one is RPC client vs server.;-)=0A= >I can easily do an upcall for a non-application data record if/when I need= to do so.=0A= >(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to wha= t=0A= > the gssd does.)=0A= >=0A= >For me, the mystery is what to do with it once the daemon gets it.=0A= >From what you said, I'll need to "trick" SSL_read into reading it.=0A= >Maybe I can push it back on the socket buffer receive queue in the kernel= =0A= >and then the daemon can do a SSL_read() to read it off the socket and hand= le=0A= >it?=0A= Oh, and one more little challenge...=0A= When I played around with the daemons using TLS1.3 (before there was a ktls= =0A= rx I could use), I would run into early data that would be handled by SSL_r= ead()=0A= done in userland by the daemon.=0A= However, I couldn't find a way to tell it to not wait to read any applicati= on data.=0A= I recall trying an SSL_read() for 0 bytes and it didn't like it.=0A= I might be stuck having the daemon do an SSL_read() for one application dat= a=0A= record and then it can pass that data back down into the kernel to be prepe= nded=0A= on the queue of received application data.=0A= =0A= >(I wouldn't want to MSG_PEEK for every record, since these will be rare.)= =0A= >I also do already have code that blocks kernel reception when the upcall= =0A= >to do the handshake is done, so the same could be used in this case.=0A= >=0A= >There is the slight trick that the client krpc code is in a socket upcall = that can't sleep,=0A= >so I'll have to hand it off to some other thread that can sleep when I nee= d to do it.=0A= >=0A= >Thanks for the hints, rick=0A= rick=0A= --=0A= John Baldwin=0A= From owner-svn-src-projects@freebsd.org Tue May 12 04:02:58 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F06372E1CD5 for ; Tue, 12 May 2020 04:02:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670084.outbound.protection.outlook.com [40.107.67.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Lkfn3Zc7z4fp5; Tue, 12 May 2020 04:02:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AduxdVAym1hzSpUl7DkYmgYY1kUe0AY2YiUPiAjTGHpu+7ibi1CgiLEmg54qxD0HRFOAjm1PKVG9sge8+qbqQIM4UTkBMODw6QPZZRdbq2S22jg5KsifFDdm8Gan60JYMuzYQoqpEIsr4KrgV8EiFo3Cz7TUYt4xjckfZPJ4auMsSVl9Z+sbs+UBN5Q7Pch88DkbiawqcqcAb38fuxY+JSigZNyAXyACoVnQGO3MjSjUe0Mkvu3xh8VxSV9Mv3HGPu5c2ZKiaSfFn8KsL8amrsyqOQNy+Q4nQ/8GYs0zvV5qAhMbsc8RqswE+URjZKJAraA6hxF4In0e+S0oi2rzww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l4kTv+J+I8LBHd8/mtLLWumWb2MauBsJia6Vk2D41fo=; b=bqO4cVZlqTWXweHFJEXPBl4iqI7tppJ4lTVMwnteRKQHzkGIFObTjHvsm1wxE4/qvgCKW65sgYt7jjVx4DDZ2VoepVSzPksKjD0pVFZi5Sfv/GI2GaB4bFNwZDtB/TsANpwoZfvc+BEi7yNjASvY7L/sKCRfy9HLXCTFQ1w2mETocw4FqXmbo4vbXHrKxW2VtiCK+d0+2q1jKI0QcFA9Xcbb9l8TPj4LnqQXkysCwR5PgYJzOyMTrA48TxNMS+QI9D+NpEaaXDQTs1cykMfXrNxsAy5wDOB5oBsEixCt0xBolERMLBQRv2MK0e0HOGuLTnnFL9EyJCMNoRfzvsT68A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM (52.132.86.26) by QB1PR01MB3091.CANPRD01.PROD.OUTLOOK.COM (52.132.87.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.29; Tue, 12 May 2020 04:02:55 +0000 Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2]) by QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 04:02:55 +0000 From: Rick Macklem To: John Baldwin , Rick Macklem , "src-committers@freebsd.org" , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Topic: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Index: AQHWJ9HZ1BQ89BXo/ki0EXCO1Y+2cKijtTOsgAAIx/+AABDwDQ== Date: Tue, 12 May 2020 04:02:55 +0000 Message-ID: References: <202005100017.04A0Hd7I058863@repo.freebsd.org>, <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org>, , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ebf3a76e-db7f-4286-2518-08d7f62959cf x-ms-traffictypediagnostic: QB1PR01MB3091: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PFL9JvtxMidpeZHyS4dYGe8Qi7F69hGOgD8mi4vjCqJ673dk8hsEngfJF2x0jIawdSe6990tVz03fOF14r48QHckUmRwvUozqvZBSxH6QDMA62DobOW3QSsBbdwpLLS7TpA5In9LkNoniblP9aad4RpH7sTG86zt/PkwIHOU+g7CeladTTmJc+v/yDosudR+cprVCn9lSR3iDzOP1aZYa7a6IyEbh2uOZiJEzQz1MkQn4y/Ri0Ht+5ip4n3ErMqHHseq7fMoknC2uphDmhqok15aQGyGnnA/S3rdKMAQGCJHLDkD16omQd5KFfgF3lfUMS3Fofnq7dX9HTvt6Gvl6AmaiKS4g5zOFSbzHTirrls7OL/VXhA2fhzJhIPZr9EEAOtQ1Eb/vacLlZW79JspiepOFEJRL1KlZzCIMmt2JdC62kF9Q0BUdzQnHJSSzNDVm+1MQxA2OB5/Jlo9sQ2KEuJzA/3cwCbxXq0MTj+LhzuHud//xUqmtV1Rn8cnjjqPYKm8di8oaQ/XawM050uS9rXFKjVOx4KoLuV6G2aS1Qq1cyXSPwyhyNRW/g7xzOZap68rGGUhYEZ6qmRyR9KnICQucjMMimfoHJm2vxE+38k= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(39860400002)(136003)(376002)(396003)(346002)(366004)(33430700001)(786003)(5660300002)(66476007)(7696005)(76116006)(86362001)(66556008)(64756008)(52536014)(8676002)(66946007)(66446008)(9686003)(110136005)(33656002)(2906002)(2940100002)(33440700001)(6506007)(316002)(186003)(966005)(478600001)(71200400001)(450100002)(55016002)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: cAEz8GpwpEC9NGKvBAoUayehV+1T2EGY7tHBrwnAuLnOrEbHjgOqX6YMvQP8yDg5BHb5EX0FrPoxe7T65De8jMcagXjkS8AvYqP8/+KRvvuRZlS4THmxg0iE2n6CKUQfs95k+fA8oi1pch+hA16vCssSNbRa2QNLftjXBrD3UZVTlFluz5PkhnQpQWD9nO8WohY8g54yE2OJOqCqr0rr2Nym68qYcUAj6Pz8hMe4swO51dT2/PC9fDSHweN5qlhzYcREQ6PTrQJ5YNmULKBA7nWPJXZ6IANGhx+WEQF4fqpXYMCZLp5bDDvumrqv0EOI3bsmB4MB2d7+U+rSmkJkdFLTqLbYCNwQu/nkFJfIjA2+R7u3wN7NjTzcH5WZIanViGW/kgXBfyj+fehUH70H4ILI5kcx4Lrm/u7tnOItz3d5KaSjFnXgc1AceDIbYtqYeNx4y48pVI4y2nntZYkRzx1eMRvGEGDgA1fUOu56oQwBHH0RVtIWZhbQCqLdg6ZfYsiOc8moyQbxNHDLRtXcS99uebkH0UFmvtjZ63D9+uhG7agqYqfKrh7SbbVdsiSe x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: ebf3a76e-db7f-4286-2518-08d7f62959cf X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 04:02:55.6544 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: AAgKa1KWPGnN+3pYuDSvtiVK85xp7I0V93xCMEx0X4Lx6pzwKTJFU084aWBkNTQjnmTidy5r6j7HKUo7LdILcQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB3091 X-Rspamd-Queue-Id: 49Lkfn3Zc7z4fp5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.84 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.80 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RWL_MAILSPIKE_POSSIBLE(0.00)[84.67.107.40.rep.mailspike.net : 127.0.0.17]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[uoguelph.ca]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-1.40)[ipnet: 40.64.0.0/10(-3.73), asn: 8075(-3.24), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[84.67.107.40.list.dnswl.org : 127.0.3.1]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; MIME_TRACE(0.00)[0:+]; ARC_ALLOW(-1.00)[i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 04:02:59 -0000 >Rick Macklem wrote:=0A= >>John Baldwin wrote:=0A= >>>On 5/9/20 5:17 PM, Rick Macklem wrote:=0A= >>>> Author: rmacklem=0A= >>>> Date: Sun May 10 00:17:39 2020=0A= >>>> New Revision: 360859=0A= >>>> URL: https://svnweb.freebsd.org/changeset/base/360859=0A= >>>>=0A= >>>> Log:=0A= >>>> Add some very basic handling of TLS_GET_RECORD control mbufs.=0A= >>>>=0A= >>>> For now, it just throws away any that are non-application data.=0A= >>>> In the future, this will need to change, but not until TLS1.3, I thi= nk?=0A= >>>=0A= >>>Ideally you'd keep an nfsd thread in userland that you could pass=0A= >>>these records onto. One possible option is the thread just keeps=0A= >>>calling SSL_read() but you do create a new flag on the socket buffer=0A= >>>that causes soreceive() to only pass non-application data datagrams=0A= >>>to userland reads() and have the in-kernel read requests block if they= =0A= >>>see a non-application data record as the next record until the user=0A= >>>thread wakes up and reads it (or EAGAIN or whatever you need it to=0A= >>>do).=0A= Actually, what might work for the krpc code is a new MSG_TLSAPPDATA=0A= flag for soreceive_generic(), which says "if the record is not application= =0A= data, return an error". (Sort of the opposite of what you said above, but= =0A= would perform the same thing.)=0A= This could be used for the krpc soreceive() calls, so that the non-applicat= ion=0A= data record remains on the socket's receive buffer.=0A= =0A= Then the krpc could do the upcall when the error is returned by soreceive()= =0A= and the userland daemon could do an SSL_read() with=0A= SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will=0A= make SSL_read() process the non-application data record but return with an= =0A= error of SSL_ERROR_WANT_READ without taking application data off the=0A= socket's receive buffer queue.=0A= --> If this all works (?), then the krpc can just go on and soreceive() the= next=0A= application data record after the upcall returns.=0A= =0A= Worth a try anyhow, I think? rick=0A= =0A= >>Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for= =0A= >>upcalls from the kernel and do the SSL stuff (mainly the handshake right = now).=0A= >(You can guess from the names which one is RPC client vs server.;-)=0A= >I can easily do an upcall for a non-application data record if/when I need= to do so.=0A= >(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to wha= t=0A= > the gssd does.)=0A= >=0A= >For me, the mystery is what to do with it once the daemon gets it.=0A= >From what you said, I'll need to "trick" SSL_read into reading it.=0A= >Maybe I can push it back on the socket buffer receive queue in the kernel= =0A= >and then the daemon can do a SSL_read() to read it off the socket and hand= le=0A= >it?=0A= Oh, and one more little challenge...=0A= When I played around with the daemons using TLS1.3 (before there was a ktls= =0A= rx I could use), I would run into early data that would be handled by SSL_r= ead()=0A= done in userland by the daemon.=0A= However, I couldn't find a way to tell it to not wait to read any applicati= on data.=0A= I recall trying an SSL_read() for 0 bytes and it didn't like it.=0A= I might be stuck having the daemon do an SSL_read() for one application dat= a=0A= record and then it can pass that data back down into the kernel to be prepe= nded=0A= on the queue of received application data.=0A= =0A= >(I wouldn't want to MSG_PEEK for every record, since these will be rare.)= =0A= >I also do already have code that blocks kernel reception when the upcall= =0A= >to do the handshake is done, so the same could be used in this case.=0A= >=0A= >There is the slight trick that the client krpc code is in a socket upcall = that can't sleep,=0A= >so I'll have to hand it off to some other thread that can sleep when I nee= d to do it.=0A= >=0A= >Thanks for the hints, rick=0A= rick=0A= --=0A= John Baldwin=0A= From owner-svn-src-projects@freebsd.org Tue May 12 05:07:23 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9178B2E2C13 for ; Tue, 12 May 2020 05:07:23 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Lm563Cdyz3Dj1; Tue, 12 May 2020 05:07:22 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: by mail-ot1-x32f.google.com with SMTP id m33so9536220otc.5; Mon, 11 May 2020 22:07:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KHtyApV04rNlxzmJsT4vGKytNj37FxWFzryXO6y9N5s=; b=p9dblCWd22bPk+NEXEd5yLg0APbtxhw7Q/lCrvdmCS4yVgH0oxEdFk/id5d949FKnL BUbrt9GgZ3AJPQmdNr5tH/VotByDG6C5/0yTDsJCP4XRfDLHXvE5jJIBx/Obdm+EoD+e ZjxqbZxWyObANMG0LlriLScZlnydEvzwty4zzBRbeHGdzRQyMjeZ96xA7q2PQ5lVc84G 4ed+DgT1AFQG30WP40WEWSIfYQRll+EASK197tlSGybyjM0J+I3B4YXB3228VNxeMlC3 i2Fs83/yaSVwvI1S8t9bW+IrfDksQFikxmquervzy+j7BKoAjTIjT5y5lM1FbJzB7NJV Iiwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KHtyApV04rNlxzmJsT4vGKytNj37FxWFzryXO6y9N5s=; b=UlxI9O1uZ7efLTWAFNF0sm6hoWkQCT6CBGu3QodapZgVhnp7Q8Oc3t+BLxAfYoAJNu nqREXj72c86f4OfwcnWHO8lSPrMugj+AfiIByW6W38XMZBy4IqEMGGSrCXnIBMv+ny7A TdkGPYog6sFxZ0/+K5+K92JFI8A/C33PBmA6shOq1W1DI7e2A1or+UDe7yYhpb5gAvBy wSPln0EsG/LgBXdUQLEcTFGgNh9IfsxCT5ArwGf9OuHPleSojlQhdE03noFic1c5/ij1 L1A4W7TWvOAZdSu9IM+R77ETKFxta377PEgcvsQhMe8z1FNizLCoTXJA2veo7XSmIehu HH7w== X-Gm-Message-State: AGi0PuYR8mncGg47fkWIQVbwJIKN0prpVXEKn2T/lNpghCi0Y/D6J0Hf 0yCS3WHTFtlDpl1TFwM0jMULgfM2DqTjTV6Gp09e0S7n X-Google-Smtp-Source: APiQypLGI/rlliKHL5J4r+rIBPDhlxe+E1YA8/6/EXJlb2ZfnI1MwNY7HdfWlbdooNooQSfjpSF+6YLODPCyjCARSA8= X-Received: by 2002:a9d:7a6:: with SMTP id 35mr13902685oto.201.1589260041168; Mon, 11 May 2020 22:07:21 -0700 (PDT) MIME-Version: 1.0 References: <202005100017.04A0Hd7I058863@repo.freebsd.org> <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> In-Reply-To: From: Benjamin Kaduk Date: Mon, 11 May 2020 22:07:10 -0700 Message-ID: Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc To: Rick Macklem Cc: John Baldwin , Rick Macklem , "src-committers@freebsd.org" , "svn-src-projects@freebsd.org" X-Rspamd-Queue-Id: 49Lm563Cdyz3Dj1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=p9dblCWd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of bjkfbsd@gmail.com designates 2607:f8b0:4864:20::32f as permitted sender) smtp.mailfrom=bjkfbsd@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(0.00)[ip: (-7.80), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.42), country: US(-0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[f.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 05:07:23 -0000 On Mon, May 11, 2020 at 9:03 PM Rick Macklem wrote: > >Rick Macklem wrote: > >>John Baldwin wrote: > >>>On 5/9/20 5:17 PM, Rick Macklem wrote: > >>>> Author: rmacklem > >>>> Date: Sun May 10 00:17:39 2020 > >>>> New Revision: 360859 > >>>> URL: https://svnweb.freebsd.org/changeset/base/360859 > >>>> > >>>> Log: > >>>> Add some very basic handling of TLS_GET_RECORD control mbufs. > >>>> > >>>> For now, it just throws away any that are non-application data. > >>>> In the future, this will need to change, but not until TLS1.3, I > think? > >>> > >>>Ideally you'd keep an nfsd thread in userland that you could pass > >>>these records onto. One possible option is the thread just keeps > >>>calling SSL_read() but you do create a new flag on the socket buffer > >>>that causes soreceive() to only pass non-application data datagrams > >>>to userland reads() and have the in-kernel read requests block if they > >>>see a non-application data record as the next record until the user > >>>thread wakes up and reads it (or EAGAIN or whatever you need it to > >>>do). > You can avoid having to play games with putting stuff back on the socket receive buffer by using a custom BIO implementation in userspace that knows how to inject the received message. > Actually, what might work for the krpc code is a new MSG_TLSAPPDATA > flag for soreceive_generic(), which says "if the record is not application > data, return an error". (Sort of the opposite of what you said above, but > would perform the same thing.) > This could be used for the krpc soreceive() calls, so that the > non-application > data record remains on the socket's receive buffer. > > Then the krpc could do the upcall when the error is returned by soreceive() > and the userland daemon could do an SSL_read() with > SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will > make SSL_read() process the non-application data record but return with an > error of SSL_ERROR_WANT_READ without taking application data off the > socket's receive buffer queue. > The typical way to consume non-application-data records without hanging trying to read any application data is to do a zero-length read. This still gets far enough into the state machine machinery to do the job before checking that the length is nonzero. > --> If this all works (?), then the krpc can just go on and soreceive() > the next > application data record after the upcall returns. > > Worth a try anyhow, I think? rick > > >>Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for > >>upcalls from the kernel and do the SSL stuff (mainly the handshake right > now). > >(You can guess from the names which one is RPC client vs server.;-) > >I can easily do an upcall for a non-application data record if/when I > need to do so. > >(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to > what > > the gssd does.) > > > >For me, the mystery is what to do with it once the daemon gets it. > >From what you said, I'll need to "trick" SSL_read into reading it. > >Maybe I can push it back on the socket buffer receive queue in the kernel > >and then the daemon can do a SSL_read() to read it off the socket and > handle > >it? > Oh, and one more little challenge... > When I played around with the daemons using TLS1.3 (before there was a ktls > rx I could use), I would run into early data that would be handled by > SSL_read() > done in userland by the daemon. > However, I couldn't find a way to tell it to not wait to read any > application data. > I recall trying an SSL_read() for 0 bytes and it didn't like it. > In the early-data case things are more complicated. Calling regular SSL_read() will drive the handshake to completion, and there's a separate function to call to just try to read early data. (You could also configure things to fully deny early data which would probably be easier.) -Ben > I might be stuck having the daemon do an SSL_read() for one application > data > record and then it can pass that data back down into the kernel to be > prepended > on the queue of received application data. > > >(I wouldn't want to MSG_PEEK for every record, since these will be rare.) > >I also do already have code that blocks kernel reception when the upcall > >to do the handshake is done, so the same could be used in this case. > > > >There is the slight trick that the client krpc code is in a socket upcall > that can't sleep, > >so I'll have to hand it off to some other thread that can sleep when I > need to do it. > > > >Thanks for the hints, rick > rick > -- > John Baldwin > _______________________________________________ > svn-src-projects@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-projects > To unsubscribe, send any mail to "svn-src-projects-unsubscribe@freebsd.org > " > From owner-svn-src-projects@freebsd.org Tue May 12 14:32:33 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F40E82F13DB for ; Tue, 12 May 2020 14:32:32 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on0612.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::612]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49M0dD0FBMz4KJ8; Tue, 12 May 2020 14:32:31 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VXy5RvJMQF1v9MBREWetldjp7hMs1fOE/vrUU2e8jTiB78z3rIMflyAc0o/qx3yCHvUYCwqzrYpTmGs0nXuIU9yRgkGMMj+XR2RVVn7UgNOqcrn0Uxr3WOW1LIxiVKzr3sztqiXy+QUAGDb+VWmv5oSboMNs1RjYkC2TaBE8Ei/GG+rDTZQ8/Hj/srKYHfplXOaZ8y2KU3JDmRJ5WYxw1OAnJRBgK4nH5WD5e4FWw+juW+LzJAk7M08js2OnFxc4oTF2PXG3FVtt+3k9BVfUVhfHzPiSmk8ofpPtHmGeJWFdDitDQsF2AkAu5pv/JG+BuNb9o3Q86drdveWrmjbE8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZOUJsSOPGByLp1+Kzgm7Uvdkx92a/FN8VsqctnpfVyI=; b=eV/CexJcN9YXSZtTpTqh8SyStrlo7wAGOvdmYrdgkEDMR+ZMG2ChwiCx4zvPRnK4Ul6en47WRv0Js7YT7wZG0PlGBx4MPANr/4zUfZyFqE7cmW1608LQts3LQ0i41ixA1/oJ+dJP2LS+OYPtguHKll0GGYbPYi+jonAsEkizxEYToWnztQ/ljopY9La0qX1/Ss8VpqBfclFD2K7QQ+X0Aq7UEgdZDtvmcDv6Xn51m2OFeX2yWDmpQTo1Vc3gS4KulXTpyTXn62Hm1uItkiKLHsYd8xgAIXx/Eq4eHncXbx77xQcv2ZR0HJgBlJP+o1ZkZqOQvX2pfe3TjPUDxMqPfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZOUJsSOPGByLp1+Kzgm7Uvdkx92a/FN8VsqctnpfVyI=; b=nIRZkH5Pj4r+XtmFDQthBsBNAPjcRELcal4d+80fwHGo78WUxsG9aNtwVyFh3XkqnGVgZ8NUB48mSsEVPu9EXZ4UrSmbteXza969bOztMifvdvSQZQ7ijwZiVqUtKv3Iqv6zqu6sxs6sZYs9k5pUIUY+Og5E4yWJdcuR+7mxW0HQgi2Ex1mLhLHIF57vZDN0Hzc/qA210+QSS+JQWEHjWEj9FN0V1QN9eRKUj+z9mHxJEaEb3rf1FM6HtzoJDpmb553ksKbeQXcTmdpp8lSIQur12mZNoxSeik4ZhNoXM/Kc4PSK+oHtQmwDWBSD1Sn+oH+sSq8XYN3tP1xXkgWmcg== Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM (52.132.86.26) by QB1PR01MB3427.CANPRD01.PROD.OUTLOOK.COM (52.132.87.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.28; Tue, 12 May 2020 14:32:29 +0000 Received: from QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2]) by QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM ([fe80::dd96:945c:b6ee:ffa2%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 14:32:29 +0000 From: Rick Macklem To: Benjamin Kaduk CC: John Baldwin , Rick Macklem , "src-committers@freebsd.org" , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Topic: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Thread-Index: AQHWJ9HZ1BQ89BXo/ki0EXCO1Y+2cKijtTOsgAAIx/+AABDwDYAAF9MAgACaLek= Date: Tue, 12 May 2020 14:32:29 +0000 Message-ID: References: <202005100017.04A0Hd7I058863@repo.freebsd.org> <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 501da98e-a172-4b08-b087-08d7f6814cd7 x-ms-traffictypediagnostic: QB1PR01MB3427: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PU3jXYgQWUpj55+aq0JoYsBhFs9WXnNyrx/qT56GgZ4AOotEcyIIPrmJMo1clyado2dtW0yhdJOvKiYblKl0fGr/PB53eUuyywJBtOR2mi9UqeWijgwYv06M/ivkh22woLIo+ERiak2srTMbb6u9tW/kqkXwcBmrp0wezpquQOlkU0F5p6IM2UPm9k1Ow/uW24zH5vC9E1u3ONrKr3bC2HF7LUpNC5HHeLW8XlwSW0BnbHKp9I2SVncMFTvBFvnuhRVyuJnmNi/CQ4mUREkMSYNe4Ar2DJIFJjEldsC4LzrczFsu2IFkfr6jJ0bixNbdiIbfsQLZI4JvM/8XmsTIwwMCtC3KREUNDTe1CXQKt+5vegIj38UwOTTPmMoRqrTHII45B4RPHYAetj5V4rqnOdymV8RT9GLj9VCv4vKZrpiEc2lFJ2O2YY2dvQs6Uro6aXi2fv/5ZL3Xt0nYcMgMHZT1nFIXr7MZy/KXTCsSVj9EoiILhGZO+t26UOcEZkll28VE8lbwkbnHvGqazJp4xM6IoLxuJAsw0j49rXL3JmF17dnAtFFnu4GVImxXbJmOINRkuU6oYNcqNDwrdQTNEPU6XB92cxoXiMWz2WD77jY= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(366004)(39860400002)(136003)(396003)(376002)(346002)(33430700001)(786003)(52536014)(86362001)(316002)(33440700001)(66556008)(66446008)(7696005)(66476007)(76116006)(966005)(55016002)(66946007)(64756008)(8676002)(54906003)(186003)(478600001)(71200400001)(2906002)(6506007)(6916009)(8936002)(33656002)(9686003)(5660300002)(4326008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: q1/ODOLA0hIMo97zFny79LtSGM/ctkPnP7sYfvHLGwMHBA/hpaQwr0wueZ0YM25HNA5UBQYG8QmyAcz1E/Z/JO28ds8th1/JjD7cxBAuLsXDrTt5GLx2PoQKHFo8YkDve/mG0MIzR1OkKeBaEd36Wyzp6wUQQuen73z1kQxSSaOJPM7jAGXnEeK0FM3akqQd7zh6r0mC/CRXFMUgPB79dYnK7jEIuH0Os5AvyGYm+nRMsBrrozuDfAYiTtdvgCST/n8AQ2ZV8T3gyIWZQFwHNy5TfcbN3JYrI4XWGQONgkmd12uKSzefdJo8jmW/SX1latiggfz6qZUNVZCbuaA9tmrvhRxNdiUGI+lFgNMY+5SM6mch1xW091Z7Wh4s1aNTywdJtyCdps5delxsuN66qd4T4zxQZDVuGcCaVGI2zgpoH8CHipa/uyGHKe0d1ofzzZ+A8o+WdmRzsZGWgN7Yil913jqdkp3KwX/JD4oVzHWXEZ1EAF3kTgOKJy1s3TsHEena7kuopXnGPn+qLtG4vqJWxARyUtbgYCDQlnQ9GPA= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 501da98e-a172-4b08-b087-08d7f6814cd7 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 14:32:29.5812 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: C1xVaJAZEyIupSLqzae1QKxSL7flaDjNKF1NMmrvE+gL4t4Wn3PAuqI6k74YZ3WDqVid9Yy+yxfYt14b2WqjUg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB3427 X-Rspamd-Queue-Id: 49M0dD0FBMz4KJ8 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=nIRZkH5P; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::612 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; RCPT_COUNT_FIVE(0.00)[5]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca.dwl.dnswl.org : 127.0.11.1]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; IP_SCORE(-1.50)[ipnet: 2a01:111:f000::/36(-4.20), asn: 8075(-3.25), country: US(-0.05)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 14:32:33 -0000 Benjamin Kaduk wrote:=0A= [stuff snipped]=0A= >You can avoid having to play games with putting stuff back on the socket= =0A= >receive buffer by using a custom BIO implementation in userspace that know= s=0A= >how to inject the received message.=0A= >Rick Macklem wrote:=0A= >>Actually, what might work for the krpc code is a new MSG_TLSAPPDATA=0A= >>flag for soreceive_generic(), which says "if the record is not applicatio= n=0A= >>data, return an error". (Sort of the opposite of what you said above, but= =0A= >>would perform the same thing.)=0A= >>This could be used for the krpc soreceive() calls, so that the non-applic= ation=0A= >>data record remains on the socket's receive buffer.=0A= Well, I'd find it a lot easier to implement MSG_TLSAPPDATA, since I've been= =0A= looking at soreceive_generic() recently.=0A= I'm guessing that a custom BIO would need to be written and the upstreamed= =0A= to openssl?=0A= =0A= Does anyone else (John maybe) have a preference?=0A= =0A= >>Then the krpc could do the upcall when the error is returned by soreceive= ()=0A= >>and the userland daemon could do an SSL_read() with=0A= >>SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will= =0A= >>make SSL_read() process the non-application data record but return with a= n=0A= >>error of SSL_ERROR_WANT_READ without taking application data off the=0A= >>socket's receive buffer queue.=0A= >=0A= >The typical way to consume non-application-data records without hanging tr= ying=0A= >to read any application data is to do a zero-length read. This still gets= far enough=0A= >into the state machine machinery to do the job before checking that the le= ngth=0A= >is nonzero.=0A= Oh, that's useful info. I recall trying it months ago and it didn't seem to= work.=0A= Maybe I was just seeing the failure and not realizing that it had worked bu= t returned=0A= failure. I'll play with this using my userland test daemons.=0A= [most stuff snipped]=0A= >In the early-data case things are more complicated. Calling regular SSL_r= ead() will=0A= >drive the handshake to completion, and there's a separate function to call= to just=0A= >try to read early data. (You could also configure things to fully deny ea= rly data which=0A= >would probably be easier.)=0A= Thanks. Good suggestion. I've never known what to do with the early data wh= en I=0A= played with the test daemons. (Again, I just threw it away.)=0A= =0A= Btw, I think I can test this using TLS1.2 because when one end does SSL_shu= tdown(),=0A= TLS1.2 sends an "alert close" and I can try handling that.=0A= =0A= Thanks for the comments, rick=0A= =0A= -Ben=0A= =0A= I might be stuck having the daemon do an SSL_read() for one application dat= a=0A= record and then it can pass that data back down into the kernel to be prepe= nded=0A= on the queue of received application data.=0A= =0A= >(I wouldn't want to MSG_PEEK for every record, since these will be rare.)= =0A= >I also do already have code that blocks kernel reception when the upcall= =0A= >to do the handshake is done, so the same could be used in this case.=0A= >=0A= >There is the slight trick that the client krpc code is in a socket upcall = that can't sleep,=0A= >so I'll have to hand it off to some other thread that can sleep when I nee= d to do it.=0A= >=0A= >Thanks for the hints, rick=0A= rick=0A= --=0A= John Baldwin=0A= _______________________________________________=0A= svn-src-projects@freebsd.org mailing l= ist=0A= https://lists.freebsd.org/mailman/listinfo/svn-src-projects=0A= To unsubscribe, send any mail to "svn-src-projects-unsubscribe@freebsd.org<= mailto:svn-src-projects-unsubscribe@freebsd.org>"=0A= From owner-svn-src-projects@freebsd.org Tue May 12 23:33:04 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A0E072E0C3C for ; Tue, 12 May 2020 23:33:04 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49MDcw28VKz4Dkv; Tue, 12 May 2020 23:33:04 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4494D18326; Tue, 12 May 2020 23:33:04 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04CNX4i5017703; Tue, 12 May 2020 23:33:04 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04CNX4t2017702; Tue, 12 May 2020 23:33:04 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005122333.04CNX4t2017702@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 12 May 2020 23:33:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360994 - projects/nfs-over-tls/sys/fs/nfs X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/fs/nfs X-SVN-Commit-Revision: 360994 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 23:33:04 -0000 Author: rmacklem Date: Tue May 12 23:33:03 2020 New Revision: 360994 URL: https://svnweb.freebsd.org/changeset/base/360994 Log: Get rid of two fields that were only used for dissecting ext_pgs mbufs. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs.h Modified: projects/nfs-over-tls/sys/fs/nfs/nfs.h ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs.h Tue May 12 22:44:51 2020 (r360993) +++ projects/nfs-over-tls/sys/fs/nfs/nfs.h Tue May 12 23:33:03 2020 (r360994) @@ -645,8 +645,6 @@ struct nfsrv_descript { NFSSOCKADDR_T nd_nam; /* and socket addr */ NFSSOCKADDR_T nd_nam2; /* return socket addr */ caddr_t nd_dpos; /* Current dissect pos */ - int nd_dextpg; /* Current ext_pgs page */ - int nd_dextpgsiz; /* Bytes left in page */ caddr_t nd_bpos; /* Current build pos */ int nd_bextpg; /* Current ext_pgs page */ int nd_bextpgsiz; /* Bytes left in page */ From owner-svn-src-projects@freebsd.org Wed May 13 00:03:41 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5D2D72E18BD for ; Wed, 13 May 2020 00:03:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49MFJF1Yy4z4G9c; Wed, 13 May 2020 00:03:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 30EA1188DA; Wed, 13 May 2020 00:03:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04D03fjr036333; Wed, 13 May 2020 00:03:41 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04D03dQD036326; Wed, 13 May 2020 00:03:39 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005130003.04D03dQD036326@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 13 May 2020 00:03:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r360997 - in projects/nfs-over-tls/sys: fs/nfsclient rpc xdr X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys: fs/nfsclient rpc xdr X-SVN-Commit-Revision: 360997 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2020 00:03:41 -0000 Author: rmacklem Date: Wed May 13 00:03:39 2020 New Revision: 360997 URL: https://svnweb.freebsd.org/changeset/base/360997 Log: Get rid of the changes done to the krpc and xdr for dissecting ext_pgs mbufs. Also do a few code cleanups, such as comments, printfs and whitespace while at it. All of the ext_pgs mbuf dissection code should now be out of this tree. Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c projects/nfs-over-tls/sys/rpc/clnt.h projects/nfs-over-tls/sys/rpc/clnt_bck.c projects/nfs-over-tls/sys/rpc/clnt_vc.c projects/nfs-over-tls/sys/rpc/svc.h projects/nfs-over-tls/sys/rpc/svc_vc.c projects/nfs-over-tls/sys/xdr/xdr_mbuf.c Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Wed May 13 00:03:39 2020 (r360997) @@ -94,7 +94,7 @@ printf("cbprogram proc=%d\n", rqst->rq_proc); rqst->rq_args = NULL; newnfs_realign(&nd.nd_mrep, M_WAITOK); nd.nd_md = nd.nd_mrep; -printf("cbreq nd_md=%p offs=%d\n", nd.nd_md, rqst->rq_xprt->xp_mbufoffs); +printf("cbreq nd_md=%p\n", nd.nd_md); nd.nd_dpos = mtod(nd.nd_md, char *); nd.nd_nam = svc_getrpccaller(rqst); nd.nd_nam2 = rqst->rq_addr; Modified: projects/nfs-over-tls/sys/rpc/clnt.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt.h Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/rpc/clnt.h Wed May 13 00:03:39 2020 (r360997) @@ -120,7 +120,6 @@ struct rpc_callextra { void *rc_feedback_arg; /* argument for callback */ struct rpc_timers *rc_timers; /* optional RTT timers */ struct rpc_err rc_err; /* detailed call status */ - u_int rc_mbufoffs; /* Offset in resultsp mbuf */ }; #endif Modified: projects/nfs-over-tls/sys/rpc/clnt_bck.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_bck.c Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/rpc/clnt_bck.c Wed May 13 00:03:39 2020 (r360997) @@ -89,7 +89,6 @@ __FBSDID("$FreeBSD$"); #include #include - struct cmessage { struct cmsghdr cmsg; struct cmsgcred cmcred; @@ -209,7 +208,6 @@ clnt_bck_call( struct mbuf *mreq = NULL, *results; struct ct_request *cr; int error, maxextsiz; - uint32_t junk; #ifdef KERN_TLS u_int maxlen; #endif @@ -425,9 +423,7 @@ got_reply: ext->rc_feedback(FEEDBACK_OK, proc, ext->rc_feedback_arg); xdrmbuf_create(&xdrs, cr->cr_mrep, XDR_DECODE); - ok = xdr_uint32_t(&xdrs, &junk); - if (ok) - ok = xdr_replymsg(&xdrs, &reply_msg); + ok = xdr_replymsg(&xdrs, &reply_msg); cr->cr_mrep = NULL; if (ok) { @@ -446,14 +442,6 @@ got_reply: } else { KASSERT(results, ("auth validated but no result")); - if (ext) { - if ((results->m_flags & M_NOMAP) != - 0) - ext->rc_mbufoffs = - xdrs.x_handy; - else - ext->rc_mbufoffs = 0; - } *resultsp = results; } } /* end successful completion */ Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Wed May 13 00:03:39 2020 (r360997) @@ -86,7 +86,6 @@ __FBSDID("$FreeBSD$"); #include #include - struct cmessage { struct cmsghdr cmsg; struct cmsgcred cmcred; @@ -535,19 +534,6 @@ got_reply: if (ext && ext->rc_feedback) ext->rc_feedback(FEEDBACK_OK, proc, ext->rc_feedback_arg); -#ifdef notnow -{ struct mbuf *m, *m2; -int txxxx; -if (cr->cr_mrep != NULL) { -txxxx = m_length(cr->cr_mrep, NULL); -if (txxxx > 0) { -m = mb_copym_ext_pgs(cr->cr_mrep, txxxx, 16384, M_WAITOK, - mb_free_mext_pgs, &m2); -m2 = cr->cr_mrep; -cr->cr_mrep = m; -m_freem(m2); -} } } -#endif xdrmbuf_create(&xdrs, cr->cr_mrep, XDR_DECODE); ok = xdr_replymsg(&xdrs, &reply_msg); cr->cr_mrep = NULL; @@ -569,14 +555,6 @@ m_freem(m2); } else { KASSERT(results, ("auth validated but no result")); - if (ext) { - if ((results->m_flags & M_NOMAP) != - 0) - ext->rc_mbufoffs = - xdrs.x_handy; - else - ext->rc_mbufoffs = 0; - } *resultsp = results; } } /* end successful completion */ @@ -932,7 +910,11 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai CTASSERT(sizeof(xid_plus_direction) == 2 * sizeof(uint32_t)); - /* RPC-over-TLS needs to block reception during handshake upcall. */ + /* + * RPC-over-TLS needs to block reception during + * upcalls since the upcall will be doing I/O on + * the socket via openssl library calls. + */ mtx_lock(&ct->ct_lock); if (ct->ct_dontrcv) { mtx_unlock(&ct->ct_lock); @@ -942,15 +924,13 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai /* * If another thread is already here, it must be in - * soreceive(), so just return. + * soreceive(), so just return to avoid races with it. * ct_upcallrefs is protected by the SOCKBUF_LOCK(), * which is held in this function, except when * soreceive() is called. */ if (ct->ct_upcallrefs > 0) -{ printf("soup another\n"); return (SU_OK); -} ct->ct_upcallrefs++; /* @@ -987,6 +967,7 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai } if (error != 0) { wakeup_all: +printf("wakeup_all err=%d\n", error); mtx_lock(&ct->ct_lock); ct->ct_error.re_status = RPC_CANTRECV; ct->ct_error.re_errno = error; Modified: projects/nfs-over-tls/sys/rpc/svc.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc.h Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/rpc/svc.h Wed May 13 00:03:39 2020 (r360997) @@ -175,7 +175,6 @@ typedef struct __rpc_svcxprt { int xp_upcallset; /* socket upcall is set up */ uint32_t xp_snd_cnt; /* # of bytes to send to socket */ uint32_t xp_snt_cnt; /* # of bytes sent to socket */ - u_int xp_mbufoffs; /* Offset into ext_pgs mbuf */ bool_t xp_dontrcv; /* Do not receive on the socket */ uint32_t xp_tls; /* RPC-over-TLS on socket */ uint64_t xp_sslsec; /* Userland SSL * */ Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc_vc.c Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/rpc/svc_vc.c Wed May 13 00:03:39 2020 (r360997) @@ -76,7 +76,6 @@ __FBSDID("$FreeBSD$"); #include - static bool_t svc_vc_rendezvous_recv(SVCXPRT *, struct rpc_msg *, struct sockaddr **, struct mbuf **); static enum xprt_stat svc_vc_rendezvous_stat(SVCXPRT *); @@ -593,32 +592,6 @@ svc_vc_process_pending(SVCXPRT *xprt) struct socket *so = xprt->xp_socket; struct mbuf *m; -#ifdef notnow -{ struct mbuf *m1, *m2, *m3, *m4; - int txxxx; - m3 = cd->mpending; - m4 = NULL; - while (m3 != NULL && (m3->m_flags & M_NOMAP) != 0) { - m4 = m3; - m3 = m3->m_next; - } - if (m3 != NULL) { - txxxx = m_length(m3, NULL); - if (txxxx > 0) { - m1 = mb_copym_ext_pgs(m3, txxxx, 16384, M_WAITOK, - mb_free_mext_pgs, &m2); - if (m4 != NULL) { - m4->m_next = m1; - m_freem(m3); - } else { - m2 = cd->mpending; - cd->mpending = m1; - m_freem(m2); - } - } - } -} -#endif /* * If cd->resid is non-zero, we have part of the * record already, otherwise we are expecting a record @@ -648,7 +621,7 @@ svc_vc_process_pending(SVCXPRT *xprt) header = ntohl(header); cd->eor = (header & 0x80000000) != 0; cd->resid = header & 0x7fffffff; - cd->resid += sizeof(uint32_t); + m_adj(cd->mpending, sizeof(uint32_t)); } /* @@ -661,14 +634,10 @@ svc_vc_process_pending(SVCXPRT *xprt) while (cd->mpending && cd->resid) { m = cd->mpending; if (cd->mpending->m_next - || cd->mpending->m_len > cd->resid) { - if ((cd->mpending->m_flags & M_NOMAP) != 0) - cd->mpending = mb_splitatpos_ext( - cd->mpending, cd->resid, M_WAITOK); - else - cd->mpending = m_split(cd->mpending, - cd->resid, M_WAITOK); - } else + || cd->mpending->m_len > cd->resid) + cd->mpending = m_split(cd->mpending, + cd->resid, M_WAITOK); + else cd->mpending = NULL; if (cd->mreq) m_last(cd->mreq)->m_next = m; @@ -702,7 +671,7 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, struct socket* so = xprt->xp_socket; XDR xdrs; int error, rcvflag; - uint32_t xid_plus_direction[3], junk; + uint32_t xid_plus_direction[2]; struct cmsghdr *cmsg; struct tls_get_record tgr; @@ -735,15 +704,15 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, m_copydata(cd->mreq, 0, sizeof(xid_plus_direction), (char *)xid_plus_direction); + xid_plus_direction[0] = + ntohl(xid_plus_direction[0]); xid_plus_direction[1] = ntohl(xid_plus_direction[1]); - xid_plus_direction[2] = - ntohl(xid_plus_direction[2]); /* Check message direction. */ - if (xid_plus_direction[2] == REPLY) { + if (xid_plus_direction[1] == REPLY) { clnt_bck_svccall(xprt->xp_p2, cd->mreq, - xid_plus_direction[1]); + xid_plus_direction[0]); cd->mreq = NULL; continue; } @@ -763,18 +732,13 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, sx_xunlock(&xprt->xp_lock); - if (! xdr_uint32_t(&xdrs, &junk) || - ! xdr_callmsg(&xdrs, msg)) { + if (! xdr_callmsg(&xdrs, msg)) { XDR_DESTROY(&xdrs); return (FALSE); } *addrp = NULL; *mp = xdrmbuf_getall(&xdrs); - if (((*mp)->m_flags & M_NOMAP) != 0) - xprt->xp_mbufoffs = xdrs.x_handy; - else - xprt->xp_mbufoffs = 0; XDR_DESTROY(&xdrs); return (TRUE); @@ -901,19 +865,6 @@ svc_vc_backchannel_recv(SVCXPRT *xprt, struct rpc_msg sx_xunlock(&xprt->xp_lock); printf("recv backch m=%p\n", m); -#ifdef notnow -{ struct mbuf *m1, *m2; -int txxxx; -if (m != NULL) { -txxxx = m_length(m, NULL); -if (txxxx > 0) { -m1 = mb_copym_ext_pgs(m, txxxx, 16384, M_WAITOK, - mb_free_mext_pgs, &m2); -m2 = m; -m = m1; -m_freem(m2); -} } } -#endif xdrmbuf_create(&xdrs, m, XDR_DECODE); if (! xdr_callmsg(&xdrs, msg)) { printf("recv backch callmsg failed\n"); @@ -922,11 +873,6 @@ printf("recv backch callmsg failed\n"); } *addrp = NULL; *mp = xdrmbuf_getall(&xdrs); - if (((*mp)->m_flags & M_NOMAP) != 0) - xprt->xp_mbufoffs = xdrs.x_handy; - else - xprt->xp_mbufoffs = 0; -printf("backch offs=%d\n", xprt->xp_mbufoffs); XDR_DESTROY(&xdrs); return (TRUE); } Modified: projects/nfs-over-tls/sys/xdr/xdr_mbuf.c ============================================================================== --- projects/nfs-over-tls/sys/xdr/xdr_mbuf.c Tue May 12 23:51:04 2020 (r360996) +++ projects/nfs-over-tls/sys/xdr/xdr_mbuf.c Wed May 13 00:03:39 2020 (r360997) @@ -122,10 +122,9 @@ xdrmbuf_getall(XDR *xdrs) xdrs->x_private = NULL; } - if (m) { - if ((m->m_flags & M_NOMAP) == 0) - m_adj(m, xdrs->x_handy); - } else + if (m) + m_adj(m, xdrs->x_handy); + else m = m_get(M_WAITOK, MT_DATA); return (m); } @@ -195,10 +194,7 @@ xdrmbuf_getbytes(XDR *xdrs, char *addr, u_int len) sz = m->m_len - xdrs->x_handy; if (sz > len) sz = len; - if ((m->m_flags & M_NOMAP) != 0) - m_copydata(m, xdrs->x_handy, sz, addr); - else - bcopy(mtod(m, const char *) + xdrs->x_handy, addr, sz); + bcopy(mtod(m, const char *) + xdrs->x_handy, addr, sz); addr += sz; xdrs->x_handy += sz; @@ -290,8 +286,6 @@ xdrmbuf_inline(XDR *xdrs, u_int len) char *p; if (!m) - return (0); - if ((m->m_flags & M_NOMAP) != 0) return (0); if (xdrs->x_op == XDR_ENCODE) { available = M_TRAILINGSPACE(m) + (m->m_len - xdrs->x_handy); From owner-svn-src-projects@freebsd.org Thu May 14 21:46:14 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8899F2E1305 for ; Thu, 14 May 2020 21:46:14 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49NQ8k3Pr0z40Hm; Thu, 14 May 2020 21:46:14 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5779019C41; Thu, 14 May 2020 21:46:14 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04ELkE3J027155; Thu, 14 May 2020 21:46:14 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04ELkCmN027147; Thu, 14 May 2020 21:46:12 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005142146.04ELkCmN027147@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Thu, 14 May 2020 21:46:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r361062 - in projects/nfs-over-tls: sys/kern sys/rpc sys/rpc/rpcsec_tls usr.sbin/rpctlscd usr.sbin/rpctlssd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls: sys/kern sys/rpc sys/rpc/rpcsec_tls usr.sbin/rpctlscd usr.sbin/rpctlssd X-SVN-Commit-Revision: 361062 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2020 21:46:14 -0000 Author: rmacklem Date: Thu May 14 21:46:12 2020 New Revision: 361062 URL: https://svnweb.freebsd.org/changeset/base/361062 Log: Add code that does upcalls to the daemons when an non-application data record is at the top of the socket receive queue. Define a new socket flag MSG_TLSAPPDATA which tells soreceive_generic() to return ENXIO if the first record in the receive queue is not an application data record. ENXIO then triggers svc_vc_recv() to do an upcall to the rpctlssd daemon, so the daemon can do a SSL_read() for 0 bytes to handle the record. This patch adds some similar code to the client side rpctlscd daemon, but the kernel code to do the upcall still needs to be written. For the client side, I do not currently know how to test it, because it is in disconnect when the "close alert" is sent via SSL_shutdown(). Testing of this client code may need to wait until TLS1.3 support is in the KERN_TLS. Modified: projects/nfs-over-tls/sys/kern/uipc_socket.c projects/nfs-over-tls/sys/rpc/rpcsec_tls.h projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlscd.x projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlssd.x projects/nfs-over-tls/sys/rpc/svc_vc.c projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Modified: projects/nfs-over-tls/sys/kern/uipc_socket.c ============================================================================== --- projects/nfs-over-tls/sys/kern/uipc_socket.c Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/sys/kern/uipc_socket.c Thu May 14 21:46:12 2020 (r361062) @@ -2052,6 +2052,31 @@ dontblock: if (m != NULL && m->m_type == MT_CONTROL) { struct mbuf *cm = NULL, *cmn; struct mbuf **cme = &cm; + struct cmsghdr *cmsg; + struct tls_get_record tgr; + + /* + * For MSG_TLSAPPDATA, check for a non-application data + * record. If found, then return ENXIO without removing + * it from the receive queue. This allows a subsequent + * call without MSG_TLSAPPDATA to receive it. + * Note that, for TLS, there should only be a single + * control mbuf with the TLS_GET_RECORD message in it. + */ + if (flags & MSG_TLSAPPDATA) { + cmsg = mtod(m, struct cmsghdr *); + if (cmsg->cmsg_type == TLS_GET_RECORD && + cmsg->cmsg_len == CMSG_LEN(sizeof(tgr))) { + memcpy(&tgr, CMSG_DATA(cmsg), sizeof(tgr)); + /* This will need to change for TLS 1.3. */ + if (tgr.tls_type != TLS_RLTYPE_APP) { + SOCKBUF_UNLOCK(&so->so_rcv); +printf("fnd non-app rec=%d\n", tgr.tls_type); + error = ENXIO; + goto release; + } + } + } do { if (flags & MSG_PEEK) { Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Thu May 14 21:46:12 2020 (r361062) @@ -47,6 +47,10 @@ /* Functions that perform upcalls to the rpctlsd daemon. */ enum clnt_stat rpctls_connect(CLIENT *newclient, struct socket *so, uint64_t *sslp); +enum clnt_stat rpctls_cl_handlerecord(uint64_t sec, uint64_t usec, + uint64_t ssl); +enum clnt_stat rpctls_srv_handlerecord(uint64_t sec, uint64_t usec, + uint64_t ssl); enum clnt_stat rpctls_cl_disconnect(uint64_t sec, uint64_t usec, uint64_t ssl); enum clnt_stat rpctls_srv_disconnect(uint64_t sec, uint64_t usec, uint64_t ssl); Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Thu May 14 21:46:12 2020 (r361062) @@ -385,6 +385,53 @@ printf("aft wakeup\n"); return (stat); } +/* Do an upcall to handle an non-application data record using TLS. */ +enum clnt_stat +rpctls_cl_handlerecord(uint64_t sec, uint64_t usec, uint64_t ssl) +{ + struct rpctlscd_handlerecord_arg arg; + enum clnt_stat stat; + CLIENT *cl; + +printf("In rpctls_cl_handlerecord\n"); + cl = rpctls_connect_client(); +printf("handlerecord_client=%p\n", cl); + if (cl == NULL) + return (RPC_FAILED); + + /* Do the handlerecord upcall. */ + arg.sec = sec; + arg.usec = usec; + arg.ssl = ssl; + stat = rpctlscd_handlerecord_1(&arg, NULL, cl); +printf("aft handlerecord upcall=%d\n", stat); + CLNT_RELEASE(cl); + return (stat); +} + +enum clnt_stat +rpctls_srv_handlerecord(uint64_t sec, uint64_t usec, uint64_t ssl) +{ + struct rpctlssd_handlerecord_arg arg; + enum clnt_stat stat; + CLIENT *cl; + +printf("In rpctls_srv_handlerecord\n"); + cl = rpctls_server_client(); +printf("srv handlerecord_client=%p\n", cl); + if (cl == NULL) + return (RPC_FAILED); + + /* Do the handlerecord upcall. */ + arg.sec = sec; + arg.usec = usec; + arg.ssl = ssl; + stat = rpctlssd_handlerecord_1(&arg, NULL, cl); +printf("aft srv handlerecord upcall=%d\n", stat); + CLNT_RELEASE(cl); + return (stat); +} + /* Do an upcall to shut down a socket using TLS. */ enum clnt_stat rpctls_cl_disconnect(uint64_t sec, uint64_t usec, uint64_t ssl) Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlscd.x ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlscd.x Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlscd.x Thu May 14 21:46:12 2020 (r361062) @@ -35,6 +35,12 @@ struct rpctlscd_connect_res { uint64_t ssl; }; +struct rpctlscd_handlerecord_arg { + uint64_t sec; + uint64_t usec; + uint64_t ssl; +}; + struct rpctlscd_disconnect_arg { uint64_t sec; uint64_t usec; @@ -48,6 +54,8 @@ program RPCTLSCD { rpctlscd_connect_res RPCTLSCD_CONNECT(void) = 1; - void RPCTLSCD_DISCONNECT(rpctlscd_disconnect_arg) = 2; + void RPCTLSCD_HANDLERECORD(rpctlscd_handlerecord_arg) = 2; + + void RPCTLSCD_DISCONNECT(rpctlscd_disconnect_arg) = 3; } = 1; } = 0x40677374; Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlssd.x ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlssd.x Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctlssd.x Thu May 14 21:46:12 2020 (r361062) @@ -38,6 +38,12 @@ struct rpctlssd_connect_res { uint32_t gid<>; }; +struct rpctlssd_handlerecord_arg { + uint64_t sec; + uint64_t usec; + uint64_t ssl; +}; + struct rpctlssd_disconnect_arg { uint64_t sec; uint64_t usec; @@ -51,6 +57,8 @@ program RPCTLSSD { rpctlssd_connect_res RPCTLSSD_CONNECT(void) = 1; - void RPCTLSSD_DISCONNECT(rpctlssd_disconnect_arg) = 2; + void RPCTLSSD_HANDLERECORD(rpctlssd_handlerecord_arg) = 2; + + void RPCTLSSD_DISCONNECT(rpctlssd_disconnect_arg) = 3; } = 1; } = 0x40677375; Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc_vc.c Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/sys/rpc/svc_vc.c Thu May 14 21:46:12 2020 (r361062) @@ -674,6 +674,7 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, uint32_t xid_plus_direction[2]; struct cmsghdr *cmsg; struct tls_get_record tgr; + enum clnt_stat ret; /* * Serialise access to the socket and our own record parsing @@ -748,6 +749,9 @@ svc_vc_recv(SVCXPRT *xprt, struct rpc_msg *msg, * If receiving is disabled so that a TLS handshake can be * done by the rpctlssd daemon, return FALSE here. */ + rcvflag = MSG_DONTWAIT; + if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0) + rcvflag |= MSG_TLSAPPDATA; tryagain: if (xprt->xp_dontrcv) { sx_xunlock(&xprt->xp_lock); @@ -765,7 +769,6 @@ tryagain: uio.uio_resid = 1000000000; uio.uio_td = curthread; ctrl = m = NULL; - rcvflag = MSG_DONTWAIT; error = soreceive(so, NULL, &uio, &m, &ctrl, &rcvflag); if (error == EWOULDBLOCK) { @@ -784,6 +787,36 @@ tryagain: return (FALSE); } + /* + * A return of ENXIO indicates that there is a + * non-application data record at the head of the + * socket's receive queue, for TLS connections. + * This record needs to be handled in userland + * via an SSL_read() call, so do an upcall to the daemon. + */ + if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0 && + error == ENXIO) { + /* Disable reception. */ + xprt->xp_dontrcv = TRUE; + sx_xunlock(&xprt->xp_lock); +printf("Call rpctls_srv_handlerecord\n"); + ret = rpctls_srv_handlerecord(xprt->xp_sslsec, + xprt->xp_sslusec, xprt->xp_sslrefno); + sx_xlock(&xprt->xp_lock); + xprt->xp_dontrcv = FALSE; + if (ret != RPC_SUCCESS) { + /* + * All we can do is soreceive() it and + * then toss it. + */ + rcvflag = MSG_DONTWAIT; + goto tryagain; + } + sx_xunlock(&xprt->xp_lock); + xprt_active(xprt); /* Harmless if already active. */ + return (FALSE); + } + if (error) { SOCKBUF_LOCK(&so->so_rcv); if (xprt->xp_upcallset) { @@ -815,15 +848,16 @@ if (ctrl->m_next != NULL) printf("EEK! svc list of con cmsg->cmsg_len == CMSG_LEN(sizeof(tgr))) { memcpy(&tgr, CMSG_DATA(cmsg), sizeof(tgr)); /* - * For now, just toss non-application - * data records. - * In the future, there may need to be - * an upcall done to the daemon. + * This should have been handled by + * the rpctls_svc_handlerecord() + * upcall. If not, all we can do is + * toss it away. */ if (tgr.tls_type != TLS_RLTYPE_APP) { printf("Got weird type=%d\n", tgr.tls_type); m_freem(m); m_free(ctrl); + rcvflag = MSG_DONTWAIT | MSG_TLSAPPDATA; goto tryagain; } } Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Thu May 14 21:46:12 2020 (r361062) @@ -371,10 +371,47 @@ rpctlscd_verbose_out("rpctlsd_connect s=%d\n", s); } bool_t +rpctlscd_handlerecord_1_svc(struct rpctlscd_handlerecord_arg *argp, + void *result, struct svc_req *rqstp) +{ + struct ssl_entry *slp; + int ret; + char junk; + + slp = NULL; + if (argp->sec == rpctls_ssl_sec && argp->usec == + rpctls_ssl_usec) { + LIST_FOREACH(slp, &rpctls_ssllist, next) { + if (slp->refno == argp->ssl) + break; + } + } + + if (slp != NULL) { + rpctlscd_verbose_out("rpctlscd_handlerecord fd=%d\n", + slp->s); + /* + * An SSL_read() of 0 bytes should fail, but it should + * handle the non-application data record before doing so. + */ + ret = SSL_read(slp->ssl, &junk, 0); + if (ret > 0) { + if (rpctls_debug_level == 0) + syslog(LOG_ERR, "SSL_read returned %d", ret); + else + fprintf(stderr, "SSL_read returned %d\n", ret); + } + } else + return (FALSE); + return (TRUE); +} + +bool_t rpctlscd_disconnect_1_svc(struct rpctlscd_disconnect_arg *argp, void *result, struct svc_req *rqstp) { struct ssl_entry *slp; + int ret; slp = NULL; if (argp->sec == rpctls_ssl_sec && argp->usec == @@ -389,6 +426,13 @@ rpctlscd_disconnect_1_svc(struct rpctlscd_disconnect_a rpctlscd_verbose_out("rpctlscd_disconnect: fd=%d closed\n", slp->s); LIST_REMOVE(slp, next); + SSL_shutdown(slp->ssl); + /* Check to see if the peer has sent a close alert. */ + ret = SSL_get_shutdown(slp->ssl); +rpctlscd_verbose_out("get_shutdown=%d\n", ret); + if ((ret & (SSL_SENT_SHUTDOWN | SSL_RECEIVED_SHUTDOWN)) == + SSL_SENT_SHUTDOWN) + SSL_shutdown(slp->ssl); SSL_free(slp->ssl); /* * For RPC-over-TLS, this upcall is expected Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Thu May 14 21:36:59 2020 (r361061) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Thu May 14 21:46:12 2020 (r361062) @@ -419,6 +419,49 @@ rpctlssd_verbose_out("rpctlsd_connect_svc s=%d\n", s); } bool_t +rpctlssd_handlerecord_1_svc(struct rpctlssd_handlerecord_arg *argp, + void *result, struct svc_req *rqstp) +{ + struct ssl_entry *slp; + int ret; + char junk; + + slp = NULL; + if (argp->sec == rpctls_ssl_sec && argp->usec == + rpctls_ssl_usec) { + LIST_FOREACH(slp, &rpctls_ssllist, next) { + if (slp->refno == argp->ssl) + break; + } + } + + if (slp != NULL) { + rpctlssd_verbose_out("rpctlssd_handlerecord fd=%d\n", + slp->s); + /* + * An SSL_read() of 0 bytes should fail, but it should + * handle the non-application data record before doing so. + */ + ret = SSL_read(slp->ssl, &junk, 0); + if (ret <= 0) { + /* Check to see if this was a close alert. */ + ret = SSL_get_shutdown(slp->ssl); +rpctlssd_verbose_out("get_shutdown=%d\n", ret); + if ((ret & (SSL_SENT_SHUTDOWN | + SSL_RECEIVED_SHUTDOWN)) == SSL_RECEIVED_SHUTDOWN) + SSL_shutdown(slp->ssl); + } else { + if (rpctls_debug_level == 0) + syslog(LOG_ERR, "SSL_read returned %d", ret); + else + fprintf(stderr, "SSL_read returned %d\n", ret); + } + } else + return (FALSE); + return (TRUE); +} + +bool_t rpctlssd_disconnect_1_svc(struct rpctlssd_disconnect_arg *argp, void *result, struct svc_req *rqstp) { @@ -442,6 +485,7 @@ rpctlssd_disconnect_1_svc(struct rpctlssd_disconnect_a * For RPC-over-TLS, this upcall is expected * to close off the socket. */ + shutdown(slp->s, SHUT_WR); close(slp->s); free(slp); } else @@ -679,7 +723,7 @@ rpctlssd_verbose_out("%s\n", cp2); } if (ret == 0) { if (rpctls_debug_level == 0) - syslog(LOG_ERR, "ktls not working\n"); + syslog(LOG_ERR, "ktls not working"); else fprintf(stderr, "ktls not working\n"); /* From owner-svn-src-projects@freebsd.org Sat May 16 02:34:57 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 563192E10F6 for ; Sat, 16 May 2020 02:34:57 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49P8WP1Z7lz3HlV; Sat, 16 May 2020 02:34:57 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 312CC6A6A; Sat, 16 May 2020 02:34:57 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04G2YvgK099464; Sat, 16 May 2020 02:34:57 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04G2Yv1t099463; Sat, 16 May 2020 02:34:57 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005160234.04G2Yv1t099463@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 16 May 2020 02:34:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r361100 - projects/nfs-over-tls/sys/rpc/rpcsec_tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc/rpcsec_tls X-SVN-Commit-Revision: 361100 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2020 02:34:57 -0000 Author: rmacklem Date: Sat May 16 02:34:56 2020 New Revision: 361100 URL: https://svnweb.freebsd.org/changeset/base/361100 Log: Add fdrop()s to the rpctls syscall for the "fp" reference. Without these fdrop()s, the daemons leaked file descriptors and it also left closed TCP sockets lying around. I didn't realize that falloc() returns with two references, one for the "fd" and one for the "fp". Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Sat May 16 02:29:10 2020 (r361099) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Sat May 16 02:34:56 2020 (r361100) @@ -240,6 +240,7 @@ printf("falloc=%d fd=%d\n", error, fd); rpctls_connect_so = NULL; mtx_unlock(&rpctls_connect_lock); finit(fp, FREAD | FWRITE, DTYPE_SOCKET, so, &socketops); + fdrop(fp, td); /* Drop fp reference. */ td->td_retval[0] = fd; } printf("returning=%d\n", fd); @@ -260,6 +261,7 @@ printf("srv falloc=%d fd=%d\n", error, fd); rpctls_server_so = NULL; mtx_unlock(&rpctls_server_lock); finit(fp, FREAD | FWRITE, DTYPE_SOCKET, so, &socketops); + fdrop(fp, td); /* Drop fp reference. */ td->td_retval[0] = fd; } printf("srv returning=%d\n", fd); From owner-svn-src-projects@freebsd.org Sat May 16 02:43:28 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFAD12E149E for ; Sat, 16 May 2020 02:43:28 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49P8jD4f7jz3J9Q; Sat, 16 May 2020 02:43:28 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9AA7B6C57; Sat, 16 May 2020 02:43:28 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04G2hSXH006011; Sat, 16 May 2020 02:43:28 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04G2hSIn006010; Sat, 16 May 2020 02:43:28 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005160243.04G2hSIn006010@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 16 May 2020 02:43:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r361101 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 361101 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2020 02:43:28 -0000 Author: rmacklem Date: Sat May 16 02:43:27 2020 New Revision: 361101 URL: https://svnweb.freebsd.org/changeset/base/361101 Log: Add support for doing upcalls to the rpctlscd daemon to the krpc client when TLS non-application data records are received. This is similar to code added to the krpc server side. However, since soreceive() is called in a socket upcall where it cannot sleep, the code needed to get a thread that is in clnt_vc_call() to do the upcall. The ct_dontrcv boolean was changed to a 5 state variable to indicate when/if an upcall is done. This code hasn't really been tested, since I don't know how to get TLS1.2 to put a non-application data record in the stream, except when doing SSL_shutdown() (a close alert) and this is initiated by the client when the socket is closing. It should get tested when TLS1.3 is available. Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c projects/nfs-over-tls/sys/rpc/krpc.h Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Sat May 16 02:34:56 2020 (r361100) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Sat May 16 02:43:27 2020 (r361101) @@ -101,6 +101,7 @@ static void clnt_vc_close(CLIENT *); static void clnt_vc_destroy(CLIENT *); static bool_t time_not_ok(struct timeval *); static int clnt_vc_soupcall(struct socket *so, void *arg, int waitflag); +static void clnt_vc_dotlsupcall(struct ct_data *ct); static struct clnt_ops clnt_vc_ops = { .cl_call = clnt_vc_call, @@ -158,6 +159,7 @@ clnt_vc_create( ct->ct_closing = FALSE; ct->ct_closed = FALSE; ct->ct_upcallrefs = 0; + ct->ct_rcvstate = RCVNORMAL; if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) { error = soconnect(so, raddr, curthread); @@ -414,6 +416,18 @@ call_again: stat = RPC_CANTRECV; goto out; } + + /* + * For TLS, do an upcall, as required. + * clnt_vc_dotlsupcall() will just return unless an + * an upcall is needed. + * Wait until any upcall is completed. + */ + clnt_vc_dotlsupcall(ct); + while (ct->ct_rcvstate != RCVNORMAL && + ct->ct_rcvstate != RCVNONAPPDATA) + msleep(&ct->ct_rcvstate, &ct->ct_lock, 0, "rpcrcvst", hz); + TAILQ_INSERT_TAIL(&ct->ct_pending, cr, cr_link); mtx_unlock(&ct->ct_lock); @@ -487,8 +501,30 @@ printf("TRY AGAIN!!\n"); goto out; } - error = msleep(cr, &ct->ct_lock, ct->ct_waitflag, ct->ct_waitchan, - tvtohz(&timeout)); + /* + * For TLS, msleep() can be awakened to handle + * an upcall via a call to clnt_vc_dotlsupcall(). + * If there was no error, it needs to loop + * around and wait for the reply. + */ + do { + /* + * Call clnt_vc_dotlsupcall() both before + * and after the msleep(). If there is + * no upcall to do, clnt_vc_dotlsupcall() + * simply returns. + * Call before msleep() in case ct_rcvstate + * is already set to UPCALLNEEDED and the + * wakeup(ct) has already been done. + * Call after msleep() in case it has its + * reply and will not be looping. + */ + clnt_vc_dotlsupcall(ct); + error = msleep(cr, &ct->ct_lock, ct->ct_waitflag, + ct->ct_waitchan, tvtohz(&timeout)); + clnt_vc_dotlsupcall(ct); + } while (cr->cr_mrep == NULL && error == 0 && + cr->cr_error == 0); TAILQ_REMOVE(&ct->ct_pending, cr, cr_link); @@ -767,9 +803,9 @@ printf("backch tls=0x%x xprt=%p\n", xprt->xp_tls, xprt case CLSET_BLOCKRCV: if (*(int *) info) - ct->ct_dontrcv = TRUE; + ct->ct_rcvstate = TLSHANDSHAKE; else - ct->ct_dontrcv = FALSE; + ct->ct_rcvstate = RCVNORMAL; break; default: @@ -898,7 +934,7 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai { struct ct_data *ct = (struct ct_data *) arg; struct uio uio; - struct mbuf *m, *m2; + struct mbuf *m, *m2, **ctrlp; struct ct_request *cr; int error, rcvflag, foundreq; uint32_t xid_plus_direction[2], header; @@ -916,7 +952,8 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai * the socket via openssl library calls. */ mtx_lock(&ct->ct_lock); - if (ct->ct_dontrcv) { + if (ct->ct_rcvstate != RCVNORMAL && ct->ct_rcvstate != + RCVNONAPPDATA) { mtx_unlock(&ct->ct_lock); return (SU_OK); } @@ -942,8 +979,13 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai uio.uio_td = curthread; m2 = m = NULL; rcvflag = MSG_DONTWAIT | MSG_SOCALLBCK; + if (ct->ct_sslrefno != 0 && ct->ct_rcvstate == RCVNORMAL) { + rcvflag |= MSG_TLSAPPDATA; + ctrlp = NULL; + } else + ctrlp = &m2; SOCKBUF_UNLOCK(&so->so_rcv); - error = soreceive(so, NULL, &uio, &m, &m2, &rcvflag); + error = soreceive(so, NULL, &uio, &m, ctrlp, &rcvflag); SOCKBUF_LOCK(&so->so_rcv); if (error == EWOULDBLOCK) { @@ -965,6 +1007,30 @@ clnt_vc_soupcall(struct socket *so, void *arg, int wai */ error = ECONNRESET; } + + /* + * A return of ENXIO indicates that there is a + * non-application data record at the head of the + * socket's receive queue, for TLS connections. + * This record needs to be handled in userland + * via an SSL_read() call, so do an upcall to the daemon. + */ + if (ct->ct_sslrefno != 0 && error == ENXIO) { + /* Disable reception, marking an upcall needed. */ + mtx_lock(&ct->ct_lock); + ct->ct_rcvstate = UPCALLNEEDED; + /* + * If an upcall in needed, wake up all thread(s) + * in clnt_vc_call() so that one of them can do it. + * Not efficient, but this should not happen + * frequently. + */ + TAILQ_FOREACH(cr, &ct->ct_pending, cr_link) + wakeup(cr); + mtx_unlock(&ct->ct_lock); +printf("Mark upcallneeded\n"); + break; + } if (error != 0) { wakeup_all: printf("wakeup_all err=%d\n", error); @@ -987,16 +1053,18 @@ if (m2->m_next != NULL) printf("EEK! list of controls\ cmsg->cmsg_len == CMSG_LEN(sizeof(tgr))) { memcpy(&tgr, CMSG_DATA(cmsg), sizeof(tgr)); /* - * For now, just toss non-application - * data records. - * In the future, there may need to be - * an upcall done to the daemon, but - * it cannot be done here. + * This should have been handled by + * setting ct_rcvstate == UPCALLNEEDED, + * but if not, all we can do is toss + * it away. */ if (tgr.tls_type != TLS_RLTYPE_APP) { printf("Got weird type=%d\n", tgr.tls_type); m_freem(m); m_free(m2); + mtx_lock(&ct->ct_lock); + ct->ct_rcvstate = RCVNORMAL; + mtx_unlock(&ct->ct_lock); continue; } } @@ -1206,4 +1274,27 @@ clnt_vc_upcallsdone(struct ct_data *ct) while (ct->ct_upcallrefs > 0) (void) msleep(&ct->ct_upcallrefs, SOCKBUF_MTX(&ct->ct_socket->so_rcv), 0, "rpcvcup", 0); +} + +/* + * Do a TLS upcall to the rpctlscd daemon, as required. + */ +static void +clnt_vc_dotlsupcall(struct ct_data *ct) +{ + enum clnt_stat ret; + + mtx_assert(&ct->ct_lock, MA_OWNED); + if (ct->ct_rcvstate == UPCALLNEEDED) { + ct->ct_rcvstate = UPCALLINPROG; + mtx_unlock(&ct->ct_lock); + ret = rpctls_cl_handlerecord(ct->ct_sslsec, ct->ct_sslusec, + ct->ct_sslrefno); + mtx_lock(&ct->ct_lock); + if (ret == RPC_SUCCESS) + ct->ct_rcvstate = RCVNORMAL; + else + ct->ct_rcvstate = RCVNONAPPDATA; + wakeup(&ct->ct_rcvstate); + } } Modified: projects/nfs-over-tls/sys/rpc/krpc.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/krpc.h Sat May 16 02:34:56 2020 (r361100) +++ projects/nfs-over-tls/sys/rpc/krpc.h Sat May 16 02:43:27 2020 (r361101) @@ -82,6 +82,14 @@ struct rc_data { bool rc_tls; /* Enable TLS on connection */ }; +enum clnt_rcvstate { + RCVNORMAL = 0, /* Normal reception. */ + RCVNONAPPDATA = 1, /* Reception of a non-application record. */ + TLSHANDSHAKE = 2, /* Reception blocked for TLS handshake. */ + UPCALLNEEDED = 3, /* Upcall to rpctlscd needed. */ + UPCALLINPROG = 4 /* Upcall to rpctlscd in progress. */ +}; + struct ct_data { struct mtx ct_lock; int ct_threads; /* number of threads in clnt_vc_call */ @@ -106,7 +114,7 @@ struct ct_data { uint64_t ct_sslsec; /* RPC-over-TLS connection. */ uint64_t ct_sslusec; uint64_t ct_sslrefno; - bool_t ct_dontrcv; /* TRUE to block receiving */ + enum clnt_rcvstate ct_rcvstate; /* Block receiving for TLS upcalls */ struct mbuf *ct_raw; /* Raw mbufs recv'd */ }; From owner-svn-src-projects@freebsd.org Sat May 16 05:26:17 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 81FAA2E5C27 for ; Sat, 16 May 2020 05:26:17 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49PDK52M3Wz3xXk; Sat, 16 May 2020 05:26:17 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: by mail-ot1-x32f.google.com with SMTP id t3so3743877otp.3; Fri, 15 May 2020 22:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tJm0cgR8WvJ/FXjbM74M5PZ3RYFb1xVp1C7SLRHlSpM=; b=jt3CBhCnDBQzu/TJy3oyJzOijwnC1zNhRR9bD/aAOD+4SinRmd849HFBkRA87Ay3lC tk7dVqURPbJYDMp47acLsdeLzj7qbnuVNlmYX459x5pEh4OozPw9n0FPUCDfO4f5xzmS TJnIgRvBwJ/DwF3a52767+/mw/uxdSIRo7Ry1jsVmCqNqq+BNP3WdcxqJjDl2YCrhJ1U xAJqpySQSAjxBPMzZIdVfkNXyFupKmxgiQhntoREm+tezqQHufgUZ1h9NBXS5dljTHUo rTF8oDpanDZccRT/TMYjxXc06CuvRy3ybficXQy2QMAHOx1JZXeXzdpnL6yoSaf2nL2H N8ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tJm0cgR8WvJ/FXjbM74M5PZ3RYFb1xVp1C7SLRHlSpM=; b=ia/ZTjcgt/DgMBe8QGzWKkx6rpwnMqSVEKYIH2GKaVOvVk6bRh+c+q8Ecd54nqA7L/ nzPWs0HYW3qxSLXkL7TRHs7bgSnnSPtKYGKQ0SEly75fvxP+Xq8vUah83tYPWKVo0IbB GWARJ0O5eVhwBQGlh0T1XSsXn2+2Yyhq/Ud3W5vTlGrInIIX04amxb0aBR0cpjiGVXK7 I6Clp1T1es9UkPlKX/Kxj17lK/4i5YsXOMqypaFisvjLXD1YfPxHncYqVmqAgL72gbMI 8MKT4EycH+hZgDd17wqdYifqK5e1ylNR9lsCHmRWlFFT4pc7e3FvD08KTo3B8IGWunTI 4ldw== X-Gm-Message-State: AOAM530GyS2PuAfhHXsK8OeErDb6YNRkAq0y+A4XWBAfiyBx9d25lvnu V1gOZdYIuPQjrSZsqpKesXmJHgnEH2/nSlkvInj5rd99 X-Google-Smtp-Source: ABdhPJxy747j9pcv4b8iPR29yOZ7za3Yw714W6aV8a8P8pYyOkf9QnCpKNFjWT/jDndmmhIN9IijEyDbwQg+L+rPlI8= X-Received: by 2002:a05:6830:1b63:: with SMTP id d3mr4521221ote.269.1589606775793; Fri, 15 May 2020 22:26:15 -0700 (PDT) MIME-Version: 1.0 References: <202005160243.04G2hSIn006010@repo.freebsd.org> In-Reply-To: <202005160243.04G2hSIn006010@repo.freebsd.org> From: Benjamin Kaduk Date: Fri, 15 May 2020 22:26:04 -0700 Message-ID: Subject: Re: svn commit: r361101 - projects/nfs-over-tls/sys/rpc To: Rick Macklem Cc: src-committers , svn-src-projects@freebsd.org X-Rspamd-Queue-Id: 49PDK52M3Wz3xXk X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2020 05:26:17 -0000 On Fri, May 15, 2020 at 7:43 PM Rick Macklem wrote: > Author: rmacklem > Date: Sat May 16 02:43:27 2020 > New Revision: 361101 > URL: https://svnweb.freebsd.org/changeset/base/361101 > > Log: > Add support for doing upcalls to the rpctlscd daemon to the krpc client > when TLS non-application data records are received. > > This is similar to code added to the krpc server side. However, since > soreceive() is called in a socket upcall where it cannot sleep, the > code needed to get a thread that is in clnt_vc_call() to do the > upcall. > The ct_dontrcv boolean was changed to a 5 state variable to indicate > when/if an upcall is done. > > This code hasn't really been tested, since I don't know how to get > TLS1.2 to put a non-application data record in the stream, except when > doing SSL_shutdown() (a close alert) and this is initiated by the > client when the socket is closing. > > A typical way to do so is to perform renegotiation (e.g., send a ClientHello in the encrypted stream). Renegotiation has some pretty weird properties to it and isn't in TLS 1.3 at all, but it should get you non-application-data records in a TLS 1.2 connection. In openssl look at the SSL_renegotiate() API. -Ben From owner-svn-src-projects@freebsd.org Sat May 16 23:24:40 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 283552D98F7 for ; Sat, 16 May 2020 23:24:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49PhFN0JQyz48nf; Sat, 16 May 2020 23:24:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 05A711DBF0; Sat, 16 May 2020 23:24:40 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04GNOdSj088377; Sat, 16 May 2020 23:24:39 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04GNOdXD088375; Sat, 16 May 2020 23:24:39 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202005162324.04GNOdXD088375@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 16 May 2020 23:24:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r361119 - projects/nfs-over-tls/sys/fs/nfs X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/fs/nfs X-SVN-Commit-Revision: 361119 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2020 23:24:40 -0000 Author: rmacklem Date: Sat May 16 23:24:39 2020 New Revision: 361119 URL: https://svnweb.freebsd.org/changeset/base/361119 Log: Make nfsm_set() a void function, since the return value is no longer used. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sat May 16 21:59:41 2020 (r361118) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Sat May 16 23:24:39 2020 (r361119) @@ -4872,7 +4872,7 @@ nfsv4_findmirror(struct nfsmount *nmp) * Fill in the fields of "struct nfsrv_descript" for a new ext_pgs mbuf. * The build argument is true for build and false for dissect. */ -int +void nfsm_set(struct nfsrv_descript *nd, u_int offs) { struct mbuf *m; @@ -4907,17 +4907,14 @@ nfsm_set(struct nfsrv_descript *nd, u_int offs) nd->nd_bpos += pgs->first_pg_off; if (offs > 0) { nd->nd_bpos += offs; - rlen = nd->nd_bextpgsiz = rlen - offs; + nd->nd_bextpgsiz = rlen - offs; } else if (nd->nd_bextpg == 0) - rlen = nd->nd_bextpgsiz = PAGE_SIZE - + nd->nd_bextpgsiz = PAGE_SIZE - pgs->first_pg_off; else - rlen = nd->nd_bextpgsiz = PAGE_SIZE; - } else { + nd->nd_bextpgsiz = PAGE_SIZE; + } else nd->nd_bpos = mtod(m, char *) + offs; - rlen = m->m_len - offs; - } - return (rlen); } /* Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_var.h ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Sat May 16 21:59:41 2020 (r361118) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_var.h Sat May 16 23:24:39 2020 (r361119) @@ -361,7 +361,7 @@ int nfsv4_sequencelookup(struct nfsmount *, struct nfs void nfsv4_freeslot(struct nfsclsession *, int); struct ucred *nfsrv_getgrpscred(struct ucred *); struct nfsdevice *nfsv4_findmirror(struct nfsmount *); -int nfsm_set(struct nfsrv_descript *, u_int); +void nfsm_set(struct nfsrv_descript *, u_int); /* nfs_clcomsubs.c */ void nfsm_uiombuf(struct nfsrv_descript *, struct uio *, int);