From owner-svn-src-projects@freebsd.org Mon Jun 29 02:32:08 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B636935BAB0 for ; Mon, 29 Jun 2020 02:32:08 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49wBMr4SgYz4BFV; Mon, 29 Jun 2020 02:32:08 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7CDE7205CA; Mon, 29 Jun 2020 02:32:08 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05T2W8aK022794; Mon, 29 Jun 2020 02:32:08 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05T2W7Zl022335; Mon, 29 Jun 2020 02:32:07 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202006290232.05T2W7Zl022335@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 29 Jun 2020 02:32:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362768 - in projects/nfs-over-tls/sys/fs: nfs nfsclient nfsserver X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/fs: nfs nfsclient nfsserver X-SVN-Commit-Revision: 362768 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2020 02:32:08 -0000 Author: rmacklem Date: Mon Jun 29 02:32:07 2020 New Revision: 362768 URL: https://svnweb.freebsd.org/changeset/base/362768 Log: Delete nfs_use_ext_pgs, since it is not needed. This patch also removes the checks for PMAP_HAS_DMAP, since they were only needed for early testing on i386 hardware, when no actual TLS was being done. It also enables use of ext_pgs mbufs for large read replies. Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Modified: projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Mon Jun 29 00:34:11 2020 (r362767) +++ projects/nfs-over-tls/sys/fs/nfs/nfs_commonsubs.c Mon Jun 29 02:32:07 2020 (r362768) @@ -100,10 +100,6 @@ int nfs_maxcopyrange = 10 * 1024 * 1024; SYSCTL_INT(_vfs_nfs, OID_AUTO, maxcopyrange, CTLFLAG_RW, &nfs_maxcopyrange, 0, "Max size of a Copy so RPC times reasonable"); -bool nfs_use_ext_pgs = false; -SYSCTL_BOOL(_vfs_nfs, OID_AUTO, use_ext_pgs, CTLFLAG_RW, &nfs_use_ext_pgs, - 0, "Set true to use TCP"); - /* * This array of structures indicates, for V4: * retfh - which of 3 types of calling args are used Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Mon Jun 29 00:34:11 2020 (r362767) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clkrpc.c Mon Jun 29 02:32:07 2020 (r362768) @@ -56,7 +56,6 @@ static int nfs_cbproc(struct nfsrv_descript *, u_int32 extern u_long sb_max_adj; extern int nfs_numnfscbd; extern int nfscl_debuglevel; -extern bool nfs_use_ext_pgs; /* * NFS client system calls for handling callbacks. Modified: projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Mon Jun 29 00:34:11 2020 (r362767) +++ projects/nfs-over-tls/sys/fs/nfsclient/nfs_clrpcops.c Mon Jun 29 02:32:07 2020 (r362768) @@ -77,7 +77,6 @@ extern int nfscl_debuglevel; extern int nfs_pnfsiothreads; extern u_long sb_max_adj; extern int nfs_maxcopyrange; -extern bool nfs_use_ext_pgs; NFSCLSTATEMUTEX; int nfstest_outofseq = 0; int nfscl_assumeposixlocks = 1; @@ -1798,8 +1797,7 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iom KASSERT(uiop->uio_iovcnt == 1, ("nfs: writerpc iovcnt > 1")); use_ext = false; - if ((NFSHASTLS(nmp) || nfs_use_ext_pgs) && - PMAP_HAS_DMAP != 0) + if (NFSHASTLS(nmp)) use_ext = true; *attrflagp = 0; tsiz = uiop->uio_resid; @@ -8495,8 +8493,7 @@ nfsrpc_setextattr(vnode_t vp, const char *name, struct *attrflagp = 0; use_ext = false; - if (uiop->uio_resid > MCLBYTES && (NFSHASTLS(nmp) || - nfs_use_ext_pgs) && PMAP_HAS_DMAP != 0) + if (uiop->uio_resid > MCLBYTES && NFSHASTLS(nmp)) use_ext = true; nfscl_reqstart(nd, NFSPROC_SETEXTATTR, nmp, VTONFS(vp)->n_fhp->nfh_fh, VTONFS(vp)->n_fhp->nfh_len, NULL, Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Mon Jun 29 00:34:11 2020 (r362767) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Mon Jun 29 02:32:07 2020 (r362768) @@ -77,7 +77,6 @@ extern struct nfsdontlisthead nfsrv_dontlisthead; extern volatile int nfsrv_dontlistlen; extern volatile int nfsrv_devidcnt; extern int nfsrv_maxpnfsmirror; -extern bool nfs_use_ext_pgs; struct vfsoptlist nfsv4root_opt, nfsv4root_newopt; NFSDLOCKMUTEX; NFSSTATESPINLOCK; @@ -2115,6 +2114,13 @@ again: vput(vp); /* + * If the siz and cnt are larger than MCLBYTES, use ext_pgs for TLS. + */ + if ((nd->nd_flag & (ND_EXTPG | ND_TLS)) == ND_TLS && siz > MCLBYTES && + cnt > MCLBYTES) + nd->nd_flag |= ND_EXTPG; + + /* * dirlen is the size of the reply, including all XDR and must * not exceed cnt. For NFSv2, RFC1094 didn't clearly indicate * if the XDR should be included in "count", but to be safe, we do. @@ -2432,6 +2438,16 @@ again: } /* + * If the reply is likely to exceed MCLBYTES, then use TLS. + * It is difficult to predict how large each entry will be and + * how many entries have been read, so just assume the directory + * entries grow by a factor of 4 when attributes are included. + */ + if ((nd->nd_flag & (ND_EXTPG | ND_TLS)) == ND_TLS && cnt > MCLBYTES && + siz > MCLBYTES / 4) + nd->nd_flag |= ND_EXTPG; + + /* * Save this position, in case there is an error before one entry * is created. */ @@ -6275,12 +6291,10 @@ nfsvno_getxattr(struct vnode *vp, char *name, uint32_t len = siz; tlen = NFSM_RNDUP(len); /* - * If the cnt is larger than MCLBYTES, use ext_pgs if - * possible. + * If the cnt is larger than MCLBYTES, use ext_pgs for TLS. * Always use ext_pgs if ND_EXTPG is set. */ - if ((flag & ND_EXTPG) != 0 || (tlen > MCLBYTES && - PMAP_HAS_DMAP != 0 && ((flag & ND_TLS) != 0 || nfs_use_ext_pgs))) + if ((flag & ND_EXTPG) != 0 || ((flag & ND_TLS) != 0 && tlen > MCLBYTES)) uiop->uio_iovcnt = nfsrv_createiovec_extpgs(tlen, maxextsiz, &m, &m2, &iv); else Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Mon Jun 29 00:34:11 2020 (r362767) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Mon Jun 29 02:32:07 2020 (r362768) @@ -67,7 +67,6 @@ extern u_long sb_max_adj; extern int nfsrv_pnfsatime; extern int nfsrv_maxpnfsmirror; extern int nfs_maxcopyrange; -extern bool nfs_use_ext_pgs; static int nfs_async = 0; SYSCTL_DECL(_vfs_nfsd); @@ -857,13 +856,11 @@ nfsrvd_read(struct nfsrv_descript *nd, __unused int is m3 = NULL; if (cnt > 0) { /* - * If the cnt is larger than MCLBYTES, use ext_pgs if - * possible. + * If the cnt is larger than MCLBYTES, use ext_pgs for TLS. * Always use ext_pgs if ND_EXTPG is set. */ - if ((nd->nd_flag & ND_EXTPG) != 0 || (PMAP_HAS_DMAP != 0 && - ((nd->nd_flag & ND_TLS) != 0 || (nfs_use_ext_pgs && - cnt > MCLBYTES)))) + if ((nd->nd_flag & ND_EXTPG) != 0 || + ((nd->nd_flag & ND_TLS) != 0 && cnt > MCLBYTES)) nd->nd_repstat = nfsvno_read(vp, off, cnt, nd->nd_cred, nd->nd_maxextsiz, p, &m3, &m2); else From owner-svn-src-projects@freebsd.org Tue Jun 30 14:49:52 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ABE0434D323 for ; Tue, 30 Jun 2020 14:49:52 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49x6hc4373z4Vdg; Tue, 30 Jun 2020 14:49:52 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6A9B019E00; Tue, 30 Jun 2020 14:49:52 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05UEnqep072918; Tue, 30 Jun 2020 14:49:52 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05UEnq2x072917; Tue, 30 Jun 2020 14:49:52 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202006301449.05UEnq2x072917@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 30 Jun 2020 14:49:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/rpc: . rpcsec_tls X-SVN-Commit-Revision: 362798 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 14:49:52 -0000 Author: rmacklem Date: Tue Jun 30 14:49:51 2020 New Revision: 362798 URL: https://svnweb.freebsd.org/changeset/base/362798 Log: Testing when a server does not respond to TLS handshake records exposed a couple of problems, since the daemon would be in SSL_connect() for 6 minutes. - When the upcall timed out and was retried, the RPCTLS_SYSC_CLSOCKET syscall was broken and did not return an error upon a retry. It allocated a file descriptor for a NULL socket. - The socket structure in the kernel could be free'd while the daemon was still using it in SSL_connect(). - Adjust the timeout a retry count so that upcalls are only attempted once with a 10minute timeout. This patch fixes these problems by changing the following: - If the handshake is in progress, don't soclose(so) in the kernel clnt_vc_destroy(). - Fix the RPCTLS_SYSC_CLSOCKET (and RPCTLS_SYSC_SRVSOCKET) to correctly return an error if the socket is NULL (which means it already has a file decriptor assigned to it). Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Tue Jun 30 11:50:52 2020 (r362797) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Tue Jun 30 14:49:51 2020 (r362798) @@ -895,12 +895,21 @@ clnt_vc_destroy(CLIENT *cl) * If the upcall fails, the socket has * probably been closed via the rpctlscd * daemon having crashed or been - * restarted. + * restarted, so ignore return stat. */ stat = rpctls_cl_disconnect(ct->ct_sslsec, ct->ct_sslusec, ct->ct_sslrefno, &reterr); - } else { + } else if ((ct->ct_rcvstate & RPCRCVSTATE_TLSHANDSHAKE) == 0) { + /* + * If the TLS handshake is in progress, leave the + * socket so that it will closed by the daemon. + * This can only occur if the daemon is waiting for + * an openssl call like SSL_connect() for a long + * time. The call will normally eventually fail and + * then the daemon will close the socket, so do not + * do it here. + */ soshutdown(so, SHUT_WR); soclose(so); } @@ -1278,17 +1287,20 @@ clnt_vc_dotlsupcall(void *data) enum clnt_stat ret; uint32_t reterr; -printf("TLSupcall started\n"); mtx_lock(&ct->ct_lock); ct->ct_rcvstate |= RPCRCVSTATE_UPCALLTHREAD; while (!ct->ct_closed) { if ((ct->ct_rcvstate & RPCRCVSTATE_UPCALLNEEDED) != 0) { ct->ct_rcvstate &= ~RPCRCVSTATE_UPCALLNEEDED; ct->ct_rcvstate |= RPCRCVSTATE_UPCALLINPROG; - mtx_unlock(&ct->ct_lock); - ret = rpctls_cl_handlerecord(ct->ct_sslsec, ct->ct_sslusec, - ct->ct_sslrefno, &reterr); - mtx_lock(&ct->ct_lock); + if (ct->ct_sslrefno != 0) { + mtx_unlock(&ct->ct_lock); +printf("at handlerecord\n"); + ret = rpctls_cl_handlerecord(ct->ct_sslsec, + ct->ct_sslusec, ct->ct_sslrefno, &reterr); +printf("aft handlerecord=%d\n", ret); + mtx_lock(&ct->ct_lock); + } ct->ct_rcvstate &= ~RPCRCVSTATE_UPCALLINPROG; if (ret == RPC_SUCCESS && reterr == RPCTLSERR_OK) ct->ct_rcvstate |= RPCRCVSTATE_NORMAL; @@ -1309,6 +1321,5 @@ printf("TLSupcall started\n"); ct->ct_rcvstate &= ~RPCRCVSTATE_UPCALLTHREAD; wakeup(&ct->ct_sslrefno); mtx_unlock(&ct->ct_lock); -printf("TLSupcall exit\n"); kthread_exit(); } Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Tue Jun 30 11:50:52 2020 (r362797) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Tue Jun 30 14:49:51 2020 (r362798) @@ -114,8 +114,9 @@ sys_rpctls_syscall(struct thread *td, struct rpctls_sy struct file *fp; struct socket *so; char path[MAXPATHLEN]; - int fd = -1, error, retry_count = 5; + int fd = -1, error, try_count; CLIENT *cl, *oldcl; + struct timeval timeo; #ifdef KERN_TLS u_int maxlen; #endif @@ -155,12 +156,26 @@ printf("got cl=%p\n", cl); /* * The number of retries defaults to INT_MAX, which * effectively means an infinite, uninterruptable loop. - * Limiting it to five retries keeps it from running - * forever. + * Set the try_count to 1 so that no retries of the + * RPC occur. Since it is an upcall to a local daemon, + * requests should not be lost and doing one of these + * RPCs multiple times is not correct. + * SSL_connect() in the openssl library has been + * observed to take 6 minutes when the server is not + * responding to the handshake records, so set the + * timeout to 10min. If it times out before the + * daemon completes the RPC, that should still be ok, + * since the daemon is single threaded and will not + * do further RPCs until the openssl library call + * returns (usually with a failure). */ - if (cl != NULL) - CLNT_CONTROL(cl, CLSET_RETRIES, &retry_count); - else + if (cl != NULL) { + try_count = 1; + CLNT_CONTROL(cl, CLSET_RETRIES, &try_count); + timeo.tv_sec = 10 * 60; + timeo.tv_usec = 0; + CLNT_CONTROL(cl, CLSET_TIMEOUT, &timeo); + } else error = EINVAL; } @@ -203,12 +218,20 @@ printf("got cl=%p\n", cl); /* * The number of retries defaults to INT_MAX, which * effectively means an infinite, uninterruptable loop. - * Limiting it to five retries keeps it from running - * forever. + * Doing even one retry of these upcalls is probably + * not a good plan, since repeating the openssl + * operations are not likely to work. + * The timeout is set fairly large, since some + * openssl operations such as SSL_connect() take a + * long time to return upon failure. */ - if (cl != NULL) - CLNT_CONTROL(cl, CLSET_RETRIES, &retry_count); - else + if (cl != NULL) { + try_count = 1; + CLNT_CONTROL(cl, CLSET_RETRIES, &try_count); + timeo.tv_sec = 2 * 60; + timeo.tv_usec = 0; + CLNT_CONTROL(cl, CLSET_TIMEOUT, &timeo); + } else error = EINVAL; } @@ -249,33 +272,41 @@ printf("srvshutd oldcl=%p\n", oldcl); break; case RPCTLS_SYSC_CLSOCKET: printf("In connect\n"); - error = falloc(td, &fp, &fd, 0); - if (error == 0) { + mtx_lock(&rpctls_connect_lock); + so = rpctls_connect_so; + rpctls_connect_so = NULL; + mtx_unlock(&rpctls_connect_lock); + if (so != NULL) { + error = falloc(td, &fp, &fd, 0); printf("falloc=%d fd=%d\n", error, fd); - mtx_lock(&rpctls_connect_lock); - so = rpctls_connect_so; - rpctls_connect_so = NULL; - mtx_unlock(&rpctls_connect_lock); - finit(fp, FREAD | FWRITE, DTYPE_SOCKET, so, &socketops); - fdrop(fp, td); /* Drop fp reference. */ - td->td_retval[0] = fd; - } -printf("returning=%d\n", fd); + if (error == 0) { + finit(fp, FREAD | FWRITE, DTYPE_SOCKET, so, + &socketops); + fdrop(fp, td); /* Drop fp reference. */ + td->td_retval[0] = fd; + } + } else + error = EPERM; +printf("clsocket err=%d fd=%d\n", error, fd); break; case RPCTLS_SYSC_SRVSOCKET: printf("In srvconnect\n"); - error = falloc(td, &fp, &fd, 0); - if (error == 0) { + mtx_lock(&rpctls_server_lock); + so = rpctls_server_so; + rpctls_server_so = NULL; + mtx_unlock(&rpctls_server_lock); + if (so != NULL) { + error = falloc(td, &fp, &fd, 0); printf("falloc=%d fd=%d\n", error, fd); - mtx_lock(&rpctls_server_lock); - so = rpctls_server_so; - rpctls_server_so = NULL; - mtx_unlock(&rpctls_server_lock); - finit(fp, FREAD | FWRITE, DTYPE_SOCKET, so, &socketops); - fdrop(fp, td); /* Drop fp reference. */ - td->td_retval[0] = fd; - } -printf("srv returning=%d\n", fd); + if (error == 0) { + finit(fp, FREAD | FWRITE, DTYPE_SOCKET, so, + &socketops); + fdrop(fp, td); /* Drop fp reference. */ + td->td_retval[0] = fd; + } + } else + error = EPERM; +printf("srvsocket err=%d fd=%d\n", error, fd); break; default: error = EINVAL; From owner-svn-src-projects@freebsd.org Tue Jun 30 15:47:12 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 755A334E654 for ; Tue, 30 Jun 2020 15:47:12 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49x7ym2V6vz4YR8; Tue, 30 Jun 2020 15:47:12 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 343821A8F0; Tue, 30 Jun 2020 15:47:12 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05UFlCge010160; Tue, 30 Jun 2020 15:47:12 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05UFlCYp010159; Tue, 30 Jun 2020 15:47:12 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202006301547.05UFlCYp010159@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 30 Jun 2020 15:47:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362799 - projects/nfs-over-tls/sys/rpc/rpcsec_tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc/rpcsec_tls X-SVN-Commit-Revision: 362799 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 15:47:12 -0000 Author: rmacklem Date: Tue Jun 30 15:47:11 2020 New Revision: 362799 URL: https://svnweb.freebsd.org/changeset/base/362799 Log: Although I have no way of testing how long the server side (SSL_accept()) might delay when the client is not responding to handshake records, I decided to make the timeout for the server the same as the client. Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Tue Jun 30 14:49:51 2020 (r362798) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Tue Jun 30 15:47:11 2020 (r362799) @@ -218,17 +218,23 @@ printf("got cl=%p\n", cl); /* * The number of retries defaults to INT_MAX, which * effectively means an infinite, uninterruptable loop. - * Doing even one retry of these upcalls is probably - * not a good plan, since repeating the openssl - * operations are not likely to work. - * The timeout is set fairly large, since some - * openssl operations such as SSL_connect() take a - * long time to return upon failure. + * Set the try_count to 1 so that no retries of the + * RPC occur. Since it is an upcall to a local daemon, + * requests should not be lost and doing one of these + * RPCs multiple times is not correct. + * SSL_connect() in the openssl library has been + * observed to take 6 minutes when the server is not + * responding to the handshake records, so set the + * timeout to 10min. If it times out before the + * daemon completes the RPC, that should still be ok, + * since the daemon is single threaded and will not + * do further RPCs until the openssl library call + * returns (usually with a failure). */ if (cl != NULL) { try_count = 1; CLNT_CONTROL(cl, CLSET_RETRIES, &try_count); - timeo.tv_sec = 2 * 60; + timeo.tv_sec = 10 * 60; timeo.tv_usec = 0; CLNT_CONTROL(cl, CLSET_TIMEOUT, &timeo); } else From owner-svn-src-projects@freebsd.org Tue Jun 30 16:02:50 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 39E9334F46A for ; Tue, 30 Jun 2020 16:02:50 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: from mail-oi1-x232.google.com (mail-oi1-x232.google.com [IPv6:2607:f8b0:4864:20::232]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49x8Jn6RDjz4bgD; Tue, 30 Jun 2020 16:02:49 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: by mail-oi1-x232.google.com with SMTP id 12so11400646oir.4; Tue, 30 Jun 2020 09:02:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NU20aYJm5ZKUUAcZkpg73NTqD59/ic4aoN755HyaY4E=; b=Dh21RsM6t5OWV6707jixiYs8HSzN1AEVf/KScGN6V7JqFdmUqKdVpdF41w0wnMEUIq 5GrQTiSFL2/IlLvfIH8ElLN+12YLOBEudB5taLtjXCUuFgoUNe/N2ULGv6dGvUS23lCL aE8DjMBhFHXk0kO+vKxDRi0wBKW6e89vYhsTwMcZCGT8O90DpctmgIXlE/rPL2ubA8w8 nGTKzaYJ55gC53IJ5ZtFkOCgqo7jhwDqsWw0f0WYJE2MDjnRI94gwGkamzjM+ci90viJ GFGz8IDJHhXil/dktFSEP6eEVqZWdFwvUmaWAuTJrxHzi0TkAPmmFxOD7TO4nzrugEm4 z2pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NU20aYJm5ZKUUAcZkpg73NTqD59/ic4aoN755HyaY4E=; b=Um1CACE9nYWmbx7rF6IZbi0x4xN91uTW688alZnHwFkDEldbnbl/ylLLyovwMlvZtw rWysy8dj01gblUxkxsiyhb8DzvOk3YGHrqcrrkpNC+TUcr1ID7bsqGAHo2RHakfSbkFj uY5rRsVzyBRxPyeErveSXFKoKRceVyipWmXJPypKQVEXMD1PnxIz+ZNgYQRmlQSzcMjW m3TwT+1gFIInwJDyRKgZm8KA7PC6neM1AWoH8X0IGiF7Oo0qrxN0qkYw+NnHapb1kRxB DzbmQ5JGIF/ZtDpSnd7HV9hVqRPwMOkYBBnZBwzDv5X5u2wZc7NZTISG1YLwTZ9zIeKs LgFA== X-Gm-Message-State: AOAM5339p7CurVSvguAmc6fTdAbT84tJOR5m6QzDtQPRqSNU47Ak5pRP noOIVKWb03CD4Llh0UUFz+1uIUhKfGJT00qdC44n1g2z X-Google-Smtp-Source: ABdhPJw+nZrS2XsB5ZEcD9kD4pxpaobHMGciyM9kVqaIykFZZwoIvUNpIOrD/lDkDV5cxEvWebNOZrQNekdU4zeEiKg= X-Received: by 2002:aca:f087:: with SMTP id o129mr15401319oih.47.1593532968564; Tue, 30 Jun 2020 09:02:48 -0700 (PDT) MIME-Version: 1.0 References: <202006301449.05UEnq2x072917@repo.freebsd.org> In-Reply-To: <202006301449.05UEnq2x072917@repo.freebsd.org> From: Benjamin Kaduk Date: Tue, 30 Jun 2020 09:02:37 -0700 Message-ID: Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls To: Rick Macklem Cc: src-committers , svn-src-projects@freebsd.org X-Rspamd-Queue-Id: 49x8Jn6RDjz4bgD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 16:02:50 -0000 On Tue, Jun 30, 2020 at 7:49 AM Rick Macklem wrote: > Author: rmacklem > Date: Tue Jun 30 14:49:51 2020 > New Revision: 362798 > URL: https://svnweb.freebsd.org/changeset/base/362798 > > Log: > Testing when a server does not respond to TLS handshake records exposed > a couple of problems, since the daemon would be in SSL_connect() for 6 > minutes. > > - When the upcall timed out and was retried, the RPCTLS_SYSC_CLSOCKET > syscall > was broken and did not return an error upon a retry. It allocated a > file > descriptor for a NULL socket. > - The socket structure in the kernel could be free'd while the daemon was > still using it in SSL_connect(). > - Adjust the timeout a retry count so that upcalls are only attempted > once > with a 10minute timeout. > > 10 minutes seems really long! It sounds from the description like the upcall so that userspace can run SSL_connect() was taking 6 minutes, and you needed 10 minutes so as to be longer than the 6 minutes that is "out of your control"? I feel like there should be some sockopts available to get the SSL_connect() timeout down, so that the upcall timeout doesn't need to be so long, either. -Ben From owner-svn-src-projects@freebsd.org Tue Jun 30 16:20:56 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6D68434FA6F for ; Tue, 30 Jun 2020 16:20:56 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-qb1can01on060d.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5c::60d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49x8jg4R25z4ckB; Tue, 30 Jun 2020 16:20:55 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eBWFtAyTDzK/MwY6CuFnlsb5aMrfe97sOsGRWJD+AHCjXxJ6AxqIoCRzK1oUwLvWklAVG5KPgYYRXKXXaiFH1c8v0ChGz3DZ5s7VhkFzMJMLYQXXuBhZtJ3x4Y7KLtdCyezNRaurgcV92QHnlUjcTSSlk7K/NR4r39Q+ovFm8heBLn/oIZsDCXrT6X2e2MabFF9q7qMzMiK09NQ3gEi6rC/BiGNmF/hj4n9hHINB6nHmGRVryBNSvIcimKUlDTCT2S3/f5apphCKCcIOjjFkSSssoUd1xO19WEqjlChS62SYL1kAfIjMioOWFTY6rjYewW9PisGQW/iXofDUS5JPfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jL92JZ/wNg+UqKE7RhgkTAKb40HQmWmTNsIwAEeWGmI=; b=OeHFVXKe3KqERb/JB9YDJkra0JjDSKSVIaZ3317dkFsR7TYgQqgLV6cdJFo9XVupiHIN6wolO9zfN03v4Kd9rxY77AYKmdZ4bsizOOekZe80Lcfxk88X0snHAtuQp08N2uR0TXUlAfZUBJrZL9unFHJwxJwp+H9AcCllM+rLJ8H1IAzz6corUZ+o2cKY833LygW9yzVBIgBwedwK0qdKcVFeqmg+uimQUCe3YYC80cO9X6utSzy6ZJSp8mbrSMPdLNTgem5Yyzzez595Mqn5aQJjq/yNSQ+rZE004bJX4nRW3iTgK2sXev5zmsSIEaXvZYgday9CK9um8okCfV0VMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jL92JZ/wNg+UqKE7RhgkTAKb40HQmWmTNsIwAEeWGmI=; b=pyBVMo9/FAYWqEAQS+opcYEDwe46ayqVTj28un7oXlrGU/jUClhR8ky2TaoMW7/jFKFCURkw9Vze1BDfn6uCHPDBffzYpkka+3OdiLLJPwM+3O69BR8BKFjOujcMbUuFW7s4ZZrnm3/eArspcieu+ueCpvJotbX2KFDpKbGwbaos1EjXYFOLW6CLNDQYSmnnt/yocxGOOpo0veDGhPyxJiBYXgnB+cTIirIxm7mYpghuS0Q7C5FbsL/g/5snbkpgn5788kr+xLvyfiWgWKnL1Nc2rJAQnqoa6MrfCVIKBVormH6qpkJU+0fkBX5d4PT/OA0ZBMydkg6T4GnHTyt+qg== Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:38::14) by QB1PR01MB2883.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:32::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.23; Tue, 30 Jun 2020 16:20:53 +0000 Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91]) by QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91%7]) with mapi id 15.20.3131.028; Tue, 30 Jun 2020 16:20:45 +0000 From: Rick Macklem To: Benjamin Kaduk , Rick Macklem CC: src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Topic: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Index: AQHWTu29MoFCP0YO4kaq4Sj4aYWIUajxUeOAgAAARmw= Date: Tue, 30 Jun 2020 16:20:45 +0000 Message-ID: References: <202006301449.05UEnq2x072917@repo.freebsd.org>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f4b42c31-df54-4918-6003-08d81d118aec x-ms-traffictypediagnostic: QB1PR01MB2883: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0450A714CB x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: h388b4tt1+wwcwQzGHeVbi0B7hD/vLCEuiJAvWWVHgQvmx4ihjGbU87dfHWbyPyHiraaCL1NoP1D2q0yVRip+YPIm2W8baZX0kuJTkWvEVjqBfVzq8TKIglLXkFwwYPZBKdfCbEhOIsM6iIlkXgdmi9mDjNOj5O2FSDJooo0Mk/8gbf1KHHe9B41VFNPxlI/MXOKhjh7l3Q69lQb/ywvYU3EVld6kjPrbNAe714mbLpLsA+/ny0pS+PfgJubwjd9zIuFdMp9QPsY2+SAr4odmznAfr5EDqO6dhkbQVmByV+Y8H6T3XJuS8MjuizvSSQsNtzEI+2Vvy408H73+ogLs5/UQWtjieGgREmZ1IOCsNu7jfI4uJPDrvtULrE/GNnGwJkbzWCTkFDiNEB5kmmiqA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(39860400002)(136003)(376002)(366004)(396003)(346002)(8676002)(71200400001)(478600001)(86362001)(91956017)(52536014)(8936002)(966005)(786003)(7696005)(2906002)(316002)(9686003)(66946007)(6506007)(5660300002)(76116006)(4326008)(66446008)(54906003)(64756008)(83380400001)(66476007)(110136005)(55016002)(33656002)(186003)(66556008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: YY1bNVF87Tenp+6G5m74p9G16LRaq+2kDjFwEMlW4oIf6kW0+nMaZrM3hjlilFVadWBAfvFygkfp4s2H9FQarZy4MG7oquZECPyAQQUPI0PRPb65uso/G0/eYurwAm4/+Ya0qO8CSF0P2yw7S7UbqFlQL4XdIAJ6RR7vuN7ic3BFX3KHk3QX1ukp/okRvJS9ETKDE3gXWDI9MbdeYU1MwF9wwi2UixWiYSlpXM9Xk9EGVqUYgWBkiOBhq4ljVGODIt+3/2gNYP5d6A5gbnz0AqsI3tZDpfByXbEXcd1jhPEkFT+Sz8jgerdLQbQpTtyMjQq1VvwqAzithDYPrv2qeFVz852xGIqbHUyoaXGb6qqCkcFoIqvijd9bcfqEFFlSMCO1vhqNLkfv5W3V9LoO4jiflstu0B3+QfpgzEk6i/uTJduKj6AQPGgPyoIqape6agQ0ioeTaTuVWhxVhQ2XUY2HtmUPZxOIxmuD7klZQYZsRrlHLFSLEVPVgyV4A0BoqZ8wrPeZ0gUbYPWWw6k6omPB6v0gjKXKvXMACee+ewA= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: f4b42c31-df54-4918-6003-08d81d118aec X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jun 2020 16:20:45.4811 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Q5vwaZL+kiAtgXt84Y0UE2FOxRg4/pjGaJu76kdg0y6k5GkfI6M/Zkt0cBv/IWZGQA4ds8tDd4BgX7OSDkD3Lg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB2883 X-Rspamd-Queue-Id: 49x8jg4R25z4ckB X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=pyBVMo9/; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5c::60d as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-5.91 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; NEURAL_HAM_LONG(-1.04)[-1.039]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; NEURAL_HAM_SHORT(-1.39)[-1.387]; FREEMAIL_TO(0.00)[gmail.com,FreeBSD.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 16:20:56 -0000 Benjamin Kaduk wrote:=0A= >On Tue, Jun 30, 2020 at 7:49 AM Rick Macklem > wrote:=0A= >Author: rmacklem=0A= >Date: Tue Jun 30 14:49:51 2020=0A= >New Revision: 362798=0A= >URL: https://svnweb.freebsd.org/changeset/base/362798=0A= >=0A= >Log:=0A= > Testing when a server does not respond to TLS handshake records exposed= =0A= > a couple of problems, since the daemon would be in SSL_connect() for 6 m= inutes.=0A= >=0A= > - When the upcall timed out and was retried, the RPCTLS_SYSC_CLSOCKET sy= scall=0A= > was broken and did not return an error upon a retry. It allocated a fi= le=0A= > descriptor for a NULL socket.=0A= > - The socket structure in the kernel could be free'd while the daemon wa= s=0A= > still using it in SSL_connect().=0A= > - Adjust the timeout a retry count so that upcalls are only attempted on= ce=0A= > with a 10minute timeout.=0A= >=0A= >=0A= >10 minutes seems really long! It sounds from the description like the upc= all so >that=0A= >userspace can run SSL_connect() was taking 6 minutes, and you needed 10 >m= inutes so=0A= >as to be longer than the 6 minutes that is "out of your control"?=0A= Well, I think a long timeout here is ok, since a timeout indicates a broken= daemon.=0A= (The upcalls to the local daemon should be reliable and cannot safely be re= done.=0A= In a perfect world, the upcall mechanism would be "exactly once" instead o= f=0A= "at least once". I think an upcall might fail when the mbuf pool in the ke= rnel=0A= is exhausted, but that should be rare.)=0A= =0A= >I feel like there should be some sockopts available to get the SSL_connect= () timeout=0A= >down, so that the upcall timeout doesn't need to be so long, either.=0A= Yes, 6 minutes does seem like a long time. I only discovered this yesterday= when=0A= I simulated a server that did not respond to handshake records.=0A= =0A= I haven't yet dug into the openssl code to see if there is a way to adjust = this=0A= timeout.=0A= I also do not know what a good timeout value for SSL_connect() might be,=0A= even if the daemon can override the default.=0A= =0A= In practice, this should only happen when trying to do an NFS mount on=0A= a broken server which responds to the "STARTTLS" Null RPC, but does not=0A= do the handshake.=0A= Having the mount attempt stuck for 6minutes before failing is not that seri= ous=0A= a problem, imho.=0A= (When systems boot after something like a power failure, delays getting NFS= =0A= mounts done, due to the NFS server/network needing to be up, is fairly=0A= normal. The "-b" option to put the mount attempt in background has been=0A= around for a long time for this.)=0A= =0A= If you happen to know how to set a timeout for SSL_connect() in the openssl= =0A= library, I would be interested in hearing that.=0A= =0A= rick=0A= =0A= -Ben=0A= From owner-svn-src-projects@freebsd.org Tue Jun 30 16:33:46 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 005A5350159 for ; Tue, 30 Jun 2020 16:33:46 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49x90T2bDFz4f9p; Tue, 30 Jun 2020 16:33:45 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from kduck.mit.edu ([24.16.140.251]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 05UGXegL006615 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 30 Jun 2020 12:33:43 -0400 Date: Tue, 30 Jun 2020 09:33:40 -0700 From: Benjamin Kaduk To: Rick Macklem Cc: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Message-ID: <20200630163340.GN58278@kduck.mit.edu> References: <202006301449.05UEnq2x072917@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 49x90T2bDFz4f9p X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of kaduk@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=kaduk@mit.edu X-Spamd-Result: default: False [-2.96 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[24.16.140.251:received]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:18.9.28.0/24]; NEURAL_HAM_LONG(-1.03)[-1.027]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[mit.edu]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[18.9.28.11:from]; NEURAL_HAM_SHORT(-0.44)[-0.445]; NEURAL_HAM_MEDIUM(-0.99)[-0.989]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3, ipnet:18.9.0.0/16, country:US]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RWL_MAILSPIKE_VERYGOOD(0.00)[18.9.28.11:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 16:33:46 -0000 On Tue, Jun 30, 2020 at 04:20:45PM +0000, Rick Macklem wrote: > Benjamin Kaduk wrote: > >On Tue, Jun 30, 2020 at 7:49 AM Rick Macklem > wrote: > >Author: rmacklem > >Date: Tue Jun 30 14:49:51 2020 > >New Revision: 362798 > >URL: https://svnweb.freebsd.org/changeset/base/362798 > > > >Log: > > Testing when a server does not respond to TLS handshake records exposed > > a couple of problems, since the daemon would be in SSL_connect() for 6 minutes. > > > > - When the upcall timed out and was retried, the RPCTLS_SYSC_CLSOCKET syscall > > was broken and did not return an error upon a retry. It allocated a file > > descriptor for a NULL socket. > > - The socket structure in the kernel could be free'd while the daemon was > > still using it in SSL_connect(). > > - Adjust the timeout a retry count so that upcalls are only attempted once > > with a 10minute timeout. > > > > > >10 minutes seems really long! It sounds from the description like the upcall so >that > >userspace can run SSL_connect() was taking 6 minutes, and you needed 10 >minutes so > >as to be longer than the 6 minutes that is "out of your control"? > Well, I think a long timeout here is ok, since a timeout indicates a broken daemon. > (The upcalls to the local daemon should be reliable and cannot safely be redone. > In a perfect world, the upcall mechanism would be "exactly once" instead of > "at least once". I think an upcall might fail when the mbuf pool in the kernel > is exhausted, but that should be rare.) > > >I feel like there should be some sockopts available to get the SSL_connect() timeout > >down, so that the upcall timeout doesn't need to be so long, either. > Yes, 6 minutes does seem like a long time. I only discovered this yesterday when > I simulated a server that did not respond to handshake records. > > I haven't yet dug into the openssl code to see if there is a way to adjust this > timeout. > I also do not know what a good timeout value for SSL_connect() might be, > even if the daemon can override the default. > > In practice, this should only happen when trying to do an NFS mount on > a broken server which responds to the "STARTTLS" Null RPC, but does not > do the handshake. > Having the mount attempt stuck for 6minutes before failing is not that serious > a problem, imho. > (When systems boot after something like a power failure, delays getting NFS > mounts done, due to the NFS server/network needing to be up, is fairly > normal. The "-b" option to put the mount attempt in background has been > around for a long time for this.) > > If you happen to know how to set a timeout for SSL_connect() in the openssl > library, I would be interested in hearing that. As it happens, I took a look before I wrote the initial note, and there doesn't seem to be any intrinsic TLS (not DTLS) handshake timeouts in libssl itself; I expect this is actually just the (kernel's!) TCP timeout. So you'd be getting the socket fd (e.g., SSL_get_fd(), if you don't have a reference already) and using setsockopt() to set the timeout(s). -Ben From owner-svn-src-projects@freebsd.org Wed Jul 1 00:43:59 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D07E6359E81 for ; Wed, 1 Jul 2020 00:43:59 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-qb1can01on0609.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5c::609]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xMt64CFmz48qT; Wed, 1 Jul 2020 00:43:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VKC/ENotjofpkWkREfOETO3OCELfbImVzpULu3NqMt3Faz5PPlC/D3lzknx8oz656xHFMt8VJw/hu6eIJfQmNVtEC4OcL+lg1cdMV4ZQQsej2wy8bySrxh6WHiv79Kr+23l6ViY3fDFVepgDPg5ME+dZXFUuhNOwe+dDp2wEpwMQ2qItqttP1ENZacQ5g3rXz7cQOtPcQfMlFwXyEor0tyssIF3sYwBXchP2kiLayTnu/l8Nw11J+u0iLXwFvBibbcq3AmUFQ8/lDai7irHhk7Dt5TstJHgIsj0l+DUQc6/PFOtsrkiTSOygryCve0ScWEmCqn4UreiNhcL2tvZcVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T2fN+GzNekKv8CwPw+KfZh2cO26Fkq/cXvNpwyT6qkw=; b=nLOyTtf8oV87C6JeTCFmy6o71aRT+jYat/x/IyAi675IZFf6cb71Ue9CYmd/U1D08pkwlAtvwE+41hho1YHD2iMpqU+TFEcxS4Yef+t6ASAEYmMyuA5RfUmWX0c5ZnuxsePkLoh4FWYqWjMwshpPEHk6S8bsBbNMvwFM1uSec3U6sToeLwusdH4cZIXAtjtiqaYSFgjs7HNUiazbwiSNNunVqd5I+h4XyvPXBRO1vHW7+3wB+g+pqkISndNW/1ibVT10sUxovvHMBhZwBB4djUMhtTVPwKD0Dumn16asQlUBhf/fjXyu3frcP3bE5sVBiUeqEboD8+EUk5YTomw7qg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T2fN+GzNekKv8CwPw+KfZh2cO26Fkq/cXvNpwyT6qkw=; b=DAWeWb0B/B6R1i3z0Wsk49oCJrkWe8w9Ar6gT793QfjQutwk96e1hN20nCiI1VS2PvRKqf+Yk0L0A5sLCSH7uEw9T2vuNaKkDiH7mBjnJ83aN7pFnQzIxF04AXSQzVY7dReMx6nL+gDsSux5yAOlwLO2pDUczVfVAg+ZIerb3LG/R40bhiWTlPQMbgut0Tj6gEkyL/Mo0oGWLkdCfwWKjzPu2zT7bPGOAPlHt2eHli8vuwMJX4nA5I3BvWAbET+x+VCwT5SGFZ9u9H/l1/IOBi2LjBHXFSkGS8rMcjw52IIfAsCdQwYgAKR2RugIcTIk/ZG6VV/YhWMjMg9//DrGTg== Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:38::14) by QB1PR01MB3362.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:37::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.20; Wed, 1 Jul 2020 00:43:51 +0000 Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91]) by QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91%7]) with mapi id 15.20.3131.028; Wed, 1 Jul 2020 00:43:41 +0000 From: Rick Macklem To: Benjamin Kaduk CC: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Topic: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Index: AQHWTu29MoFCP0YO4kaq4Sj4aYWIUajxUeOAgAAARmyAAAhnAIAAg0Q8 Date: Wed, 1 Jul 2020 00:43:41 +0000 Message-ID: References: <202006301449.05UEnq2x072917@repo.freebsd.org> , <20200630163340.GN58278@kduck.mit.edu> In-Reply-To: <20200630163340.GN58278@kduck.mit.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 317b9ed8-3f6c-4fdc-5c2d-08d81d57ccfc x-ms-traffictypediagnostic: QB1PR01MB3362: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 04519BA941 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: mtmTXF0qV4lgLtEhUTE+BQT4TzIA1Oil0NLvw6W9iEp8E53ntz1OqUDJnxGfC3dU4XWjnXq8Ly7oKNjtI8rBkJTMaDhsicR1bCFRsv9yULe6ML/5Sb3nE6mFlQtTOkBLfqnkKLMgRWkStglh0hwHkS4AVKVrFsxJExBwyYazIeqzfA2Xs+fkFm9knI9F9aWMoknPtmSCnJ2S76082e4fHpQOdBh8CrgkkopqfKIMQqrUTdD4Xq4oWB++V8lkI3iyN20OoSJPtL3efzsL32ooOmnPujnyztDLHmZx7w4yzceLSSXZqqExMEdbSvYIcLhH2DHvOQi3AL1k0nFk+Te7suj5OmHFi6xxV5wgdQWNUv54RNUcRygGNuLR4TpwXowdrjmK9wqsPZxGAlfYXOoOqg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(366004)(396003)(39860400002)(346002)(376002)(136003)(8676002)(71200400001)(64756008)(66446008)(76116006)(2906002)(66946007)(91956017)(66476007)(4326008)(786003)(66556008)(316002)(8936002)(966005)(33656002)(54906003)(186003)(6506007)(86362001)(52536014)(478600001)(7696005)(9686003)(5660300002)(55016002)(6916009)(83380400001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: eKd2+LsYaiJh0W7Wn8PXJ+IQqWZaxnJgwmy+UvJKg0RO2xP52AQom6x1LYeW1ZbkM2DO1oRJtNxSLjVEXdEeibHMQnYYNHexfRlWDIY/2b65SYqWxJIbhEi89Nbk1MXC8IvDj5zaqLGUHfIogqKKMjhlbXnrOWSjBHMSyAcVSKLPXEIzyXKpB3AMRDo6ZSFNBPBeq/VV94lnqcTBVgOftc/JXzYKAMrvZ9++FuVJ8+EA+CdC0ss+mwY7+7XJ/Ov+p2qJyX7dbrcwidKjiwkYgLRCTpeJW9DKjn8TBdanrq4i3tHESO0tnqO78nfQ+aKaqw7aIx6YlWsJhpjiV/5bfSI7Ow1OgIgPyUotZRXlMsDvqIQZb3sgi1Reu0JcRZdbHWPsCHTMok7T5vdXqvfGJMayekSa46Iq2SAWghLySFNZf2LKQiIeXu0tllhAZrWUE8IZe2Lp0A3RGtP3RT7xBfV36QPMtLkNjTsG5RnV0DHodzqn9beo2sVaT1pRRZpU1vGrb61Sd/KWJtH5IBdSb3kya25WkYmc3PYEmxa+pTs= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 317b9ed8-3f6c-4fdc-5c2d-08d81d57ccfc X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2020 00:43:41.0246 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: gfw2OJGaSaQJppdT0IOK1r4j38SbW/Mb38UyjaJdMF4BCuMR0nn2hHk6Aueym9yfNeirokz5o0gdGWJmPTwehA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB3362 X-Rspamd-Queue-Id: 49xMt64CFmz48qT X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=DAWeWb0B; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5c::609 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.67 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.016]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_LONG(-1.02)[-1.024]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; RCPT_COUNT_FIVE(0.00)[5]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; NEURAL_HAM_SHORT(-0.13)[-0.128]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 00:43:59 -0000 Benjamin Kaduk wrote:=0A= >On Tue, Jun 30, 2020 at 04:20:45PM +0000, Rick Macklem wrote:=0A= >> Benjamin Kaduk wrote:=0A= >> >On Tue, Jun 30, 2020 at 7:49 AM Rick Macklem > wrote:=0A= >> >Author: rmacklem=0A= >> >Date: Tue Jun 30 14:49:51 2020=0A= >> >New Revision: 362798=0A= >> >URL: https://svnweb.freebsd.org/changeset/base/362798=0A= >> >=0A= >> >Log:=0A= >> > Testing when a server does not respond to TLS handshake records expos= ed=0A= >> > a couple of problems, since the daemon would be in SSL_connect() for = 6 minutes.=0A= >> >=0A= >> > - When the upcall timed out and was retried, the RPCTLS_SYSC_CLSOCKET= syscall=0A= >> > was broken and did not return an error upon a retry. It allocated a= file=0A= >> > descriptor for a NULL socket.=0A= >> > - The socket structure in the kernel could be free'd while the daemon= was=0A= >> > still using it in SSL_connect().=0A= >> > - Adjust the timeout a retry count so that upcalls are only attempted= once=0A= >> > with a 10minute timeout.=0A= >> >=0A= >> >=0A= >> >10 minutes seems really long! It sounds from the description like the = upcall so >>that=0A= >> >userspace can run SSL_connect() was taking 6 minutes, and you needed 10= >>minutes so=0A= >> >as to be longer than the 6 minutes that is "out of your control"?=0A= >> Well, I think a long timeout here is ok, since a timeout indicates a bro= ken daemon.=0A= >> (The upcalls to the local daemon should be reliable and cannot safely be= redone.=0A= >> In a perfect world, the upcall mechanism would be "exactly once" instea= d of=0A= >> "at least once". I think an upcall might fail when the mbuf pool in the= kernel=0A= >> is exhausted, but that should be rare.)=0A= >>=0A= >> >I feel like there should be some sockopts available to get the SSL_conn= ect() timeout=0A= >> >down, so that the upcall timeout doesn't need to be so long, either.=0A= >> Yes, 6 minutes does seem like a long time. I only discovered this yester= day when=0A= >> I simulated a server that did not respond to handshake records.=0A= >>=0A= >> I haven't yet dug into the openssl code to see if there is a way to adju= st this=0A= >> timeout.=0A= >> I also do not know what a good timeout value for SSL_connect() might be,= =0A= >> even if the daemon can override the default.=0A= >>=0A= >> In practice, this should only happen when trying to do an NFS mount on= =0A= >> a broken server which responds to the "STARTTLS" Null RPC, but does not= =0A= >> do the handshake.=0A= >> Having the mount attempt stuck for 6minutes before failing is not that s= erious=0A= >> a problem, imho.=0A= >> (When systems boot after something like a power failure, delays getting = NFS=0A= >> mounts done, due to the NFS server/network needing to be up, is fairly= =0A= >> normal. The "-b" option to put the mount attempt in background has been= =0A= >> around for a long time for this.)=0A= >>=0A= >> If you happen to know how to set a timeout for SSL_connect() in the open= ssl=0A= >> library, I would be interested in hearing that.=0A= >=0A= >As it happens, I took a look before I wrote the initial note, and there=0A= >doesn't seem to be any intrinsic TLS (not DTLS) handshake timeouts in=0A= >libssl itself; I expect this is actually just the (kernel's!) TCP timeout.= =0A= >So you'd be getting the socket fd (e.g., SSL_get_fd(), if you don't have a= =0A= >reference already) and using setsockopt() to set the timeout(s).=0A= Interesting. The test case I simulated did not close the TCP socket used by= =0A= SSL_connect(). The server just replied to the STARTTLS Null RPC, but did no= t=0A= call SSL_accept(), so the server side just isn't playing "handshake".=0A= "netstat -a" showed the connection as ESTABLISHED.=0A= During debugging, I also used the trick of putting:=0A= while (1)=0A= sleep(1);=0A= right after the SSL_connect() call and, when watching it via "ps",=0A= it would switch from "sbwait" to "nanoslp" after 6 minutes and=0A= a syslog() call showed that SSL_connect() had returned -1.=0A= =0A= So, if the TCP connection was "established", what caused the SSL_connect()= =0A= to return with an error (-1) after 6 minutes?=0A= =0A= Now, there is a 6 minute idle timeout in the RPC code for TCP where it,=0A= by default, closes the connection when there is 6 minutes without any=0A= activity. (I have to look if waiting for a reply for the upcall implies "no= activity" and if=0A= this also happens for AF_LOCAL sockets, which is what the upcalls use.)=0A= =0A= Now, if that happens, a SIGPIPE would be posted to the daemon, which=0A= is SIG_IGN'd by the daemon. But maybe the SIGPIPE somehow causes=0A= SSL_connect() to return -1 by making the syscall it is doing (read/recv on = the=0A= TCP socket sitting in sbwait) return EINTR, or something like that?=0A= =0A= I can change this 6minute timeout to see if that affects it.=0A= =0A= When you've got upcalls and library functions both talking to sockets it=0A= can get interesting.=0A= =0A= Thanks for the comments, rick=0A= =0A= -Ben=0A= =0A= From owner-svn-src-projects@freebsd.org Wed Jul 1 01:23:54 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3859D35A73F for ; Wed, 1 Jul 2020 01:23:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660061.outbound.protection.outlook.com [40.107.66.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xNm90TSNz4BnK; Wed, 1 Jul 2020 01:23:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HZHCo1Gfz749WGJjRZdkUmvL0dSu6lXotzWF5A8QF//lOWawrv+JaGEXrE9bWY/ucE7jY2C39X5vK/mDxJm/n0RnFsMDPaIYiGWx2DnAn9ZyBVRTSzeGd9zGNtJAJC0yXjksIGce+eQbgC7CsoAnTJj48dvjwb8SRxNXrWKXbzT8jmsjZqPRWd1Y7LLzAkFg0uSZBC6bv9gx+0s8wKEheamzcbDc/G3kv5C+cZqIlk9w6tZwhVek3+Z57xw005tE15KNtFjK1Tar/fLa+0WrScT5IvxmQGIi7D8vzmrd8qFn6K6bUnVtitG/kqhDCuqj5WKUd+Pl44WobC+M3lrkdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ozr9pB6zgv7DLg4hWE0f5QBtT3J4oR1Y4QbJyTZT18U=; b=EqePFfvSyJznboavWrVBicBBk4H3T/Pj7DAEkZYoy4hslnrikrjEnc1G1tukil8Dox0xsIzu4Gs/iesVQSo/Hv6bdaIm4XYWX60d5jC5BCapIpTOHEBFRq5RfvohLARwgbIdkoc6gc+hfsgTHqoYwzo4cWGkzWoj4HpV4fdOnoubqZN7sUxyyipseNhWE5EZl+/j3xL+N9noF/WT2CqxXn9tt0qTPzmJw84NV5mTJ+y+4iJr++qDxZTyenEA9vNa6EPFj2+MqRg7NhoONCxCqxcd6rFpMDg51XTKnqGDdMb78xU07gG0+pPGJBnjaW+yJ9fYHMWsSgAgkPbgk8E3LA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ozr9pB6zgv7DLg4hWE0f5QBtT3J4oR1Y4QbJyTZT18U=; b=JvHzFsorzuHaJB+D4VYdm6FKx+OyjaWKw93SkL46vn+7OIQ2IAo6qt4BRYZPabQZh4BXTWjkglOYSHioFwUafpT5kk2DjD5nA+5RzJliYC1i10rIX1KeswRXk/YNAlQz3cOryBY2+bIOk5B9CkvEsdd+sFdQmsQr78VMJR9VN8DWhcMPUM9Y/ubFVcvEWxlotDv/GEA32IhJwhabOyNsun+LUyl6Csx0v95wTN2EUG+OQJYKRWE0yGmE7ZGV1rxKtKtgRY6x8VkezYjT7u0LwP2J6kTc+bCHAIe6RG/DHGeMOmaSbARBrL8tHVagOMWlF5hziD6n3NI/hPXY7kOjFA== Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:38::14) by YQXPR01MB3463.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:44::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.20; Wed, 1 Jul 2020 01:23:50 +0000 Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91]) by QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91%7]) with mapi id 15.20.3131.028; Wed, 1 Jul 2020 01:23:50 +0000 From: Rick Macklem To: Benjamin Kaduk CC: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Topic: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Index: AQHWTu29MoFCP0YO4kaq4Sj4aYWIUajxUeOAgAAARmyAAAhnAIAAg0Q8gAANlPs= Date: Wed, 1 Jul 2020 01:23:50 +0000 Message-ID: References: <202006301449.05UEnq2x072917@repo.freebsd.org> , <20200630163340.GN58278@kduck.mit.edu>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ca60a95d-369d-4ba6-4d10-08d81d5d693a x-ms-traffictypediagnostic: YQXPR01MB3463: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 04519BA941 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: +dg6EQ4GXaYmq0BhvOtUObazblbiHuKsPvrrYXuNOKOpDtOUiDtXXCuJILKiOxnNtHYP0YCEPFxEWwvESbM02EWRMpgB5YMvVTjGu5cGZcrBcG5b1KqUy8vdGt5cz5w6gEmLgaybo7ihsYJTO8t7DZ8htbNmubqqJG4e/jm4ULJjFlhbEeio6RpTX5w5b7IthdYOO0iedNCp8ez0pMnGF/SirR4YAyS4u7lx3zQXTQYGDScSLCUpyE6UzB2tsJWHDcoc9ohtIj7mgK0zV4qijkKMvVAs6oeiRP5aKU4gb2TC7UKFXsu4g5eLdPVwa5uW+Z/XMitZ+db/Ozkz5Vvn+VTXG38zyrrFbTt+roOTI20dMYyImVWCmgg0W9CwsgSb9t8eR0Po/xWhBJaG9JzA8A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(136003)(366004)(39860400002)(346002)(396003)(376002)(786003)(5660300002)(966005)(316002)(2906002)(52536014)(478600001)(6506007)(86362001)(4326008)(54906003)(64756008)(6916009)(66446008)(9686003)(33656002)(7696005)(55016002)(83380400001)(8676002)(186003)(2940100002)(71200400001)(66946007)(91956017)(66476007)(76116006)(66556008)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 5HJv2hcW2BbM/PV83g3ynUM99cbEsQNjef6EkRId5wcGsO0P0PvX1QWQL1dB1d+9T07LdiNn3lgHccKsi+9/2J1cFXv0pDDcX51lwk28pecqpbXiOOZbMw2/Nd7sn8nuuB5w/D+gBh9Ew1Lq8CefH/Q4LROfu8JGEsZiLu5f5d98bLGkN27UJJW2hPylDAzziK6/Z2G3xPt+ZQRJNmpZjU37Ln41UbT4WQfrqzcRy7a5QD0FqesbZJjfT+hpdzrl20/FYnb9iTC+tYfdK9qaskqU6YhTU5kdkGIYABWic9sNJKg9r9mMUNX9yRfSxK3DwznvuUbMvYTurR4T7/0Tgpdc1P0L9c2HlLY38rVN2TL/zHpLCYsfKhjwAAiCn5zAJhToPsL1LlfNiM5wAeWpR5o0h7n2Fhme7giRIsXp6wx5DZDy7T4DfL92XV1bnZbOuE7wwVmV204aV2SmYoHssfNgy2a0HIENr2xk+hKqeaRSpxKJ3bUYmhwgWd65xBDij+z7cN3vaW87XWT40lf17NFPEXzKMZG2LnwL9elZrOs= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: ca60a95d-369d-4ba6-4d10-08d81d5d693a X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2020 01:23:50.6788 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GPcCY1G5LfltHhUUyISHSqQmn7gUNe6iembeQlNa2x75zjZQjgPzshq9n2+hBI8Hqjy6G1cM2H5Bt2OrCKDJTw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB3463 X-Rspamd-Queue-Id: 49xNm90TSNz4BnK X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=JvHzFsor; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.66.61 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.88 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.66.61:from]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-1.02)[-1.015]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; RCPT_COUNT_FIVE(0.00)[5]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; NEURAL_HAM_SHORT(-0.36)[-0.364]; RCVD_IN_DNSWL_NONE(0.00)[40.107.66.61:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 01:23:54 -0000 Rick Macklem wrote:=0A= >Benjamin Kaduk wrote:=0A= >>On Tue, Jun 30, 2020 at 04:20:45PM +0000, Rick Macklem wrote:=0A= >>> Benjamin Kaduk wrote:=0A= >>> >On Tue, Jun 30, 2020 at 7:49 AM Rick Macklem > wrote:=0A= >>> >Author: rmacklem=0A= >>> >Date: Tue Jun 30 14:49:51 2020=0A= >>> >New Revision: 362798=0A= >>> >URL: https://svnweb.freebsd.org/changeset/base/362798=0A= >>> >=0A= >>> >Log:=0A= >>> > Testing when a server does not respond to TLS handshake records expo= sed=0A= >>> > a couple of problems, since the daemon would be in SSL_connect() for= 6 minutes.=0A= >>> >=0A= >>> > - When the upcall timed out and was retried, the RPCTLS_SYSC_CLSOCKE= T syscall=0A= >>> > was broken and did not return an error upon a retry. It allocated = a file=0A= >>> > descriptor for a NULL socket.=0A= >>> > - The socket structure in the kernel could be free'd while the daemo= n was=0A= >>> > still using it in SSL_connect().=0A= >>> > - Adjust the timeout a retry count so that upcalls are only attempte= d once=0A= >>> > with a 10minute timeout.=0A= >>> >=0A= >>> >=0A= >>> >10 minutes seems really long! It sounds from the description like the= upcall so >>that=0A= >>> >userspace can run SSL_connect() was taking 6 minutes, and you needed 1= 0 >>>minutes so=0A= >>> >as to be longer than the 6 minutes that is "out of your control"?=0A= >>> Well, I think a long timeout here is ok, since a timeout indicates a br= oken daemon.=0A= >>> (The upcalls to the local daemon should be reliable and cannot safely b= e redone.=0A= >>> In a perfect world, the upcall mechanism would be "exactly once" inste= ad of=0A= >>> "at least once". I think an upcall might fail when the mbuf pool in th= e kernel=0A= >>> is exhausted, but that should be rare.)=0A= >>>=0A= >>> >I feel like there should be some sockopts available to get the SSL_con= nect() timeout=0A= >>> >down, so that the upcall timeout doesn't need to be so long, either.= =0A= >>> Yes, 6 minutes does seem like a long time. I only discovered this yeste= rday when=0A= >>> I simulated a server that did not respond to handshake records.=0A= >>>=0A= >>> I haven't yet dug into the openssl code to see if there is a way to adj= ust this=0A= >>> timeout.=0A= >>> I also do not know what a good timeout value for SSL_connect() might be= ,=0A= >>> even if the daemon can override the default.=0A= >>>=0A= >>> In practice, this should only happen when trying to do an NFS mount on= =0A= >>> a broken server which responds to the "STARTTLS" Null RPC, but does not= =0A= >>> do the handshake.=0A= >>> Having the mount attempt stuck for 6minutes before failing is not that = serious=0A= >>> a problem, imho.=0A= >>> (When systems boot after something like a power failure, delays getting= NFS=0A= >>> mounts done, due to the NFS server/network needing to be up, is fairly= =0A= >>> normal. The "-b" option to put the mount attempt in background has bee= n=0A= >>> around for a long time for this.)=0A= >>>=0A= >>> If you happen to know how to set a timeout for SSL_connect() in the ope= nssl=0A= >>> library, I would be interested in hearing that.=0A= >>=0A= >>As it happens, I took a look before I wrote the initial note, and there= =0A= >>doesn't seem to be any intrinsic TLS (not DTLS) handshake timeouts in=0A= >>libssl itself; I expect this is actually just the (kernel's!) TCP timeout= .=0A= >>So you'd be getting the socket fd (e.g., SSL_get_fd(), if you don't have = a=0A= >>reference already) and using setsockopt() to set the timeout(s).=0A= >Interesting. The test case I simulated did not close the TCP socket used b= y=0A= >SSL_connect(). The server just replied to the STARTTLS Null RPC, but did n= ot=0A= >call SSL_accept(), so the server side just isn't playing "handshake".=0A= >"netstat -a" showed the connection as ESTABLISHED.=0A= >During debugging, I also used the trick of putting:=0A= > while (1)=0A= > sleep(1);=0A= >right after the SSL_connect() call and, when watching it via "ps",=0A= >it would switch from "sbwait" to "nanoslp" after 6 minutes and=0A= >a syslog() call showed that SSL_connect() had returned -1.=0A= >=0A= >So, if the TCP connection was "established", what caused the SSL_connect()= =0A= >to return with an error (-1) after 6 minutes?=0A= >=0A= >Now, there is a 6 minute idle timeout in the RPC code for TCP where it,=0A= >by default, closes the connection when there is 6 minutes without any=0A= >activity. (I have to look if waiting for a reply for the upcall implies "n= o activity" and >if=0A= >this also happens for AF_LOCAL sockets, which is what the upcalls use.)=0A= Ok, I figured out what is happening for this test.=0A= It is the 6 minute idle timeout, but it occurs at the server end, where the= NFS server=0A= end shuts down the TCP connection.=0A= Now, the client cannot assume all servers will do this.=0A= I'm going to try playing around with doing a shutdown of the socket on the= =0A= client end after a shorter timeout on the upcall and see if that can get=0A= SSL_connect() to return with a failure in the daemon.=0A= =0A= >Now, if that happens, a SIGPIPE would be posted to the daemon, which=0A= >is SIG_IGN'd by the daemon. But maybe the SIGPIPE somehow causes=0A= >SSL_connect() to return -1 by making the syscall it is doing (read/recv on= the=0A= >TCP socket sitting in sbwait) return EINTR, or something like that?=0A= Ignore this "theory". It was bunk.=0A= =0A= >I can change this 6minute timeout to see if that affects it.=0A= Can't be changed, since it is at the server end of the TCP connection.=0A= (A comment in the krpc code mentions a 5minute timeout in the client,=0A= but I don't see that in the code?)=0A= =0A= >When you've got upcalls and library functions both talking to sockets it= =0A= >can get interesting.=0A= >=0A= >Thanks for the comments, rick=0A= =0A= Correcting myself, rick=0A= =0A= -Ben=0A= =0A= From owner-svn-src-projects@freebsd.org Wed Jul 1 02:11:17 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A6EA35ACB1 for ; Wed, 1 Jul 2020 02:11:17 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xPpr71YHz4CpC; Wed, 1 Jul 2020 02:11:16 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D2FCF2200F; Wed, 1 Jul 2020 02:11:16 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0612BGeY098825; Wed, 1 Jul 2020 02:11:16 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0612BG0u098822; Wed, 1 Jul 2020 02:11:16 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202007010211.0612BG0u098822@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 1 Jul 2020 02:11:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362820 - projects/nfs-over-tls/sys/rpc X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/rpc X-SVN-Commit-Revision: 362820 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 02:11:17 -0000 Author: rmacklem Date: Wed Jul 1 02:11:15 2020 New Revision: 362820 URL: https://svnweb.freebsd.org/changeset/base/362820 Log: Use a reserved value for ssl refno to indicate that a handshake is in progress. This is needed so clnt_vc_destroy() will not do a soclose() on the socket, since the daemon may still be in SSL_connect(). Modified: projects/nfs-over-tls/sys/rpc/clnt_rc.c projects/nfs-over-tls/sys/rpc/clnt_vc.c projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Modified: projects/nfs-over-tls/sys/rpc/clnt_rc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_rc.c Wed Jul 1 01:12:23 2020 (r362819) +++ projects/nfs-over-tls/sys/rpc/clnt_rc.c Wed Jul 1 02:11:15 2020 (r362820) @@ -198,6 +198,16 @@ clnt_reconnect_connect(CLIENT *cl) (struct sockaddr *) &rc->rc_addr, rc->rc_prog, rc->rc_vers, rc->rc_sendsz, rc->rc_recvsz, rc->rc_intr); if (rc->rc_tls && newclient != NULL) { + /* + * Set ssl refno so that clnt_vc_destroy() will not + * close the socket and will leave that for the + * daemon to do. It is possible that the upcall + * will time out, so that closing the socket via + * the CLNT_CLOSE() below would happen too soon. + */ + ssl[0] = ssl[1] = 0; + ssl[2] = RPCTLS_REFNO_HANDSHAKE; + CLNT_CONTROL(newclient, CLSET_TLS, ssl); printf("at rpctls_connect\n"); stat = rpctls_connect(newclient, so, ssl, &reterr); printf("aft rpctls_connect=%d ssl=%jd\n", stat, (uintmax_t)ssl[2]); Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Wed Jul 1 01:12:23 2020 (r362819) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Wed Jul 1 02:11:15 2020 (r362820) @@ -775,12 +775,15 @@ printf("backch tls=0x%x xprt=%p\n", xprt->xp_tls, xprt ct->ct_sslsec = *p++; ct->ct_sslusec = *p++; ct->ct_sslrefno = *p; - mtx_unlock(&ct->ct_lock); - /* Start the kthread that handles upcalls. */ - error = kthread_add(clnt_vc_dotlsupcall, ct, - NULL, NULL, 0, 0, "krpctls%u", thrdnum++); - if (error != 0) - panic("Can't add KRPC thread error %d", error); + if (ct->ct_sslrefno != RPCTLS_REFNO_HANDSHAKE) { + mtx_unlock(&ct->ct_lock); + /* Start the kthread that handles upcalls. */ + error = kthread_add(clnt_vc_dotlsupcall, ct, + NULL, NULL, 0, 0, "krpctls%u", thrdnum++); + if (error != 0) + panic("Can't add KRPC thread error %d", error); + } else + mtx_unlock(&ct->ct_lock); return (TRUE); case CLSET_BLOCKRCV: @@ -892,24 +895,22 @@ clnt_vc_destroy(CLIENT *cl) if (so) { if (ct->ct_sslrefno != 0) { /* - * If the upcall fails, the socket has - * probably been closed via the rpctlscd - * daemon having crashed or been - * restarted, so ignore return stat. + * If the TLS handshake is in progress, the upcall + * will fail, but the socket should be closed by the + * daemon, since the connect upcall has just failed. */ - stat = rpctls_cl_disconnect(ct->ct_sslsec, - ct->ct_sslusec, ct->ct_sslrefno, - &reterr); - } else if ((ct->ct_rcvstate & RPCRCVSTATE_TLSHANDSHAKE) == 0) { - /* - * If the TLS handshake is in progress, leave the - * socket so that it will closed by the daemon. - * This can only occur if the daemon is waiting for - * an openssl call like SSL_connect() for a long - * time. The call will normally eventually fail and - * then the daemon will close the socket, so do not - * do it here. - */ + if (ct->ct_sslrefno != RPCTLS_REFNO_HANDSHAKE) { + /* + * If the upcall fails, the socket has + * probably been closed via the rpctlscd + * daemon having crashed or been + * restarted, so ignore return stat. + */ + stat = rpctls_cl_disconnect(ct->ct_sslsec, + ct->ct_sslusec, ct->ct_sslrefno, + &reterr); + } + } else { soshutdown(so, SHUT_WR); soclose(so); } @@ -1293,7 +1294,8 @@ clnt_vc_dotlsupcall(void *data) if ((ct->ct_rcvstate & RPCRCVSTATE_UPCALLNEEDED) != 0) { ct->ct_rcvstate &= ~RPCRCVSTATE_UPCALLNEEDED; ct->ct_rcvstate |= RPCRCVSTATE_UPCALLINPROG; - if (ct->ct_sslrefno != 0) { + if (ct->ct_sslrefno != 0 && ct->ct_sslrefno != + RPCTLS_REFNO_HANDSHAKE) { mtx_unlock(&ct->ct_lock); printf("at handlerecord\n"); ret = rpctls_cl_handlerecord(ct->ct_sslsec, Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Wed Jul 1 01:12:23 2020 (r362819) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Wed Jul 1 02:11:15 2020 (r362820) @@ -78,6 +78,9 @@ bool rpctls_getinfo(u_int *maxlen, bool rpctlscd_run, /* String for AUTH_TLS reply verifier. */ #define RPCTLS_START_STRING "STARTTLS" +/* ssl refno value to indicate TLS handshake being done. */ +#define RPCTLS_REFNO_HANDSHAKE 0xFFFFFFFFFFFFFFFFULL + #endif /* _KERNEL */ #endif /* _RPC_RPCSEC_TLS_H_ */ From owner-svn-src-projects@freebsd.org Wed Jul 1 02:20:46 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BB8B435B29A for ; Wed, 1 Jul 2020 02:20:46 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49xQ1n33SPz4DVW; Wed, 1 Jul 2020 02:20:45 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from kduck.mit.edu ([24.16.140.251]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 0612Kfvo024033 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 30 Jun 2020 22:20:43 -0400 Date: Tue, 30 Jun 2020 19:20:40 -0700 From: Benjamin Kaduk To: Rick Macklem Cc: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Message-ID: <20200701022040.GE58278@kduck.mit.edu> References: <202006301449.05UEnq2x072917@repo.freebsd.org> <20200630163340.GN58278@kduck.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 49xQ1n33SPz4DVW X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of kaduk@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=kaduk@mit.edu X-Spamd-Result: default: False [-2.37 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[24.16.140.251:received]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:18.9.28.0/24]; NEURAL_HAM_LONG(-1.03)[-1.029]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[mit.edu]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.15)[0.154]; RCVD_IN_DNSWL_MED(-0.20)[18.9.28.11:from]; NEURAL_HAM_MEDIUM(-0.99)[-0.994]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3, ipnet:18.9.0.0/16, country:US]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RWL_MAILSPIKE_VERYGOOD(0.00)[18.9.28.11:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 02:20:46 -0000 On Wed, Jul 01, 2020 at 01:23:50AM +0000, Rick Macklem wrote: > Rick Macklem wrote: > >Benjamin Kaduk wrote: > >>On Tue, Jun 30, 2020 at 04:20:45PM +0000, Rick Macklem wrote: > >>> If you happen to know how to set a timeout for SSL_connect() in the openssl > >>> library, I would be interested in hearing that. > >> > >>As it happens, I took a look before I wrote the initial note, and there > >>doesn't seem to be any intrinsic TLS (not DTLS) handshake timeouts in > >>libssl itself; I expect this is actually just the (kernel's!) TCP timeout. > >>So you'd be getting the socket fd (e.g., SSL_get_fd(), if you don't have a > >>reference already) and using setsockopt() to set the timeout(s). > >Interesting. The test case I simulated did not close the TCP socket used by > >SSL_connect(). The server just replied to the STARTTLS Null RPC, but did not > >call SSL_accept(), so the server side just isn't playing "handshake". > >"netstat -a" showed the connection as ESTABLISHED. > >During debugging, I also used the trick of putting: > > while (1) > > sleep(1); > >right after the SSL_connect() call and, when watching it via "ps", > >it would switch from "sbwait" to "nanoslp" after 6 minutes and > >a syslog() call showed that SSL_connect() had returned -1. > > > >So, if the TCP connection was "established", what caused the SSL_connect() > >to return with an error (-1) after 6 minutes? > > > >Now, there is a 6 minute idle timeout in the RPC code for TCP where it, > >by default, closes the connection when there is 6 minutes without any > >activity. (I have to look if waiting for a reply for the upcall implies "no activity" and >if > >this also happens for AF_LOCAL sockets, which is what the upcalls use.) > Ok, I figured out what is happening for this test. > It is the 6 minute idle timeout, but it occurs at the server end, where the NFS server > end shuts down the TCP connection. Ah, that makes sense. > Now, the client cannot assume all servers will do this. Right. > I'm going to try playing around with doing a shutdown of the socket on the > client end after a shorter timeout on the upcall and see if that can get > SSL_connect() to return with a failure in the daemon. > > >Now, if that happens, a SIGPIPE would be posted to the daemon, which > >is SIG_IGN'd by the daemon. But maybe the SIGPIPE somehow causes > >SSL_connect() to return -1 by making the syscall it is doing (read/recv on the > >TCP socket sitting in sbwait) return EINTR, or something like that? > Ignore this "theory". It was bunk. Non-ignored signals would cause SSL_connect() to return, but ignored ones should be wholly ignored, yes. > >I can change this 6minute timeout to see if that affects it. > Can't be changed, since it is at the server end of the TCP connection. Can't you set a client-side (e.g., read) timeout, though? -Ben > (A comment in the krpc code mentions a 5minute timeout in the client, > but I don't see that in the code?) > > >When you've got upcalls and library functions both talking to sockets it > >can get interesting. > > > >Thanks for the comments, rick > > Correcting myself, rick > > -Ben > From owner-svn-src-projects@freebsd.org Wed Jul 1 04:12:42 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1889A35CF9F for ; Wed, 1 Jul 2020 04:12:42 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xSVx6syvz4KKr; Wed, 1 Jul 2020 04:12:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CF98123703; Wed, 1 Jul 2020 04:12:41 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0614Cf3v074507; Wed, 1 Jul 2020 04:12:41 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0614Cft0074505; Wed, 1 Jul 2020 04:12:41 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202007010412.0614Cft0074505@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 1 Jul 2020 04:12:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362826 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/rpc: . rpcsec_tls X-SVN-Commit-Revision: 362826 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 04:12:42 -0000 Author: rmacklem Date: Wed Jul 1 04:12:41 2020 New Revision: 362826 URL: https://svnweb.freebsd.org/changeset/base/362826 Log: Thanks to Ben Kaduk's comments, I looked and the 6 minute timeout was actually the NFS server closing the TCP connection. As such, this patch reduces the timeout to 15sec and does a shutdown(SHUT_RD) of the socket so that SSL_connect() in the daemon will fail and return an error. It does the same thing for the server, although I have not been able to get the server side daemon stuck in the openssl library. It has been at least tested by faking a RPC_TIMEDOUT return from the upcall. Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Modified: projects/nfs-over-tls/sys/rpc/clnt_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/clnt_vc.c Wed Jul 1 04:00:54 2020 (r362825) +++ projects/nfs-over-tls/sys/rpc/clnt_vc.c Wed Jul 1 04:12:41 2020 (r362826) @@ -831,8 +831,10 @@ clnt_vc_close(CLIENT *cl) mtx_unlock(&ct->ct_lock); SOCKBUF_LOCK(&ct->ct_socket->so_rcv); - soupcall_clear(ct->ct_socket, SO_RCV); - clnt_vc_upcallsdone(ct); + if (ct->ct_socket->so_rcv.sb_upcall != NULL) { + soupcall_clear(ct->ct_socket, SO_RCV); + clnt_vc_upcallsdone(ct); + } SOCKBUF_UNLOCK(&ct->ct_socket->so_rcv); /* Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Wed Jul 1 04:00:54 2020 (r362825) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Wed Jul 1 04:12:41 2020 (r362826) @@ -160,19 +160,16 @@ printf("got cl=%p\n", cl); * RPC occur. Since it is an upcall to a local daemon, * requests should not be lost and doing one of these * RPCs multiple times is not correct. - * SSL_connect() in the openssl library has been - * observed to take 6 minutes when the server is not - * responding to the handshake records, so set the - * timeout to 10min. If it times out before the - * daemon completes the RPC, that should still be ok, - * since the daemon is single threaded and will not - * do further RPCs until the openssl library call - * returns (usually with a failure). + * If the server is not working correctly, the + * daemon can get stuck in SSL_connect() trying + * to read data from the socket during the upcall. + * Set a timeout (currently 15sec) and assume the + * daemon is hung when the timeout occurs. */ if (cl != NULL) { try_count = 1; CLNT_CONTROL(cl, CLSET_RETRIES, &try_count); - timeo.tv_sec = 10 * 60; + timeo.tv_sec = 15; timeo.tv_usec = 0; CLNT_CONTROL(cl, CLSET_TIMEOUT, &timeo); } else @@ -222,19 +219,13 @@ printf("got cl=%p\n", cl); * RPC occur. Since it is an upcall to a local daemon, * requests should not be lost and doing one of these * RPCs multiple times is not correct. - * SSL_connect() in the openssl library has been - * observed to take 6 minutes when the server is not - * responding to the handshake records, so set the - * timeout to 10min. If it times out before the - * daemon completes the RPC, that should still be ok, - * since the daemon is single threaded and will not - * do further RPCs until the openssl library call - * returns (usually with a failure). + * Set a timeout (currently 15sec) and assume that + * the daemon is hung if a timeout occurs. */ if (cl != NULL) { try_count = 1; CLNT_CONTROL(cl, CLSET_RETRIES, &try_count); - timeo.tv_sec = 10 * 60; + timeo.tv_sec = 15; timeo.tv_usec = 0; CLNT_CONTROL(cl, CLSET_TIMEOUT, &timeo); } else @@ -413,6 +404,15 @@ printf("aft connect upcall=%d\n", stat); *sslp++ = res.usec; *sslp = res.ssl; } + } else if (stat == RPC_TIMEDOUT) { + /* + * Do a shutdown on the socket, since the daemon is probably + * stuck in SSL_connect() trying to read the socket. + * Do not soclose() the socket, since the daemon will close() + * the socket after SSL_connect() returns an error. + */ + soshutdown(so, SHUT_RD); +printf("did soshutdown rd\n"); } CLNT_RELEASE(cl); @@ -595,6 +595,15 @@ printf("got uid=%d ngrps=%d gidv=%p gids=%p\n", *uid, for (i = 0; i < *ngrps; i++) *gidp++ = *gidv++; } + } else if (stat == RPC_TIMEDOUT) { + /* + * Do a shutdown on the socket, since the daemon is probably + * stuck in SSL_accept() trying to read the socket. + * Do not soclose() the socket, since the daemon will close() + * the socket after SSL_accept() returns an error. + */ + soshutdown(so, SHUT_RD); +printf("did soshutdown rd\n"); } printf("aft server upcall stat=%d flags=0x%x\n", stat, res.flags); CLNT_RELEASE(cl); @@ -695,7 +704,8 @@ printf("authtls: null reply=%d\n", call_stat); xprt->xp_gidp = gidp; printf("got uid=%d ngrps=%d gidp=%p\n", uid, ngrps, gidp); } - } + } else if (stat == RPC_TIMEDOUT) + xprt->xp_upcallset = 0; /* upcall cleared by soshutdown(). */ sx_xunlock(&xprt->xp_lock); xprt_active(xprt); /* Harmless if already active. */ printf("authtls: aft handshake stat=%d\n", stat); From owner-svn-src-projects@freebsd.org Wed Jul 1 21:19:34 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C4CB35B9FE for ; Wed, 1 Jul 2020 21:19:34 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xvHn6Wx5z3Y6t; Wed, 1 Jul 2020 21:19:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AA069FC80; Wed, 1 Jul 2020 21:19:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 061LJXpb007674; Wed, 1 Jul 2020 21:19:33 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 061LJXeL007672; Wed, 1 Jul 2020 21:19:33 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202007012119.061LJXeL007672@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 1 Jul 2020 21:19:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362865 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/nfs-over-tls/sys/rpc: . rpcsec_tls X-SVN-Commit-Revision: 362865 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 21:19:34 -0000 Author: rmacklem Date: Wed Jul 1 21:19:32 2020 New Revision: 362865 URL: https://svnweb.freebsd.org/changeset/base/362865 Log: Add a new xp_tls flag to indicate handshake failure. This new flag is used to disable the kernel code from closing the socket upon handshake failure, so that the daemon can close it once SSL_accept() has returned failure. Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls.h projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c projects/nfs-over-tls/sys/rpc/svc_vc.c Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls.h ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Wed Jul 1 20:45:26 2020 (r362864) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls.h Wed Jul 1 21:19:32 2020 (r362865) @@ -48,6 +48,7 @@ int rpctls_syscall(int, const char *); #define RPCTLS_FLAGS_VERIFIED 0x08 #define RPCTLS_FLAGS_DISABLED 0x10 #define RPCTLS_FLAGS_CERTUSER 0x20 +#define RPCTLS_FLAGS_HANDSHFAIL 0x40 /* Error return values for upcall rpcs. */ #define RPCTLSERR_OK 0 Modified: projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Wed Jul 1 20:45:26 2020 (r362864) +++ projects/nfs-over-tls/sys/rpc/rpcsec_tls/rpctls_impl.c Wed Jul 1 21:19:32 2020 (r362865) @@ -704,8 +704,10 @@ printf("authtls: null reply=%d\n", call_stat); xprt->xp_gidp = gidp; printf("got uid=%d ngrps=%d gidp=%p\n", uid, ngrps, gidp); } - } else if (stat == RPC_TIMEDOUT) - xprt->xp_upcallset = 0; /* upcall cleared by soshutdown(). */ + } else { + /* Mark that TLS handshake failed. */ + xprt->xp_tls = RPCTLS_FLAGS_HANDSHFAIL; + } sx_xunlock(&xprt->xp_lock); xprt_active(xprt); /* Harmless if already active. */ printf("authtls: aft handshake stat=%d\n", stat); Modified: projects/nfs-over-tls/sys/rpc/svc_vc.c ============================================================================== --- projects/nfs-over-tls/sys/rpc/svc_vc.c Wed Jul 1 20:45:26 2020 (r362864) +++ projects/nfs-over-tls/sys/rpc/svc_vc.c Wed Jul 1 21:19:32 2020 (r362865) @@ -455,18 +455,20 @@ svc_vc_destroy_common(SVCXPRT *xprt) uint32_t reterr; if (xprt->xp_socket) { - if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0) { + if ((xprt->xp_tls & (RPCTLS_FLAGS_HANDSHAKE | + RPCTLS_FLAGS_HANDSHFAIL)) == 0) + (void)soclose(xprt->xp_socket); + else if ((xprt->xp_tls & RPCTLS_FLAGS_HANDSHAKE) != 0) { /* * If the upcall fails, the socket has * probably been closed via the rpctlssd * daemon having crashed or been - * restarted. + * restarted, so just ignore returned stat. */ stat = rpctls_srv_disconnect(xprt->xp_sslsec, xprt->xp_sslusec, xprt->xp_sslrefno, &reterr); - } else - (void)soclose(xprt->xp_socket); + } } if (xprt->xp_netid) From owner-svn-src-projects@freebsd.org Wed Jul 1 21:21:26 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1226635C309 for ; Wed, 1 Jul 2020 21:21:26 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xvKx5zYlz3Xg3; Wed, 1 Jul 2020 21:21:25 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A7694FC10; Wed, 1 Jul 2020 21:21:25 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 061LLPps010896; Wed, 1 Jul 2020 21:21:25 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 061LLP8L010895; Wed, 1 Jul 2020 21:21:25 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202007012121.061LLP8L010895@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 1 Jul 2020 21:21:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362866 - projects/nfs-over-tls/usr.sbin/rpctlssd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/usr.sbin/rpctlssd X-SVN-Commit-Revision: 362866 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 21:21:26 -0000 Author: rmacklem Date: Wed Jul 1 21:21:25 2020 New Revision: 362866 URL: https://svnweb.freebsd.org/changeset/base/362866 Log: Clarify a comment related to closing a socket after handshake failure. Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Modified: projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Wed Jul 1 21:19:32 2020 (r362865) +++ projects/nfs-over-tls/usr.sbin/rpctlssd/rpctlssd.c Wed Jul 1 21:21:25 2020 (r362866) @@ -382,7 +382,7 @@ rpctlssd_verbose_out("rpctlsd_connect_svc s=%d\n", s); "accept failed\n"); /* * For RPC-over-TLS, this upcall is expected - * to close off the socket. + * to close off the socket upon handshake failure. */ close(s); return (FALSE); From owner-svn-src-projects@freebsd.org Wed Jul 1 21:27:33 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E89AE35C33A for ; Wed, 1 Jul 2020 21:27:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xvT15gy9z3YCm; Wed, 1 Jul 2020 21:27:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A654BFBC6; Wed, 1 Jul 2020 21:27:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 061LRX9K013767; Wed, 1 Jul 2020 21:27:33 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 061LRXS5013766; Wed, 1 Jul 2020 21:27:33 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202007012127.061LRXS5013766@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 1 Jul 2020 21:27:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r362867 - projects/nfs-over-tls/usr.sbin/rpctlscd X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/usr.sbin/rpctlscd X-SVN-Commit-Revision: 362867 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 21:27:34 -0000 Author: rmacklem Date: Wed Jul 1 21:27:33 2020 New Revision: 362867 URL: https://svnweb.freebsd.org/changeset/base/362867 Log: Get rid of a shutdown() call, since it is now done in the kernel for this case. While here, delete a few debug output lines and fix a wraparound line. Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Modified: projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c ============================================================================== --- projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Wed Jul 1 21:21:25 2020 (r362866) +++ projects/nfs-over-tls/usr.sbin/rpctlscd/rpctlscd.c Wed Jul 1 21:27:33 2020 (r362867) @@ -318,7 +318,6 @@ rpctlscd_connect_1_svc(void *argp, rpctlscd_verbose_out("rpctlsd_connect: started\n"); /* Get the socket fd from the kernel. */ s = rpctls_syscall(RPCTLS_SYSC_CLSOCKET, ""); -rpctlscd_verbose_out("rpctlsd_connect s=%d\n", s); if (s < 0) { result->reterr = RPCTLSERR_NOSOCKET; return (TRUE); @@ -345,7 +344,6 @@ rpctlscd_verbose_out("rpctlsd_connect s=%d\n", s); * For RPC-over-TLS, this upcall is expected * to close off the socket. */ - shutdown(s, SHUT_WR); close(s); return (TRUE); } @@ -387,7 +385,6 @@ rpctlscd_handlerecord_1_svc(struct rpctlscd_handlereco if (ret <= 0) { /* Check to see if this was a close alert. */ ret = SSL_get_shutdown(slp->ssl); -rpctlscd_verbose_out("get_shutdown2=%d\n", ret); if ((ret & (SSL_SENT_SHUTDOWN | SSL_RECEIVED_SHUTDOWN)) == SSL_RECEIVED_SHUTDOWN) SSL_shutdown(slp->ssl); @@ -424,7 +421,6 @@ rpctlscd_disconnect_1_svc(struct rpctlscd_disconnect_a slp->s); LIST_REMOVE(slp, next); ret = SSL_get_shutdown(slp->ssl); -rpctlscd_verbose_out("get_shutdown0=%d\n", ret); /* * Do an SSL_shutdown() unless a close alert has * already been sent. @@ -660,7 +656,8 @@ rpctls_connect(SSL_CTX *ctx, int s) rpctlscd_verbose_out("rpctls_connect: BIO_get_ktls_send=%d\n", ret); if (ret != 0) { ret = BIO_get_ktls_recv(SSL_get_rbio(ssl)); - rpctlscd_verbose_out("rpctls_connect: BIO_get_ktls_recv=%d\n", ret); + rpctlscd_verbose_out("rpctls_connect: BIO_get_ktls_recv=%d\n", + ret); } if (ret == 0) { if (rpctls_debug_level == 0) From owner-svn-src-projects@freebsd.org Wed Jul 1 22:47:31 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8512335DB3E for ; Wed, 1 Jul 2020 22:47:31 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on0630.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::630]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xxFG2JBWz3fPR; Wed, 1 Jul 2020 22:47:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n6b9R8ofTUaaGXoHHCu7TumjjrbRoc+qcnE4OirDrW2OmPB/ewL9pbxzirYPp6y/Pl+QDzruCo/oYEKypk1MHRl+w2ycng0TsWaeLVQtQNsg9wwTPF+jq9zkg7hkWKuMLpc6gwvCa0bE752Dw+Cq+5YCqkGX1y8gmZYeI3ekOW01TzC0te8EESLGBTPn7hfjOTdIP/raEPNb0I+6dCi5UV2TZPe1KNCXtJsku7rZYy2K+hBrdZQu/6F+kAQ0f3RvTnsCM9vVU2GGN1VUcv6t/EVZaayYhknbg/hrDcOWD7FJ5Ra8fySajKCja9P2ugVOBc3QX9XgBh2Uw8tV3NLEkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2561Kt0zqN4iBHrO6VEcWZo21ogOSSvow2fiXr6FD9M=; b=W/Y11QHKgf7inllUPGPqPIIbvVXDgqnLgmH9DRhN3dXFKS6VodCA0sSNjupGM5cZEXsXPhP0dz1ybKj5raZ5u5Fue25uF4aZKpKzw6Go7ayzZknEQJ183kzlQW8Mrvkcx86/ZMacruVRE2vtx+FTZ/ahTa8wImGgntik49i4qxk9rl722HAMqQQ4H0SKzqCy/tHBSz8c8Iy5D7mwX1PCBZuwf4/hZNwWnSCokKJGMA6pzO3AE4dq4ADwL+ObUMyW7DjctQD1hKKbpx7gtjSxJ1iM246bXRS+fkeK+9M+2XY9WNQRoAf8cB9PSfeVbV4w+GhdFe87tXapw5XaAo0d9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2561Kt0zqN4iBHrO6VEcWZo21ogOSSvow2fiXr6FD9M=; b=lrVADFL41Uvp4uiFluxPuTezA5xuztv0Rsv0R4l6BEjwnzqGKnm6NuLXjhP5xbdwQuMhW/qSzldNVMCweA+MI9ssAX1z1xSApYEAYJPFybP7m1JOXoiG74eT6A3sCqZKzBQM+xw+2g5G/p1mWsSuRM3isV2+Qtplg8nqVb5namzcDd6BhuJ/6HnjF9RcXI9I6cfA/66ZsKRjJhU6hP9TJB/HUC5Pvgo1vmuE3DRH9SDpQOrWmMeCa7547jee/oKgod/xRPOAcnYcjAQolggBmoisbj6TG4cu+wlkn2hdRcCSToqPSgmXnY0It+CsSeIhgr0YdebsKsUL2JxZ9mPF4g== Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:38::14) by YQBPR0101MB1939.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:f::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3153.23; Wed, 1 Jul 2020 22:47:27 +0000 Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91]) by QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91%7]) with mapi id 15.20.3131.034; Wed, 1 Jul 2020 22:47:19 +0000 From: Rick Macklem To: Benjamin Kaduk CC: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Topic: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Index: AQHWTu29MoFCP0YO4kaq4Sj4aYWIUajxUeOAgAAARmyAAAhnAIAAg0Q8gAANlPuAABMqAIABUrap Date: Wed, 1 Jul 2020 22:47:19 +0000 Message-ID: References: <202006301449.05UEnq2x072917@repo.freebsd.org> <20200630163340.GN58278@kduck.mit.edu> , <20200701022040.GE58278@kduck.mit.edu> In-Reply-To: <20200701022040.GE58278@kduck.mit.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3a9a8693-8ace-48f2-8d2a-08d81e10b5dc x-ms-traffictypediagnostic: YQBPR0101MB1939: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2512; x-forefront-prvs: 04519BA941 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Zgp2R3Gpjp1kOesdg6MKIIwRPXQ3F1wc2CKnZlu5+EEphx/pcgi1MjPoCOVBqMkLFobNhlZuy/XHUp+3TZhXC2+eaXmucJ031B7zk7n+YIaDEjNk03bfRiqeJGCGdqfvzdpccn2bm42GZ+JR7N8qhMFg77XQv2j4BjAaHOPIoC4H0hwqirDTaNlzeSamDmHU1/3HTH+iQye6T/uYcFuzlFi7I0yjYnyPknT3sorMYf8GqQEkWbeiIdYa+AYH6OyWoYJ7hGNhxmO5SIB9MsruXlOXSNYhEz87Vw5iYWNBZinkJ463DlY8JbshLGL8jDaNn4xgV7Mz0K7I5Qy+sD1tFA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(396003)(366004)(136003)(346002)(376002)(39860400002)(83380400001)(5660300002)(478600001)(6506007)(7696005)(66946007)(66446008)(8936002)(64756008)(66476007)(4326008)(33656002)(66556008)(8676002)(76116006)(2906002)(71200400001)(55016002)(186003)(86362001)(786003)(316002)(6916009)(52536014)(54906003)(9686003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: wvopeOYFoC6e4gPYlEJCKQcwRjcSdyDgTNRcDHCX1r+Wr+49JCq1f4bXZFW7LDLlf3rzEPph2ktaysTSD2Z/hXAT+PgsSOgsQDFOXNcuBqq/Oh/MSFQpo8frb2AmbipkYfX4O7/6h+3ImBkGpvatakUzChJ3ZkYHo1Uxo4QaaKYjunZ/EKs1l4QZDvKGmP1ImcQOwm0/98n0jBL1NU4VLDAdZMN+KuHFsoYdNsnvtRoMoeUmGBbRhi67iE6tKyq/ukTRgdWgdLo14pX6KgjuedVF9r1gl0FTRlRM8pJ5jckqpW7hrrj7M9wKwZWo8Mw/lF9n4i0KZvM/peJzTMnSDwI1ePMZ+RgYjKJ/i/3UbheEu8t7Pl5RNsFoQkXbqQmsWxwXX+TvQTi5ZAQV4l+6aH/QCrpJm9HSqA+NourDYd6PwHOUXNh602X9PwIuuOakDccS4OLowTgyQigxvDO2Ue2Ri446tyEVH26uKVW6Mz5Wj0fth4vsdTu1pRdSI2QWNLEdLT7HLU4gIwhyGFCU015ykbZQqhMn4Mlg6Im/QEMGgiE9WZ1jfiugW7n/CAAM x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 3a9a8693-8ace-48f2-8d2a-08d81e10b5dc X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2020 22:47:19.1176 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: TTosy5HwhkIH2wc/nKieXvhGOw1xFe6VW/9lf62YhML88y5NeR496T3DWORjKB6m1o0xDvAzdeBWNrXhg1u1mA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB1939 X-Rspamd-Queue-Id: 49xxFG2JBWz3fPR X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=lrVADFL4; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::630 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.83 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.003]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_LONG(-1.00)[-1.004]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; RCPT_COUNT_FIVE(0.00)[5]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; NEURAL_HAM_SHORT(-0.33)[-0.327]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 22:47:31 -0000 Benjamin Kaduk wrote:=0A= >On Wed, Jul 01, 2020 at 01:23:50AM +0000, Rick Macklem wrote:=0A= >> Rick Macklem wrote:=0A= >> >Benjamin Kaduk wrote:=0A= >> >>On Tue, Jun 30, 2020 at 04:20:45PM +0000, Rick Macklem wrote:=0A= >> >>> If you happen to know how to set a timeout for SSL_connect() in the = openssl=0A= >> >>> library, I would be interested in hearing that.=0A= >> >>=0A= >> >>As it happens, I took a look before I wrote the initial note, and ther= e=0A= >> >>doesn't seem to be any intrinsic TLS (not DTLS) handshake timeouts in= =0A= >> >>libssl itself; I expect this is actually just the (kernel's!) TCP time= out.=0A= >> >>So you'd be getting the socket fd (e.g., SSL_get_fd(), if you don't ha= ve a=0A= >> >>reference already) and using setsockopt() to set the timeout(s).=0A= >> >Interesting. The test case I simulated did not close the TCP socket use= d by=0A= >> >SSL_connect(). The server just replied to the STARTTLS Null RPC, but di= d not=0A= >> >call SSL_accept(), so the server side just isn't playing "handshake".= =0A= >> >"netstat -a" showed the connection as ESTABLISHED.=0A= >> >During debugging, I also used the trick of putting:=0A= >> > while (1)=0A= >> > sleep(1);=0A= >> >right after the SSL_connect() call and, when watching it via "ps",=0A= >> >it would switch from "sbwait" to "nanoslp" after 6 minutes and=0A= >> >a syslog() call showed that SSL_connect() had returned -1.=0A= >> >=0A= >> >So, if the TCP connection was "established", what caused the SSL_connec= t()=0A= >> >to return with an error (-1) after 6 minutes?=0A= >> >=0A= >> >Now, there is a 6 minute idle timeout in the RPC code for TCP where it,= =0A= >> >by default, closes the connection when there is 6 minutes without any= =0A= >> >activity. (I have to look if waiting for a reply for the upcall implies= "no activity" and >if=0A= >> >this also happens for AF_LOCAL sockets, which is what the upcalls use.)= =0A= >> Ok, I figured out what is happening for this test.=0A= >> It is the 6 minute idle timeout, but it occurs at the server end, where = the NFS server=0A= >> end shuts down the TCP connection.=0A= >=0A= >Ah, that makes sense.=0A= >=0A= >> Now, the client cannot assume all servers will do this.=0A= >=0A= >Right.=0A= >=0A= >> I'm going to try playing around with doing a shutdown of the socket on t= he=0A= >> client end after a shorter timeout on the upcall and see if that can get= =0A= >> SSL_connect() to return with a failure in the daemon.=0A= >>=0A= >> >Now, if that happens, a SIGPIPE would be posted to the daemon, which=0A= >> >is SIG_IGN'd by the daemon. But maybe the SIGPIPE somehow causes=0A= >> >SSL_connect() to return -1 by making the syscall it is doing (read/recv= on the=0A= >> >TCP socket sitting in sbwait) return EINTR, or something like that?=0A= >> Ignore this "theory". It was bunk.=0A= >=0A= >Non-ignored signals would cause SSL_connect() to return, but ignored ones= =0A= >should be wholly ignored, yes.=0A= >=0A= >> >I can change this 6minute timeout to see if that affects it.=0A= >> Can't be changed, since it is at the server end of the TCP connection.= =0A= >=0A= >Can't you set a client-side (e.g., read) timeout, though?=0A= Well, in this case it would be the read (or recv or ??) that is done inside= the=0A= SSL_connect().=0A= =0A= The timer I can control is the one that I had set to 10minutes, which times= out=0A= the upcall RPC to the userland daemon. I had set it to 10minutes so the=0A= SSL_connect() would time out first, but now that I know that won't always h= appen..=0A= This timer is now set to 15sec and after it times out, the kernel code does= a=0A= soshutdown(so, SHUT_RD) in the client, which seems to be sufficient to get= =0A= SSL_connect() to return an error.=0A= =0A= This seems sufficient and works ok for the testing I've done.=0A= =0A= 15sec is pretty arbitrary, but I figure a timeout on the order of seconds i= s=0A= reasonable for RPC upcalls to the local daemon. (I'd guess that taking even= =0A= 1sec to do an upcall would indicate something is broken.)=0A= If others feel 15sec isn't an appropriate timeout, feel free to comment.=0A= (Note that this timeout should only happen when something is broken, like= =0A= the server that does a "STARTTLS" reply but does not do a TLS handshake.)= =0A= =0A= Thanks for the comments, rick=0A= =0A= to return an error.=0A= -Ben=0A= =0A= > (A comment in the krpc code mentions a 5minute timeout in the client,=0A= > but I don't see that in the code?)=0A= >=0A= > >When you've got upcalls and library functions both talking to sockets it= =0A= > >can get interesting.=0A= > >=0A= > >Thanks for the comments, rick=0A= >=0A= > Correcting myself, rick=0A= >=0A= > -Ben=0A= >=0A= =0A= From owner-svn-src-projects@freebsd.org Wed Jul 1 22:50:17 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F19635DC16 for ; Wed, 1 Jul 2020 22:50:17 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49xxJS62mRz3fjF; Wed, 1 Jul 2020 22:50:16 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from kduck.mit.edu ([24.16.140.251]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 061MoCii024955 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 1 Jul 2020 18:50:14 -0400 Date: Wed, 1 Jul 2020 15:50:11 -0700 From: Benjamin Kaduk To: Rick Macklem Cc: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Message-ID: <20200701225011.GH58278@kduck.mit.edu> References: <202006301449.05UEnq2x072917@repo.freebsd.org> <20200630163340.GN58278@kduck.mit.edu> <20200701022040.GE58278@kduck.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 49xxJS62mRz3fjF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of kaduk@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=kaduk@mit.edu X-Spamd-Result: default: False [-2.64 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[24.16.140.251:received]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:18.9.28.0/24]; NEURAL_HAM_LONG(-0.99)[-0.986]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[mit.edu]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[18.9.28.11:from]; NEURAL_HAM_SHORT(-0.16)[-0.163]; NEURAL_HAM_MEDIUM(-0.99)[-0.992]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3, ipnet:18.9.0.0/16, country:US]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RWL_MAILSPIKE_VERYGOOD(0.00)[18.9.28.11:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 22:50:17 -0000 On Wed, Jul 01, 2020 at 10:47:19PM +0000, Rick Macklem wrote: > Benjamin Kaduk wrote: > >On Wed, Jul 01, 2020 at 01:23:50AM +0000, Rick Macklem wrote: > >> Rick Macklem wrote: > >> >Benjamin Kaduk wrote: > >> >>On Tue, Jun 30, 2020 at 04:20:45PM +0000, Rick Macklem wrote: > >> >>> If you happen to know how to set a timeout for SSL_connect() in the openssl > >> >>> library, I would be interested in hearing that. > >> >> > >> >>As it happens, I took a look before I wrote the initial note, and there > >> >>doesn't seem to be any intrinsic TLS (not DTLS) handshake timeouts in > >> >>libssl itself; I expect this is actually just the (kernel's!) TCP timeout. > >> >>So you'd be getting the socket fd (e.g., SSL_get_fd(), if you don't have a > >> >>reference already) and using setsockopt() to set the timeout(s). > >> >Interesting. The test case I simulated did not close the TCP socket used by > >> >SSL_connect(). The server just replied to the STARTTLS Null RPC, but did not > >> >call SSL_accept(), so the server side just isn't playing "handshake". > >> >"netstat -a" showed the connection as ESTABLISHED. > >> >During debugging, I also used the trick of putting: > >> > while (1) > >> > sleep(1); > >> >right after the SSL_connect() call and, when watching it via "ps", > >> >it would switch from "sbwait" to "nanoslp" after 6 minutes and > >> >a syslog() call showed that SSL_connect() had returned -1. > >> > > >> >So, if the TCP connection was "established", what caused the SSL_connect() > >> >to return with an error (-1) after 6 minutes? > >> > > >> >Now, there is a 6 minute idle timeout in the RPC code for TCP where it, > >> >by default, closes the connection when there is 6 minutes without any > >> >activity. (I have to look if waiting for a reply for the upcall implies "no activity" and >if > >> >this also happens for AF_LOCAL sockets, which is what the upcalls use.) > >> Ok, I figured out what is happening for this test. > >> It is the 6 minute idle timeout, but it occurs at the server end, where the NFS server > >> end shuts down the TCP connection. > > > >Ah, that makes sense. > > > >> Now, the client cannot assume all servers will do this. > > > >Right. > > > >> I'm going to try playing around with doing a shutdown of the socket on the > >> client end after a shorter timeout on the upcall and see if that can get > >> SSL_connect() to return with a failure in the daemon. > >> > >> >Now, if that happens, a SIGPIPE would be posted to the daemon, which > >> >is SIG_IGN'd by the daemon. But maybe the SIGPIPE somehow causes > >> >SSL_connect() to return -1 by making the syscall it is doing (read/recv on the > >> >TCP socket sitting in sbwait) return EINTR, or something like that? > >> Ignore this "theory". It was bunk. > > > >Non-ignored signals would cause SSL_connect() to return, but ignored ones > >should be wholly ignored, yes. > > > >> >I can change this 6minute timeout to see if that affects it. > >> Can't be changed, since it is at the server end of the TCP connection. > > > >Can't you set a client-side (e.g., read) timeout, though? > Well, in this case it would be the read (or recv or ??) that is done inside the > SSL_connect(). > > The timer I can control is the one that I had set to 10minutes, which times out > the upcall RPC to the userland daemon. I had set it to 10minutes so the > SSL_connect() would time out first, but now that I know that won't always happen.. > This timer is now set to 15sec and after it times out, the kernel code does a > soshutdown(so, SHUT_RD) in the client, which seems to be sufficient to get > SSL_connect() to return an error. > > This seems sufficient and works ok for the testing I've done. I don't think what you ended up with is wrong, to be clear. But, you have an SSL* as input to SSL_connect(), and you can call SSL_get_fd() on that SSL*, which will give you a socket fd that you can call setsockopt() on, if you're so inclined. The SSL_connect() abstraction barrier is not leak-proof :) > 15sec is pretty arbitrary, but I figure a timeout on the order of seconds is > reasonable for RPC upcalls to the local daemon. (I'd guess that taking even > 1sec to do an upcall would indicate something is broken.) > If others feel 15sec isn't an appropriate timeout, feel free to comment. > (Note that this timeout should only happen when something is broken, like > the server that does a "STARTTLS" reply but does not do a TLS handshake.) Understood. Thanks, Ben From owner-svn-src-projects@freebsd.org Wed Jul 1 23:21:11 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3187835E381 for ; Wed, 1 Jul 2020 23:21:11 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-qb1can01on060d.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5c::60d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49xy0618pbz3y0p; Wed, 1 Jul 2020 23:21:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H/nzfbVa7JpC1mNsb7XtfU7f5Pc0NpXD0CocFFFegmVaqO0huEUYDzkwbEkuFcChOZTcJXzDupalyPYJ/bJUK6Mipl0JHmReoYasDOpDI7D2l/xIYL6eDbSCVz5H/p4Z/hERMueVUfi3t0CaybOQMZeDW2knqadyjjSLIgFefQX4R8VHCAgqHasZ9p3xy+jHkI8CEo1E7T3j1QGUF0Xc+7cBq8tQw7aS8WCNXY8G74nA5x8jCkRVnP0vdHkvy3QPzP4vcPBc8+vDi3edqg5A54PLGecyt+8GeX3oZHvXI7Wb0LPFExRJWsKuu7d1+JWxWeBBjrVCKjUI3F+7vjP85Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xzf2TYlHFIzvOYJ2bWy5vk+/PJ7aCtYEaofGP7t+k6c=; b=gMkC9LkI0WOgmgFsx49IZvLkAMEvNHPCcenG8W7S4dFb07yEZ0FBpEuAWHl9zrimn5Qi7EpTbIwtL5UKzKbd9bzgUBPrHVgBjUuX9kQTA3l4Pi7Eh1Rs2mOmCLQ9KH9nQYmBVoOoi3gJRZEd0kIpYpG33XDhe4Y+MobXBNzv5zvEFTvsT9g8b7CLRmP1ZVKdqh4xfciXmmc9G8/yV6QdFQIKzwRbLs+yzhFdtGPslOw64q0qJeShsoN2T74+qTErIqLmHamavsWobKA+cAqY+rFHFtB2o1RYPJCO+YUNPpudSc9dJLyceyYW6CJBtmTff56lpVXtyowxQL4m+zYw7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xzf2TYlHFIzvOYJ2bWy5vk+/PJ7aCtYEaofGP7t+k6c=; b=S+ReE++aKJEm9Iu82fbkVqW/oH2D/L/Xio2ECIqZYkkluXv0zLkSSLKvJcgFQmYKwjPucT+Koxut02HPyMXciR4ImkihD+wJyPNTpBMo+ZRAi3ehJDVEqqtJil/FMid3ZzPFsFITYg6Mhl92T8BV6nOllvPVhWpWNVzSyMQPyK2r/PRE/afddGdfbjK8D+JvazMxlZ5ZfcBDGP1ncSJ+J9GF8/jjqNCy6uMJjJADTdsUd66L5IX8WPjX5pzMnFZKhyhha+rRXdYzus7VOtJpeGPsbmrqx9U1zhh83zw+zaAQFOxw1h5B59/v4J+TF3d9OjUar58X5SiVw1XeT4Wjfg== Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:38::14) by QB1PR01MB2882.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:36::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3153.20; Wed, 1 Jul 2020 23:21:07 +0000 Received: from QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91]) by QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM ([fe80::60f3:4ca2:8a4a:1e91%7]) with mapi id 15.20.3131.034; Wed, 1 Jul 2020 23:21:07 +0000 From: Rick Macklem To: Benjamin Kaduk CC: Benjamin Kaduk , Rick Macklem , src-committers , "svn-src-projects@freebsd.org" Subject: Re: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Topic: svn commit: r362798 - in projects/nfs-over-tls/sys/rpc: . rpcsec_tls Thread-Index: AQHWTu29MoFCP0YO4kaq4Sj4aYWIUajxUeOAgAAARmyAAAhnAIAAg0Q8gAANlPuAABMqAIABUrapgAAE0ICAAAH3+w== Date: Wed, 1 Jul 2020 23:21:07 +0000 Message-ID: References: <202006301449.05UEnq2x072917@repo.freebsd.org> <20200630163340.GN58278@kduck.mit.edu> <20200701022040.GE58278@kduck.mit.edu> , <20200701225011.GH58278@kduck.mit.edu> In-Reply-To: <20200701225011.GH58278@kduck.mit.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 269d0c7c-0885-4951-4003-08d81e156eb1 x-ms-traffictypediagnostic: QB1PR01MB2882: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2733; x-forefront-prvs: 04519BA941 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 5EB1QmQG3Tz9+oS17/lU5jCyopbfx9cGM6dPdB71j0RPlB2DMzbEkoCCB+0TXGrgN5XKW2V6n5TUsHWPsHYZvSp1JIqIjJIGitb1AuVIjalVZczierDplgJ0M5KKu9BDhpX3tct7UriMlApz8q1T618lXRMfRkxQkXuwFayYvZ9WnOFmfdCDHKhvVvTX35KfQZ90UAfkd3Zd2QFg2IEGEvgG+m9D3jx9qtsL7JpEh/ItDgvURHnBQEJiGevOe5yEqMOXl5HNkNNCoZ84YPp2HjxEn+BH+h/KveP3xVs3KqiZblE1j8gBiuoRW1ru9jFBibuyOr+FIjBXfM/Ew6MwOw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFTY:; SFS:(366004)(346002)(39850400004)(396003)(136003)(376002)(64756008)(66556008)(66476007)(66446008)(6506007)(9686003)(7696005)(478600001)(71200400001)(186003)(4326008)(83380400001)(86362001)(786003)(54906003)(316002)(8676002)(52536014)(55016002)(76116006)(91956017)(66946007)(5660300002)(33656002)(2906002)(6916009)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: IJU1gzjemOSQSjrhjlsifO07GQIL6IFZKTJ4+GGqvlzsw5eZD/qnd4sTesKvPX8SHZ1uFkMEeYPqGJb2amp08GrAjU8XmB9Jm3CLTc+CmMZ4Ggi9WLd0AbMCOpDTg+jNgF/IOVk0qBamIly/CfFpQIpy5Warrkb2c592engSzZTdRQmOqMxnZGBCBqhgO3lD3tWlHb3DqSxOyQI+xtHrSweZY+wy0t1FHwgOiVBAUZJ1AaBwIt4D9+mdPMVFz0JyuKOo0msEROuxLL1tp9j9MRJ7cKH0o2z3S1mJxn6LnjnfEuIAPU/UxjA50rlOx+OW5fxDVAkbkgX16RUA2dfDUzz5e2n+QRcvKbep8rUP3cANURfZHFxlc1ruA6t+HY2bThjKtr6H5IF6Wh6NZlEKrRU6w/T61yl9yrPYmKLKRT+U4iUOw0oPG2DKRrQrxLgESPfVKfxcJvPFZhT7A8wedv4wTl6FVJrtbwuiScqHwG6AtKJ6L64dzm8Dx9AYhKVecCgnpUzXnYnhSpSBbzvOoJNXShVEjR4u9jDEAaFFSDGzA5c4OqAA5Wr0ow1EBBXB x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: QB1PR01MB3364.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 269d0c7c-0885-4951-4003-08d81e156eb1 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2020 23:21:07.2552 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BV+7QvCQLriK/ALVSV1nNgNSsdG0ZX2BosoLzdIPxYwVMM0MuyOFOzLGd97e7evre2h8dl2yF1s015ahLRHC9Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB2882 X-Rspamd-Queue-Id: 49xy0618pbz3y0p X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=S+ReE++a; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5c::60d as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.84 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.005]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_LONG(-1.00)[-1.004]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; RCPT_COUNT_FIVE(0.00)[5]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; NEURAL_HAM_SHORT(-0.33)[-0.332]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; FREEMAIL_CC(0.00)[gmail.com,FreeBSD.org,freebsd.org]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 23:21:11 -0000 Benjamin Kaduk wrote:=0A= >Rick Macklem wrote:=0A= >> >Benjamin Kaduk wrote:=0A= [stuff snipped]=0A= >> >=0A= >> >Can't you set a client-side (e.g., read) timeout, though?=0A= >> Well, in this case it would be the read (or recv or ??) that is done ins= ide the=0A= >> SSL_connect().=0A= >>=0A= >> The timer I can control is the one that I had set to 10minutes, which ti= mes out=0A= >> the upcall RPC to the userland daemon. I had set it to 10minutes so the= =0A= >> SSL_connect() would time out first, but now that I know that won't alway= s happen..=0A= >> This timer is now set to 15sec and after it times out, the kernel code d= oes a=0A= >> soshutdown(so, SHUT_RD) in the client, which seems to be sufficient to g= et=0A= >> SSL_connect() to return an error.=0A= >>=0A= >> This seems sufficient and works ok for the testing I've done.=0A= >=0A= >I don't think what you ended up with is wrong, to be clear.=0A= >=0A= >But, you have an SSL* as input to SSL_connect(), and you can call=0A= >SSL_get_fd() on that SSL*, which will give you a socket fd that you can=0A= >call setsockopt() on, if you're so inclined. The SSL_connect() abstractio= n=0A= >barrier is not leak-proof :)=0A= Well, actually in this case it is kind of the opposite..=0A= - The kernel RPC creates a socket ("struct socket *", no file descriptor)= =0A= - The daemon does a rpctls_syscall() to acquire a file descriptor referenci= ng=0A= the socket during the connect upcall.=0A= - The daemon does a SSL_set_fd() to tell the openssl library about the sock= et=0A= to use.=0A= But, yes, the daemon does know what the socket file descriptor is.=0A= =0A= What you were suggesting was a setsockopt(so, SO_RCVTIMEO..) then?=0A= =0A= That would work too, I'd guess.=0A= =0A= I would have some reservations about doing it that way.=0A= - I suspect the openssl library will always be prepared for a socket clos= ure (EOF),=0A= but I'm not so sure the library would expect a read failure with errno= =3D=3D=0A= EWOULDBLOCK. I'd have to look at the library sources and I'd also be= =0A= concerned that the situation might change w.r.t. this in the library.= =0A= =0A= But if soshutdown(so, SHUT_RD) stops working, a setsockopt(so, SO_RCVTIMEO)= =0A= would certainly be an alternate approach.=0A= =0A= It is always a little scary when you are using a "black box" like the opens= sl=0A= libraries and doing somewhat unusual things.=0A= =0A= rick=0A= =0A= =0A= > 15sec is pretty arbitrary, but I figure a timeout on the order of seconds= is=0A= > reasonable for RPC upcalls to the local daemon. (I'd guess that taking ev= en=0A= > 1sec to do an upcall would indicate something is broken.)=0A= > If others feel 15sec isn't an appropriate timeout, feel free to comment.= =0A= > (Note that this timeout should only happen when something is broken, like= =0A= > the server that does a "STARTTLS" reply but does not do a TLS handshake.= )=0A= =0A= Understood.=0A= =0A= Thanks,=0A= =0A= Ben=0A= =0A=