Date: Mon, 11 May 2020 22:07:10 -0700 From: Benjamin Kaduk <bjkfbsd@gmail.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: John Baldwin <jhb@freebsd.org>, Rick Macklem <rmacklem@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org>, "svn-src-projects@freebsd.org" <svn-src-projects@freebsd.org> Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc Message-ID: <CAJ5_RoBw-vyPH1EdeKHS=a_kCm3HPoiAkpA%2BrTxhPYcqf3Lz-w@mail.gmail.com> In-Reply-To: <QB1PR01MB36494D1076388A2AD64F5B02DDBE0@QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM> References: <202005100017.04A0Hd7I058863@repo.freebsd.org> <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> <QB1PR01MB3649E7E08BDFE39C4B8B70A4DDBE0@QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM> <QB1PR01MB3649282401A7562B36D3F84EDDBE0@QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM> <QB1PR01MB36494D1076388A2AD64F5B02DDBE0@QB1PR01MB3649.CANPRD01.PROD.OUTLOOK.COM>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, May 11, 2020 at 9:03 PM Rick Macklem <rmacklem@uoguelph.ca> wrote: > >Rick Macklem wrote: > >>John Baldwin wrote: > >>>On 5/9/20 5:17 PM, Rick Macklem wrote: > >>>> Author: rmacklem > >>>> Date: Sun May 10 00:17:39 2020 > >>>> New Revision: 360859 > >>>> URL: https://svnweb.freebsd.org/changeset/base/360859 > >>>> > >>>> Log: > >>>> Add some very basic handling of TLS_GET_RECORD control mbufs. > >>>> > >>>> For now, it just throws away any that are non-application data. > >>>> In the future, this will need to change, but not until TLS1.3, I > think? > >>> > >>>Ideally you'd keep an nfsd thread in userland that you could pass > >>>these records onto. One possible option is the thread just keeps > >>>calling SSL_read() but you do create a new flag on the socket buffer > >>>that causes soreceive() to only pass non-application data datagrams > >>>to userland reads() and have the in-kernel read requests block if they > >>>see a non-application data record as the next record until the user > >>>thread wakes up and reads it (or EAGAIN or whatever you need it to > >>>do). > You can avoid having to play games with putting stuff back on the socket receive buffer by using a custom BIO implementation in userspace that knows how to inject the received message. > Actually, what might work for the krpc code is a new MSG_TLSAPPDATA > flag for soreceive_generic(), which says "if the record is not application > data, return an error". (Sort of the opposite of what you said above, but > would perform the same thing.) > This could be used for the krpc soreceive() calls, so that the > non-application > data record remains on the socket's receive buffer. > > Then the krpc could do the upcall when the error is returned by soreceive() > and the userland daemon could do an SSL_read() with > SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will > make SSL_read() process the non-application data record but return with an > error of SSL_ERROR_WANT_READ without taking application data off the > socket's receive buffer queue. > The typical way to consume non-application-data records without hanging trying to read any application data is to do a zero-length read. This still gets far enough into the state machine machinery to do the job before checking that the length is nonzero. > --> If this all works (?), then the krpc can just go on and soreceive() > the next > application data record after the upcall returns. > > Worth a try anyhow, I think? rick > > >>Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for > >>upcalls from the kernel and do the SSL stuff (mainly the handshake right > now). > >(You can guess from the names which one is RPC client vs server.;-) > >I can easily do an upcall for a non-application data record if/when I > need to do so. > >(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to > what > > the gssd does.) > > > >For me, the mystery is what to do with it once the daemon gets it. > >From what you said, I'll need to "trick" SSL_read into reading it. > >Maybe I can push it back on the socket buffer receive queue in the kernel > >and then the daemon can do a SSL_read() to read it off the socket and > handle > >it? > Oh, and one more little challenge... > When I played around with the daemons using TLS1.3 (before there was a ktls > rx I could use), I would run into early data that would be handled by > SSL_read() > done in userland by the daemon. > However, I couldn't find a way to tell it to not wait to read any > application data. > I recall trying an SSL_read() for 0 bytes and it didn't like it. > In the early-data case things are more complicated. Calling regular SSL_read() will drive the handshake to completion, and there's a separate function to call to just try to read early data. (You could also configure things to fully deny early data which would probably be easier.) -Ben > I might be stuck having the daemon do an SSL_read() for one application > data > record and then it can pass that data back down into the kernel to be > prepended > on the queue of received application data. > > >(I wouldn't want to MSG_PEEK for every record, since these will be rare.) > >I also do already have code that blocks kernel reception when the upcall > >to do the handshake is done, so the same could be used in this case. > > > >There is the slight trick that the client krpc code is in a socket upcall > that can't sleep, > >so I'll have to hand it off to some other thread that can sleep when I > need to do it. > > > >Thanks for the hints, rick > rick > -- > John Baldwin > _______________________________________________ > svn-src-projects@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-projects > To unsubscribe, send any mail to "svn-src-projects-unsubscribe@freebsd.org > " >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ5_RoBw-vyPH1EdeKHS=a_kCm3HPoiAkpA%2BrTxhPYcqf3Lz-w>