From owner-svn-src-projects@freebsd.org Tue May 12 05:07:23 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9178B2E2C13 for ; Tue, 12 May 2020 05:07:23 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Lm563Cdyz3Dj1; Tue, 12 May 2020 05:07:22 +0000 (UTC) (envelope-from bjkfbsd@gmail.com) Received: by mail-ot1-x32f.google.com with SMTP id m33so9536220otc.5; Mon, 11 May 2020 22:07:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KHtyApV04rNlxzmJsT4vGKytNj37FxWFzryXO6y9N5s=; b=p9dblCWd22bPk+NEXEd5yLg0APbtxhw7Q/lCrvdmCS4yVgH0oxEdFk/id5d949FKnL BUbrt9GgZ3AJPQmdNr5tH/VotByDG6C5/0yTDsJCP4XRfDLHXvE5jJIBx/Obdm+EoD+e ZjxqbZxWyObANMG0LlriLScZlnydEvzwty4zzBRbeHGdzRQyMjeZ96xA7q2PQ5lVc84G 4ed+DgT1AFQG30WP40WEWSIfYQRll+EASK197tlSGybyjM0J+I3B4YXB3228VNxeMlC3 i2Fs83/yaSVwvI1S8t9bW+IrfDksQFikxmquervzy+j7BKoAjTIjT5y5lM1FbJzB7NJV Iiwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KHtyApV04rNlxzmJsT4vGKytNj37FxWFzryXO6y9N5s=; b=UlxI9O1uZ7efLTWAFNF0sm6hoWkQCT6CBGu3QodapZgVhnp7Q8Oc3t+BLxAfYoAJNu nqREXj72c86f4OfwcnWHO8lSPrMugj+AfiIByW6W38XMZBy4IqEMGGSrCXnIBMv+ny7A TdkGPYog6sFxZ0/+K5+K92JFI8A/C33PBmA6shOq1W1DI7e2A1or+UDe7yYhpb5gAvBy wSPln0EsG/LgBXdUQLEcTFGgNh9IfsxCT5ArwGf9OuHPleSojlQhdE03noFic1c5/ij1 L1A4W7TWvOAZdSu9IM+R77ETKFxta377PEgcvsQhMe8z1FNizLCoTXJA2veo7XSmIehu HH7w== X-Gm-Message-State: AGi0PuYR8mncGg47fkWIQVbwJIKN0prpVXEKn2T/lNpghCi0Y/D6J0Hf 0yCS3WHTFtlDpl1TFwM0jMULgfM2DqTjTV6Gp09e0S7n X-Google-Smtp-Source: APiQypLGI/rlliKHL5J4r+rIBPDhlxe+E1YA8/6/EXJlb2ZfnI1MwNY7HdfWlbdooNooQSfjpSF+6YLODPCyjCARSA8= X-Received: by 2002:a9d:7a6:: with SMTP id 35mr13902685oto.201.1589260041168; Mon, 11 May 2020 22:07:21 -0700 (PDT) MIME-Version: 1.0 References: <202005100017.04A0Hd7I058863@repo.freebsd.org> <6739df0b-e621-2ca5-8f92-821822733772@FreeBSD.org> In-Reply-To: From: Benjamin Kaduk Date: Mon, 11 May 2020 22:07:10 -0700 Message-ID: Subject: Re: svn commit: r360859 - projects/nfs-over-tls/sys/rpc To: Rick Macklem Cc: John Baldwin , Rick Macklem , "src-committers@freebsd.org" , "svn-src-projects@freebsd.org" X-Rspamd-Queue-Id: 49Lm563Cdyz3Dj1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=p9dblCWd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of bjkfbsd@gmail.com designates 2607:f8b0:4864:20::32f as permitted sender) smtp.mailfrom=bjkfbsd@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(0.00)[ip: (-7.80), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.42), country: US(-0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[f.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2020 05:07:23 -0000 On Mon, May 11, 2020 at 9:03 PM Rick Macklem wrote: > >Rick Macklem wrote: > >>John Baldwin wrote: > >>>On 5/9/20 5:17 PM, Rick Macklem wrote: > >>>> Author: rmacklem > >>>> Date: Sun May 10 00:17:39 2020 > >>>> New Revision: 360859 > >>>> URL: https://svnweb.freebsd.org/changeset/base/360859 > >>>> > >>>> Log: > >>>> Add some very basic handling of TLS_GET_RECORD control mbufs. > >>>> > >>>> For now, it just throws away any that are non-application data. > >>>> In the future, this will need to change, but not until TLS1.3, I > think? > >>> > >>>Ideally you'd keep an nfsd thread in userland that you could pass > >>>these records onto. One possible option is the thread just keeps > >>>calling SSL_read() but you do create a new flag on the socket buffer > >>>that causes soreceive() to only pass non-application data datagrams > >>>to userland reads() and have the in-kernel read requests block if they > >>>see a non-application data record as the next record until the user > >>>thread wakes up and reads it (or EAGAIN or whatever you need it to > >>>do). > You can avoid having to play games with putting stuff back on the socket receive buffer by using a custom BIO implementation in userspace that knows how to inject the received message. > Actually, what might work for the krpc code is a new MSG_TLSAPPDATA > flag for soreceive_generic(), which says "if the record is not application > data, return an error". (Sort of the opposite of what you said above, but > would perform the same thing.) > This could be used for the krpc soreceive() calls, so that the > non-application > data record remains on the socket's receive buffer. > > Then the krpc could do the upcall when the error is returned by soreceive() > and the userland daemon could do an SSL_read() with > SSL_MODE_AUTO_RETRY turned off. If I understand the man page, that will > make SSL_read() process the non-application data record but return with an > error of SSL_ERROR_WANT_READ without taking application data off the > socket's receive buffer queue. > The typical way to consume non-application-data records without hanging trying to read any application data is to do a zero-length read. This still gets far enough into the state machine machinery to do the job before checking that the length is nonzero. > --> If this all works (?), then the krpc can just go on and soreceive() > the next > application data record after the upcall returns. > > Worth a try anyhow, I think? rick > > >>Well, I currently have daemons (rpctlssd and rpctlscd) that just wait for > >>upcalls from the kernel and do the SSL stuff (mainly the handshake right > now). > >(You can guess from the names which one is RPC client vs server.;-) > >I can easily do an upcall for a non-application data record if/when I > need to do so. > >(The upcalls are done via Sun RPC using an AF_LOCAL socket, similar to > what > > the gssd does.) > > > >For me, the mystery is what to do with it once the daemon gets it. > >From what you said, I'll need to "trick" SSL_read into reading it. > >Maybe I can push it back on the socket buffer receive queue in the kernel > >and then the daemon can do a SSL_read() to read it off the socket and > handle > >it? > Oh, and one more little challenge... > When I played around with the daemons using TLS1.3 (before there was a ktls > rx I could use), I would run into early data that would be handled by > SSL_read() > done in userland by the daemon. > However, I couldn't find a way to tell it to not wait to read any > application data. > I recall trying an SSL_read() for 0 bytes and it didn't like it. > In the early-data case things are more complicated. Calling regular SSL_read() will drive the handshake to completion, and there's a separate function to call to just try to read early data. (You could also configure things to fully deny early data which would probably be easier.) -Ben > I might be stuck having the daemon do an SSL_read() for one application > data > record and then it can pass that data back down into the kernel to be > prepended > on the queue of received application data. > > >(I wouldn't want to MSG_PEEK for every record, since these will be rare.) > >I also do already have code that blocks kernel reception when the upcall > >to do the handshake is done, so the same could be used in this case. > > > >There is the slight trick that the client krpc code is in a socket upcall > that can't sleep, > >so I'll have to hand it off to some other thread that can sleep when I > need to do it. > > > >Thanks for the hints, rick > rick > -- > John Baldwin > _______________________________________________ > svn-src-projects@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-projects > To unsubscribe, send any mail to "svn-src-projects-unsubscribe@freebsd.org > " >