From owner-freebsd-current@freebsd.org Mon Jan 13 20:53:25 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE3E51EC741 for ; Mon, 13 Jan 2020 20:53:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670076.outbound.protection.outlook.com [40.107.67.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47xQm54ybMz49w4; Mon, 13 Jan 2020 20:53:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Por5fjiOIDIyrTrylAXc40QDNiBWGH8iTuHPpB/0EFh6JQJDkQ6SadszOECJSRCMozk2NTG+/idBwaD6snwuCUTJiWy2AM7nRN/MdeJvX3aLi4g7FCknqTrcBVxmFiXVS1H/uRITUZIGkhVL/m88wA19vjApZ0NtQzuHDdVLZqQWPvSeE+IgBxEpuFIrt6GFTVAjU/KmcXZ66fydpjl4aLfaEjMe4r2Grsc1+LAhZZf0PKs/m8xpENbLYx2DhD5Hiho1Usdh6oob1TbWcKuzdiGtO9sNPiyS/fQwtsIzjW+NRSflCtneXmNFCPujEuaPkwLcElYeDfYLBTUrzWEq1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K0N44kanHIMYNtWA+AmW9VKTckQ82NJzFLhKVAcb7xQ=; b=W2J6T9wb28m0HoyuI8e0FuPnzGpPQluAaph4Wl2JFBxDzqqfNjldDgB137zKpt4irhYZkuAfgO3+LOl9xqQQPW7DkWMcIg/FvgR1hkpwonTrgBzZmwYZ3oh03T5AtfpKDvAn6XdJ+LxpwIZiYkEh5kidLOoZR7C8/x77Ws3e4MmvfzYbFD0zO6Hm4tEHkn1pJdnfBX3hEm1xTxYmjbLFNS48A3JxVddtfBnjcYFpfCSPRzFjq7aTcBd/B7GnCr/M4penuztcnmoeSvN9iT2EUkqzboF6KwRbV9ui+2UmjGhhzI/c4n1lG0Y6tRC83ZHYJtGhZ6d2grW4xOIuyWGA/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM (52.132.69.153) by YQBPR0101MB0931.CANPRD01.PROD.OUTLOOK.COM (52.132.67.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.10; Mon, 13 Jan 2020 20:53:23 +0000 Received: from YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM ([fe80::7512:8580:8d82:6c94]) by YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM ([fe80::7512:8580:8d82:6c94%6]) with mapi id 15.20.2623.015; Mon, 13 Jan 2020 20:53:23 +0000 From: Rick Macklem To: John Baldwin , Benjamin Kaduk CC: "freebsd-current@FreeBSD.org" Subject: Re: how to use the ktls Thread-Topic: how to use the ktls Thread-Index: AQHVxa2HeRfmo36hWEyrGcMaBhE88KfhEeoAgAHfXuuABRWwAIAAzp8AgABD2ZM= Date: Mon, 13 Jan 2020 20:53:23 +0000 Message-ID: References: <5be57c87-90fe-fcbe-ea37-bdb1bcff2da8@FreeBSD.org> <20200113042310.GF27483@kduck.mit.edu>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ca4d8fdb-8cfa-4e78-5190-08d7986aa143 x-ms-traffictypediagnostic: YQBPR0101MB0931: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 028166BF91 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(189003)(199004)(81166006)(81156014)(53546011)(186003)(8676002)(26005)(55016002)(9686003)(71200400001)(2906002)(6506007)(33656002)(498600001)(4326008)(110136005)(91956017)(52536014)(5660300002)(76116006)(66946007)(66476007)(66556008)(64756008)(66446008)(7696005)(8936002)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB0931; H:YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ijrvEmuH/xVlxm86XUTF0SkTbYg/S+asGs35Q0a8Z3zOHnkeJAP8w7B+rKSMmlJIUDsTyuWxKqJ10q1KuNMTb7+pgJftwaQIUsp5Ds/tGlRPDb1CvYuDp6QFPNjbyvcqPkgmL2VHhkJEfU5mXfj2dmZwioP/xcXULgLyxnVPsP/OuTfDoZp16VRsBdCw0e/XMKcPOWHKg0fqAkrtVtPL0tTb5y83Ul2y4txbpsnk2VSyFvsXo/x6umqu6+zrGnU1epQZxN6XhsmovY4oFCmR0Q2+o3s32rSB3vuhpk3hXZZ3GPhNmOZDVdGL8xASx20zsZlIIH+eweJtFuMS4cu8tJDOUKmM7Sfxm0VGFfSWwyn14vFWGZiQ202uiOit7A58yYtJP94lQTiZ/64ns0HLXnOSDtuBomKkF04R6moPBCtPyms4HIfrPMPzD76IBqNQ x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: ca4d8fdb-8cfa-4e78-5190-08d7986aa143 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Jan 2020 20:53:23.4654 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: u42ObiRqfcxHhohF16W9jZVo0Ic5rgYqfWIuOmKp94iedYaQ+O/6zK8NXFCGJYn4nps23yDT+oVC2gjhMHgsxQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB0931 X-Rspamd-Queue-Id: 47xQm54ybMz49w4 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.99 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jan 2020 20:53:25 -0000 John Baldwin wrote:=0A= >On 1/12/20 8:23 PM, Benjamin Kaduk wrote:=0A= >> On Thu, Jan 09, 2020 at 10:53:38PM +0000, Rick Macklem wrote:=0A= >>> John Baldwin wrote:=0A= >>>> On 1/7/20 3:02 PM, Rick Macklem wrote:=0A= >>>>> Hi,=0A= >>>>>=0A= >>>>> Now that I've completed NFSv4.2 I'm on to the next project, which is = making NFS=0A= >>>>> work over TLS.=0A= >>>>> Of course, I know absolutely nothing about TLS, which will make this = an interesting=0A= >>>>> exercise for me.=0A= >>>>> I did find simple server code in the OpenSSL doc. which at least give= s me a starting=0A= >>>>> point for the initialization stuff.=0A= >>>>> As I understand it, this initialization must be done in userspace?=0A= >>>>>=0A= >>>>> Then somehow, the ktls takes over and does the encryption of the=0A= >>>>> data being sent on the socket via sosend_generic(). Does that sound r= ight?=0A= >>>>>=0A= >>>>> So, how does the kernel know the stuff that the initialization phase = (handshake)=0A= >>>>> figures out, or is it magic I don't have to worry about?=0A= >>>>>=0A= >>>>> Don't waste much time replying to this. A few quick hints will keep m= e going for=0A= >>>>> now. (From what I've seen sofar, this TLS stuff isn't simple. And I t= hought Kerberos=0A= >>>>> was a pain.;-)=0A= >>>>>=0A= >>>>> Thanks in advance for any hints, rick=0A= >>>>=0A= >>>> Hmmm, this might be a fair bit of work indeed.=0A= >>> If it was easy, it wouldn't be fun;-) FreeBSD13 is a ways off and if i= t doesn't make that, oh well..=0A= >>>=0A= >>>> Right now KTLS only works for transmit (though I have some WIP for rec= eive).=0A= >>> Hopefully your WIP will make progress someday, or I might be able to wo= rk on it.=0A= >>>=0A= >>>> KTLS does assumes that the initial handshake and key negotiation is ha= ndled by=0A= >>>> OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel whi= ch=0A= >>>> session keys to use.=0A= >>> Yea, I figured I'd need a daemon like the gssd for this. The krpc makes= it a little=0A= >>> more fun, since it handles TCP connections in the kernel.=0A= >>>=0A= >>>> I think what you would want to do is use something like OpenSSL_connec= t() in=0A= >>>> userspace, and then check to see if KTLS "worked".=0A= >>> Thanks (and for the code below). I found the simple server code in the = OpenSSL doc,=0A= >>> but the client code gets a web page and is quite involved.=0A= >>>=0A= >>>> If it did, you can tell=0A= >>>> the kernel it can write to the socket directly, otherwise you will hav= e to=0A= >>>> bounce data back out to userspace to run it through SSL_write() and ha= ve=0A= >>>> userspace do SSL_read() and then feed data into the kernel.=0A= >>> I don't think bouncing the data up/down to/from userland would work wel= l.=0A= >>> I'd say "if it can't be done in the kernel, too bad". The above could b= e used for=0A= >>> a NULL RPC to see it is working, for the client.=0A= >>=0A= >> So you're saying that we'd only support rpc-over-tls as an NFS client an= d=0A= >> not as a server, at least until the WIP for ktls read appears?=0A= Actually, I'd say that neither NFS client nor server will work over tls unt= il=0A= the receive side works, since NFS RPCs result in bi-directional traffic.=0A= =0A= >To be clear, I have KTLS RX working with TOE right now. I have a design i= n my=0A= >head for KTLS RX that would use software and co-processor engines via OCF = such=0A= >as aesni(4) and ccr(4) that I hope to implement in the next few months, so= KTLS=0A= >RX isn't too far off. OpenSSL already supports KTLS RX on Linux and the F= reeBSD=0A= >patches I already have use the same API. (Each received TLS frame is read= via=0A= >recvmsg() with the TLS header fields in a cmsg.)=0A= Sounds good. It will be a while before I get to the stage where I need it.= =0A= I'm currently working on how to give userland access to a socket created i= n the=0A= kernel, so that a daemon can use it.=0A= =0A= Have fun with it, rick=0A= =0A= --=0A= John Baldwin=0A=