From owner-freebsd-current@freebsd.org Sun Jan 26 20:08:28 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E0C3F1F9C08 for ; Sun, 26 Jan 2020 20:08:28 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on061e.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::61e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 485P8C25DYz4Rk4; Sun, 26 Jan 2020 20:08:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WUZ2umTp9QKhMDoM8u6HmOAC2Om55XYKdNwnfrGR310QHM+tvw1UAYKOgaSREQK7b9vIFKkWX4+c2ojL5zsx4/3mBGM37Ecr08c1B5NKAk5BBlCazQaEvWPSrn8OUhzf1iNRIrHdLUkVNcf4BzwOQjTSUxkYyqPcJ3zCHEgI7u/SYUeaKeSnenjVtDexCei6TKVRgkphAzOiGhZQ/ofqKdqddFl6gAZ0wbepCzlbaVM3BGOsOozXby9hf2gmjNQmDlvUlImsaZ8/IKpfujbJXA1hMeki3Udkb4sY+EOci6nkLNx1r+c0VwPsm/wUWQt+2/VBU/Mt9OaXU++l1ZJ8RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=agFfBABy7TtdJq/Bou2AUfLvBLfYCaWDrz9gBqnyCPM=; b=V6cAfFSFB8UJ/SKIQpAet10TmvWAGg3VHNKUGYAWHBqFwewAVyj1QhsjMNAzmt65iPBD14V7uGgwuym5Q8mkKXa9BS5BAYfvdD2VlHBOW32UU6aBQXQvZleHJruLHWkPXn31klliKvIRZLrscNJPZPtYptCrGmEPleqWyIofscecWRDn8d1NoELxjkJzxpaunqU7pfSQhuZnCte4wQi86Q31EgtB3UWSakqD4Q9IpKFots00ddSRhcWc/3R1iwZCEzEYGH4aq+FoegCU4d3Q1TZuD+/TH4/jAQfekd6NApcNalSCLsNeqfxLCup6GV4ZFiPiegw7aGoKDytGeAMC6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM (52.132.69.153) by YQBPR0101MB0820.CANPRD01.PROD.OUTLOOK.COM (52.132.66.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2665.23; Sun, 26 Jan 2020 20:08:25 +0000 Received: from YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM ([fe80::6588:45c3:4892:f98]) by YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM ([fe80::6588:45c3:4892:f98%7]) with mapi id 15.20.2665.017; Sun, 26 Jan 2020 20:08:25 +0000 From: Rick Macklem To: John Baldwin , "freebsd-current@FreeBSD.org" Subject: Re: how to use the ktls Thread-Topic: how to use the ktls Thread-Index: AQHVxa2HeRfmo36hWEyrGcMaBhE88KfhEeoAgBxnlpk= Date: Sun, 26 Jan 2020 20:08:25 +0000 Message-ID: References: , <5be57c87-90fe-fcbe-ea37-bdb1bcff2da8@FreeBSD.org> In-Reply-To: <5be57c87-90fe-fcbe-ea37-bdb1bcff2da8@FreeBSD.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ac21c625-e933-437d-20d7-08d7a29b803d x-ms-traffictypediagnostic: YQBPR0101MB0820: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5797; x-forefront-prvs: 02945962BD x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(39860400002)(376002)(136003)(396003)(366004)(189003)(199004)(71200400001)(66556008)(66476007)(66946007)(64756008)(8936002)(66446008)(110136005)(316002)(786003)(5660300002)(81156014)(81166006)(8676002)(7696005)(2906002)(76116006)(9686003)(55016002)(478600001)(186003)(33656002)(450100002)(86362001)(6506007)(52536014); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB0820; H:YQBPR0101MB1427.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Vq54Er8ACAHCyRwQUH07PsFQMnuPUz5Ia2xggjlxJo3uwSmZmzx5QBPeTotZwplLDEzsI0mQKynJe+e8wXRXDAVFuUOMEZK9Pvsj01mPS4u9Wsf7Pw+BCLU+mp7Z13D2HP4QEwtohJJBpVQxOx6ADbPwULsSY5Y6y2l6RrA9H+TmqHw5WWkwhFK6Ci2XvDu3fwCiQTGflBRQuHw4P6cGbn36ej4oaA8nbnzWPsY5y7dVrYLhPZWjq9bn+EPam7o/TLtmLEgeO3OWO5y83PKaHPpmyIpq3j4NjkgrfRJwfXTcZyBHEME5c9AdAbGu/DpOsIWmuukMncwHBxwIjyP6wLXnK1YLt7ac7q8SWpIzrTWQG61I94LlMzLa4sRXRyXGnDKlfx+8URZRjWH4mXlrEPYZHon1nYfcucOmi/wlwNdm1t4tvV5ReqKdGXrQa9vf x-ms-exchange-antispam-messagedata: WEok+WQLlgVVgTiPrY0C86VFaQQ5gp6Uib9mFxMaCG5scDS9c3zEaSj/9j8Y8UtMf7uM5eLMtU1FLXGfjQcIJdUyPPdnj8XolP0op+0vffgF3sFvHdDWJBEMqrFrdOP5ckwZXV97ZyCfW5oLhtHzlEPLynzYnL5nfvVp+1Mx3g9jBM+nrE962vVVpw6oa/lByippxR7cpm8j2gAUjKIHVQ== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: ac21c625-e933-437d-20d7-08d7a29b803d X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Jan 2020 20:08:25.0855 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oOwamABIWonIMvqbMgQFfWdmK1N1DJxJhr6aRzeWGjZ1A6fiHOgpv15DNs1UzMwxlK/TfM4U6sGdxYxrenx/aQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB0820 X-Rspamd-Queue-Id: 485P8C25DYz4Rk4 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::61e as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.68 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[uoguelph.ca]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.38)[ipnet: 2a01:111:f000::/36(-3.83), asn: 8075(-3.04), country: US(-0.05)]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[i=1] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jan 2020 20:08:28 -0000 John Baldwin wrote:=0A= [stuff snipped]=0A= >Hmmm, this might be a fair bit of work indeed.=0A= >=0A= >Right now KTLS only works for transmit (though I have some WIP for receive= ).=0A= >=0A= >KTLS does assumes that the initial handshake and key negotiation is handle= d by=0A= >OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which= =0A= >session keys to use.=0A= >=0A= >I think what you would want to do is use something like OpenSSL_connect() = in=0A= >userspace, and then check to see if KTLS "worked". If it did, you can tel= l=0A= >the kernel it can write to the socket directly, otherwise you will have to= =0A= >bounce data back out to userspace to run it through SSL_write() and have= =0A= >userspace do SSL_read() and then feed data into the kernel.=0A= >=0A= >The pseudo-code might look something like:=0A= >=0A= >SSL *s;=0A= >=0A= >s =3D SSL_new(...);=0A= >=0A= >/* fd is the existing TCP socket */=0A= >SSL_set_fd(s, fd);=0A= >OpenSSL_connect(s);=0A= >if (BIO_get_ktls_send(SSL_get_wbio(s)) {=0A= > /* Can use KTLS for transmit. */=0A= >}=0A= >if (BIO_get_ktls_recv(SSL_get_rbio(s)) {=0A= > /* Can use KTLS for receive. */=0A= >}=0A= =0A= So, I've been making some progress. The first stab at the daemons that do t= he=0A= handshake are now on svn in base/projects/nfs-over-tls/usr.sbin/rpctlscd an= d=0A= rpctlssd.=0A= =0A= A couple of questions...=0A= 1 - I haven't found BIO_get_ktls_send() or BIO_get_ktls_recv(). Are they in= some=0A= different library?=0A= 2 - After a successful SSL_connect(), the receive queue for the socket has = 478bytes=0A= of stuff in it. SSL_read() seems to know how to skip over it, but I ha= ven't=0A= figured out a good way to do this. (I currently just do a recv(..478,0= ) on the=0A= socket.)=0A= Any idea what to do with this? (Or will the receive side of the ktls f= igure out=0A= how to skip over it?)=0A= =0A= I'm currently testing with a kernel that doesn't have options KERN_TLS and= =0A= (so long as I get rid of the 478 bytes), it then just does unencrypted RPCs= .=0A= =0A= So, I guess the big question is.... can I get access to your WIP code for K= TLS=0A= receive? (I have no idea if I can make progress on it, but I can't do a lot= more=0A= before I have that.)=0A= =0A= Oh, and for anyone out there...=0A= What is the easiest freebie way to test signed certificates?=0A= (I currently am using a self-signed certificate, but I need to test the "re= al" version=0A= at some point soon.)=0A= =0A= Thanks, rick=0A= =0A= =0A= --=0A= John Baldwin=0A=