From owner-freebsd-arch@freebsd.org Mon Jan 8 17:30:21 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE19BE79631; Mon, 8 Jan 2018 17:30:21 +0000 (UTC) (envelope-from Anton.Rang@dell.com) Received: from esa7.dell-outbound.iphmx.com (esa7.dell-outbound.iphmx.com [68.232.153.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.dell-outbound.iphmx.com", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 64EA963900; Mon, 8 Jan 2018 17:30:20 +0000 (UTC) (envelope-from Anton.Rang@dell.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=dell.com; i=@dell.com; q=dns/txt; s=smtpout; t=1515432417; x=1546968417; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=RVPEUPFalTgnjitjhLJD1U2wK2JMWE91e51agZNjHXg=; b=QaJdDKlUKfNrUElkgy7iZqy6fpm+J8TosUBjUuoun7a5VV19XMVVUnw4 y8+0ijGYUZ16tr+HL1XAObSh+hpU2l3IfTVc3oBcJ6WoyS/r0U+kadRiB C8W4GD1V6M3BZZbGYLGYg8tfPJuXVqaQSXmbfZg3QrvtdtmvubK46mHNH k=; IronPort-PHdr: =?us-ascii?q?9a23=3AKl2V5hSSbYgP3Z2XFjVoAN9xe9psv+yvbD5Q0YIu?= =?us-ascii?q?jvd0So/mwa6zYhKN2/xhgRfzUJnB7Loc0qyK6/mmATRIyK3CmUhKSIZLWR4BhJ?= =?us-ascii?q?detC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TW94jEIBxrwKxd+?= =?us-ascii?q?KPjrFY7OlcS30P2594HObwlSizexfa5+IA+qoQnNq8IbnZZsJqEtxxXTv3BGYf?= =?us-ascii?q?5WxWRmJVKSmxbz+MK994N9/ipTpvws6ddOXb31cKokQ7NYCi8mM30u683wqRbD?= =?us-ascii?q?VwqP6WACXWgQjxFFHhLK7BD+Xpf2ryv6qu9w0zSUMMHqUbw5Xymp4rx1QxH0li?= =?us-ascii?q?gIKz858HnWisNuiqJbvAmhrAF7z4LNfY2ZKOZycqbbcNgHR2ROQ9xRWjRBDI2i?= =?us-ascii?q?coUPE+QPM+VWr4b/plsBsRSwCga3CePz0TBIg2P60bEm3+kjFwzNwQwuH8gJsH?= =?us-ascii?q?TRtNj5OrscXvqzzKnH1TnIcu9b2THh6IjPdBAtr+yHULVsfMrX1UkvEAXFgk+M?= =?us-ascii?q?p4P/OTOV2f8AvHWF4OpkUeKjkXIoqwZ0ojW2wMonl4fHhoUQyl/e9CV5xp44Jd?= =?us-ascii?q?OiSEFlf9GrC4BQuDyAO4txWMMiTGdlszs5xL0eoZO2fSsHxI45yxPRdfCLaZWE?= =?us-ascii?q?7xLtWeqLPzt0mHxodKqiixqu60Ss1PHwWtWu3FtKqidJiMTAu34O2hDL98SKS/?= =?us-ascii?q?9w8l2/1TuP2A3f8OBJLVoqmafVJZMsxKM7mIAJvkTZBCD2nV37jKqRdko55Oel?= =?us-ascii?q?8//nYrD6pp+EMI90lx3+PrwumsOhBeQ4NRADUHaA+eum1LDv51D2T6tOjv0yi6?= =?us-ascii?q?XZt43aJdgAqa6+Hg9V1Jss5wilAzenyNQYnXwHLV1fdB2biIjpPknCIPH+Dfih?= =?us-ascii?q?n1ShiDZmyvPcMrH/DJjBMGLPnKrhcLtz8UJQ1hY/wN5H65JREL4BIfbzWkHrtN?= =?us-ascii?q?zfCx80KxC5w+D7CNV60IMSQ36BDbWfMKPdqlKH+/wgI+2IZIMPpDn9LP0l6+b0?= =?us-ascii?q?jXAlgV8dYbWp3ZwPZXC2BPRpPVuWbmH3gtgcCGsFpBA+Q/DqiFCZXz5TfWi9UL?= =?us-ascii?q?wn6TEgFY2qF4DDRpqigLaZxie0AoVWZnxaClCLCXroeZ+EVOkSZy2JOc9ujyUI?= =?us-ascii?q?Vbi7RIA91hGhqhX6y6F8I+ra4C0Xq4zs28Nu5+LOjx0y8iZ0D8vOm12KGl5znG?= =?us-ascii?q?gJSjQ2lJhiqkx00B/Xzq96n/FbPcRO7PNASEE8OIKKnMJgDNWnEDjIeNjNAH+g?= =?us-ascii?q?XtKgS3llZ9QtxNlIWU97FP2ugxTHmSGtBulGxPSwGJUo//eEjDDKLMFnxiODjf?= =?us-ascii?q?F5gg=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2HMAAA7qVNamMuZ6ERcGwEBAQEDAQEBC?= =?us-ascii?q?QEBAYQVEHQnB4QKmH2CApk/ChgLhElPAhqEHEMUAQEBAQEBAQEBAQIQAQEBAQE?= =?us-ascii?q?ICwsGKC+COCQBDkstAQEBAQEBAQEBIwEBAQEBASMCCAVdAQEBAwEBASERDA4RD?= =?us-ascii?q?wsBBAsCAQgOCgICJgICAiUBCQEVEAIECAYBBAEcAgKKCAgQr3qCJ4MRhyABAQE?= =?us-ascii?q?BAQEBAQEBAQEBAQEBAQEBFwMFgQ+ER1+BV4FoKYMFgzCBTgEBgzUxgjSjXQYCi?= =?us-ascii?q?AWHI4YUlAmNM4k3AgQCBAUCGoE8NoFzb1IqAYF/glQQDIFneAiIK4ElgRcBAQE?= X-IPAS-Result: =?us-ascii?q?A2HMAAA7qVNamMuZ6ERcGwEBAQEDAQEBCQEBAYQVEHQnB4Q?= =?us-ascii?q?KmH2CApk/ChgLhElPAhqEHEMUAQEBAQEBAQEBAQIQAQEBAQEICwsGKC+COCQBD?= =?us-ascii?q?kstAQEBAQEBAQEBIwEBAQEBASMCCAVdAQEBAwEBASERDA4RDwsBBAsCAQgOCgI?= =?us-ascii?q?CJgICAiUBCQEVEAIECAYBBAEcAgKKCAgQr3qCJ4MRhyABAQEBAQEBAQEBAQEBA?= =?us-ascii?q?QEBAQEBFwMFgQ+ER1+BV4FoKYMFgzCBTgEBgzUxgjSjXQYCiAWHI4YUlAmNM4k?= =?us-ascii?q?3AgQCBAUCGoE8NoFzb1IqAYF/glQQDIFneAiIK4ElgRcBAQE?= Received: from esa5.dell-outbound2.iphmx.com ([68.232.153.203]) by esa7.dell-outbound.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jan 2018 11:25:50 -0600 From: "Rang, Anton" Received: from mailuogwdur.emc.com ([128.221.224.79]) by esa5.dell-outbound2.iphmx.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jan 2018 23:24:58 +0600 Received: from maildlpprd55.lss.emc.com (maildlpprd55.lss.emc.com [10.106.48.159]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id w08HSv2w031920 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 8 Jan 2018 12:28:59 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com w08HSv2w031920 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1515432540; bh=YvVrLgzXNb65dmU0ndxfAQfw68w=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-ID:Content-Transfer-Encoding:MIME-Version; b=UOrAT0KnywKn1buXZTGGCk44fQg6XCdlU5IkzgxlTGwAke8n6a9i4u+ENCzbf6n/J WMCltwoL/a9wp4CAHlpHL9vMjepKooWPy9G6ZxwIFd2CAd/s95+jygUD3yZUO8me1/ Y/6nDHtHDzsOijqMBKIwY1QdhsGg0I3npOu2TdK8= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com w08HSv2w031920 Received: from mailusrhubprd51.lss.emc.com (mailusrhubprd51.lss.emc.com [10.106.48.24]) by maildlpprd55.lss.emc.com (RSA Interceptor); Mon, 8 Jan 2018 12:28:34 -0500 Received: from MXHUB216.corp.emc.com (MXHUB216.corp.emc.com [10.253.68.86]) by mailusrhubprd51.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id w08HSLDp027832 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=FAIL); Mon, 8 Jan 2018 12:28:23 -0500 Received: from MX103CL02.corp.emc.com ([169.254.6.233]) by MXHUB216.corp.emc.com ([10.253.68.86]) with mapi id 14.03.0352.000; Mon, 8 Jan 2018 12:28:20 -0500 To: Wojciech Puchar CC: Warner Losh , Eric McCorkle , "freebsd-hackers@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: A more general possible meltdown/spectre countermeasure Thread-Topic: A more general possible meltdown/spectre countermeasure Thread-Index: AQHTiKYTEmeeeTOFpkyH+zs3xbP9kQ== Date: Mon, 8 Jan 2018 17:28:20 +0000 Message-ID: <6AF5185E-201A-4C8E-B4ED-B30C65E36E99@isilon.com> References: <33bcd281-4018-7075-1775-4dfcd58e5a48@metricspace.net> <73d2f1a5-55f7-0ae7-7660-3e680ba3d32e@metricspace.net> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.200.59.138] Content-Type: text/plain; charset="utf-8" Content-ID: <5E962322D258134786F410FBD9824AD6@mail.corp.emc.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd51.lss.emc.com X-RSA-Classifications: public X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jan 2018 17:30:21 -0000 VGhlIHRhYmxlcyBhcmVu4oCZdCBjaGFuZ2VkIG9uIGVhY2ggdHJhbnNpdGlvbjsgcmF0aGVyLCB0 d28gcGFnZSB0YWJsZXMgYXJlIG1haW50YWluZWQsIG9uZSB3aGljaCBoYXMgYW4gZW50cnkgZm9y IHRoZSBrZXJuZWwgbWFwcGluZ3MgYXQgdGhlIHRvcCBsZXZlbCBhbmQgb25lIHdoaWNoIGRvZXMg bm90Lg0KDQpUaGVuIGl04oCZcyBzaW1wbHkgYSBtYXR0ZXIgb2YgY2hhbmdpbmcgd2hpY2ggcGFn ZSB0YWJsZSBpcyBleGFtaW5lZCAoYnkgd3JpdGluZyB0byBDUjMpIG9uIHRyYW5zaXRpb24uDQoN CklmIFBDSUQgaXMgYXZhaWxhYmxlLCBwcmV2aW91c2x5LXVzZWQgbWFwcGluZ3MgY2FuIHN0YXkg Y2FjaGVkIGluIHRoZSBUTEIgdGhyb3VnaCB0aGlzLCB0aG91Z2ggdGhleSB3b27igJl0IGJlIHNo YXJlZCBiZXR3ZWVuIHVzZXIva2VybmVsIChzbyBpbiBnZW5lcmFsIHN5c2NhbGxzIHdpbGwgaW5j dXIgYW4gYWRkaXRpb25hbCB0cmFuc2xhdGlvbiBwZXIgYnVmZmVyIHBhZ2UpLg0KDQpBbnRvbg0K DQo+IE9uIEphbiA2LCAyMDE4LCBhdCAyOjQxIFBNLCBXb2pjaWVjaCBQdWNoYXIgPHdvanRla0Bw dWNoYXIubmV0PiB3cm90ZToNCj4gDQo+PiBUaGUgb25seSB3b3JrYXJvdW5kIHRoYXQncyBjb21w bGV0ZWx5IGVmZmVjdGl2ZSBpcyB0byB1bm1hcCBhbGwgb2Yga2VybmVsIG1lbW9yeSB3aGVuIHJ1 bm5pbmcgaW4gdXNlcmxhbmQuIEl0J3MgYSBiaXQgdHJpY2t5IGJlY2F1c2UNCj4gDQo+IHRoaXMg bWVhbnMgb24gZXZlcnkgc3lzY2FsbCBvbiBpbnRlcnJ1cHQ6DQo+IA0KPiAtIG1lbWNvcHkgcGFy dCBvZiB0b3AgbGV2ZWwgUFRFIG9uIGVudGVyLCBiemVybyBvbiBleGl0DQo+IC0gVExCIGZsdXNo IGJvdGggb24gZW50ZXIgYW5kIGV4aXQuDQo+IA0KPiBJTUhPIGl0IHdvdWxkIG1ha2UgbXVjaCBt b3JlIHRoYW4gMzAlIG92ZXJoZWFkIGluIG1hbnkgY2FzZXMuIGFtIGkgd3Jvbmc/DQo+IA0KPj4g dGhlcmUncyBzbWFsbCBwYXJ0cyB0aGF0IGhhdmUgdG8gc3RheSBtYXBwZWQgZm9yIHZhcmlvdXMg YXJjaGl0ZWN0dXJhbCByZWFzb25zLiBUaGlzIG1lYW5zIEtBU0xSIG9uIHRoZXNlIENQVXMgbGlr ZWx5IGNhbiBuZXZlciBiZQ0KPj4gZWZmZWN0aXZlIHNpbmNlIG1lbHRkb3duIHdpbGwgbGV0IHlv dSBmaW5kIHdoYXQgdGhlIHRyYXAgYWRkcmVzcyBpcyBhbmQgZnJvbSB0aGF0IGZpbmQgdGhlIGtl cm5lbCAodGhvdWdoIHRoZXJlJ3Mgc29tZSBydW1ibGluZ3MNCj4+IHRoYXQgdGhlIGluZGlyZWN0 aW9uIExpbnV4IGlzIGRvaW5nIHdpbGwgc3VmZmljZSkuDQo+PiBXYXJuZXINCj4+IA0KPiBfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPiBmcmVlYnNkLWFy Y2hAZnJlZWJzZC5vcmcgbWFpbGluZyBsaXN0DQo+IGh0dHBzOi8vbGlzdHMuZnJlZWJzZC5vcmcv bWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLWFyY2gNCj4gVG8gdW5zdWJzY3JpYmUsIHNlbmQgYW55 IG1haWwgdG8gImZyZWVic2QtYXJjaC11bnN1YnNjcmliZUBmcmVlYnNkLm9yZyINCg0K From owner-freebsd-arch@freebsd.org Wed Jan 10 03:52:32 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2679E63D1F for ; Wed, 10 Jan 2018 03:52:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 797E280806 for ; Wed, 10 Jan 2018 03:52:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22a.google.com with SMTP id c102so9275267itd.0 for ; Tue, 09 Jan 2018 19:52:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:from:date:message-id:subject:to; bh=1CixJyPzJxFxiuQuQiuGkT0PDV6idY+F+UtPbza2UIQ=; b=dnCtncOGvk8hgXZhctH36JgMF3ElCeFlIw80PcQx9wMOF91V+rp01Skq1P1JJNXh0e drhL0EVjRW+UG0AlPvZhtjnt5enZL9udaBDP0opnyiNhANwpa3HvdNMg7+lzW0yabacV 9N2xXmDdEB03E95KPclG4JCQ6IDsYUnA0qdnQUiPFVo6UocL+rzurMvVLSyv4udGZfxu O1xNpPVvIn4Mpp/PmPYtHHKWZAXJKl/uiVtyGckni/sTldtFrBZkWgIzhMcSEHKXiDiw QVplV0AM3bAJoxtnMm3RK/FBvcZUd/e2utknaYkSF+7Y0KzMIiHpJO9yVaZR7qLRR4ha QYDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=1CixJyPzJxFxiuQuQiuGkT0PDV6idY+F+UtPbza2UIQ=; b=r0t5TvUbbxjL6xDpY0+nW8uWnqW7wcttXwk+frLf0Wssbz4EtTTuEbqDQ3WdgCdBgF 7hVnLJYrbulFWLKmF6Foc9XGL6ToKemt0Ic7pVMOxV1P7MqmaSP/ugTYlqtKMtbQRWmH 8eqDUKvld2sgQClBrvoiTUXFSjQYnVgIjos8DLQaei8KOM565EZ+R9N2Rz/cC8PlIf6d +PmhOJKLzJKdjAyVg97b3TncCRBfktu2+sN03mMctLQCnIBggX2GShaMwMnIVYr6zLdQ AGYaR6Zc8LGuUkXJ0+qQeU8FqsE1rQtMR30Z1Kopfxmozc0G8aRbZv688jZIRIENAJSG 5Lbw== X-Gm-Message-State: AKwxytcG6EXpG/bWl0zb39ucPuWPn1kp3nhjO5Qn1AKxmSOvmBUQuVpx wa61IqUTuTiM9dH7Aj9MJUckUyjIKzWOpbf6g9pXvDQh X-Google-Smtp-Source: ACJfBovFhfCHkEbIGjfFfHu2LYSiuIGIh4wN7XpFAFjZaGFsUH09KlIM2Cs/x6jRLdubMeAkY0nQW69UMDTVN74c4r0= X-Received: by 10.36.91.210 with SMTP id g201mr14008668itb.50.1515556351345; Tue, 09 Jan 2018 19:52:31 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.160.217 with HTTP; Tue, 9 Jan 2018 19:52:30 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:18a2:a4f7:170:8dd9] From: Warner Losh Date: Tue, 9 Jan 2018 20:52:30 -0700 X-Google-Sender-Auth: mMPIUMD7pqa6PCRAgY81fAiRQso Message-ID: Subject: To be retired soon... To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 03:52:32 -0000 About 18 months ago, there was a thread that talked about retiring older storage devices and some ISA devices that have outlived their usefulness. Here's the list that John Baldwin posted. - Older storage adapters: - aha (ISA) - adv (ISA / PCI) - adw (PCI)? - bt (ISA / PCI) - aic (ISA / PCCard) - dpt (ISA / PCI) - ncv (PCCard / PCI) - nsp (PCCard) - stg (ISA / PCCard / PCI) - mse(4) (ISA-only non-PS-2, non-serial mouse) - joy(4) (ISA-only, was on various Sound Cards, etc., but I haven't seen a "game port" on a modern box in a long while) I plan on marking these drivers with a new API ( https://reviews.freebsd.org/D13818) in the coming days saying they will be removed in 12.x (meaning not in 12.0 or later). Man pages will be updated as well. I then plan to MFC all this code to 11 shortly after. Current users will start to receive warnings. I get that we should have done this sooner given our loosely observed deprecation practices in the past, but we're doing it now (which beats how we've done this in the past, which is why I'm adding the gone_in() to allow things we plan on retiring in 13 before 12-stable is branched). In about a month (sometime after Feb 15th), I plan on removing these from FreeBSD, absent any actual users reporting that recent FreeBSD 12 systems work. There are a number of special cases in CAM that be removed once these are gone, as well as code that's only used by these drivers. I've spoken to Scott Long about these drivers, and he's 100% on-board with retiring them. As far as he knows, the only parallel SCSI driver that works is mpt with LSI 2x320 cards, so these cards are (a) too slow to even be remotely relevant (b) have all kinds of problems that haven't been resolved in 5-ish years of reports of malfunction. mse/joy are also at least 15 years past their even marginal relevance. Comments? Warner From owner-freebsd-arch@freebsd.org Wed Jan 10 04:07:08 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 727D9E648C4 for ; Wed, 10 Jan 2018 04:07:08 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (mail.soaustin.net [192.108.105.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.soaustin.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5873B8102D for ; Wed, 10 Jan 2018 04:07:08 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from lonesome.com (bones.soaustin.net [192.108.105.22]) by mail.soaustin.net (Postfix) with ESMTPSA id E6AB11212; Tue, 9 Jan 2018 22:07:00 -0600 (CST) Date: Tue, 9 Jan 2018 22:06:59 -0600 From: Mark Linimon To: Warner Losh Cc: "freebsd-arch@freebsd.org" Subject: Re: To be retired soon... Message-ID: <20180110040659.GA18065@lonesome.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 04:07:08 -0000 On Tue, Jan 09, 2018 at 08:52:30PM -0700, Warner Losh wrote: > - aha (ISA) > - bt (ISA / PCI) Oh no, how am I ever going to go back to the 1990s now? Uh, wait. Actually, I hated that decade's music. So, forget it. mcl (yes, there are probably one of each in the junkbox still ...) From owner-freebsd-arch@freebsd.org Wed Jan 10 06:45:13 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FC8FE6C72C for ; Wed, 10 Jan 2018 06:45:13 +0000 (UTC) (envelope-from grog@lemis.com) Received: from www.lemis.com (www.lemis.com [208.86.226.86]) by mx1.freebsd.org (Postfix) with ESMTP id 297641A1E for ; Wed, 10 Jan 2018 06:45:12 +0000 (UTC) (envelope-from grog@lemis.com) Received: from eureka.lemis.com (lemis.com [192.109.197.81]) by www.lemis.com (Postfix) with ESMTP id C84F51B72806; Wed, 10 Jan 2018 06:45:05 +0000 (UTC) Received: by eureka.lemis.com (Postfix, from userid 1004) id 663954494B4; Wed, 10 Jan 2018 17:45:04 +1100 (AEDT) Date: Wed, 10 Jan 2018 17:45:04 +1100 From: Greg 'groggy' Lehey To: Warner Losh Cc: "freebsd-arch@freebsd.org" Subject: Re: To be retired soon... Message-ID: <20180110064504.GC27612@eureka.lemis.com> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VywGB/WGlW4DM4P8" Content-Disposition: inline In-Reply-To: Organization: The FreeBSD Project Phone: +61-3-5346-1370, +61-3-5309-0418 Mobile: 0401 265 606. Use only as instructed. WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 06:45:13 -0000 --VywGB/WGlW4DM4P8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tuesday, 9 January 2018 at 20:52:30 -0700, Warner Losh wrote: > About 18 months ago, there was a thread that talked about retiring older > storage devices and some ISA devices that have outlived their usefulness. > > Here's the list that John Baldwin posted. > > ... > > I plan on marking these drivers with a new API ( > https://reviews.freebsd.org/D13818) in the coming days saying they > will be removed in 12.x (meaning not in 12.0 or later). Is that "not" intended? 12.x sounds very much like 12.0 or later. Or is it maybe "not in 12.0, but later in 12"? Greg -- Sent from my desktop computer. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA --VywGB/WGlW4DM4P8 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAlpVtnAACgkQIubykFB6QiO+OQCaAy0NsWzoidL/cogWhD3gy5Jm kuYAnA87x+4c88wysWu8QMXy/e1KCpFg =JhFh -----END PGP SIGNATURE----- --VywGB/WGlW4DM4P8-- From owner-freebsd-arch@freebsd.org Wed Jan 10 06:50:45 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03D6CE6CAC8 for ; Wed, 10 Jan 2018 06:50:45 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail.karels.net (mail.karels.net [216.160.39.52]) by mx1.freebsd.org (Postfix) with ESMTP id B09611C45 for ; Wed, 10 Jan 2018 06:50:44 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.15.2/8.15.2) with ESMTP id w0A6OikT079235; Wed, 10 Jan 2018 00:24:44 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201801100624.w0A6OikT079235@mail.karels.net> To: Warner Losh cc: "freebsd-arch@freebsd.org" From: Mike Karels Reply-to: mike@karels.net Subject: Re: To be retired soon... In-reply-to: Your message of Tue, 09 Jan 2018 20:52:30 -0700. MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <79233.1515565484.1@mail.karels.net> Date: Wed, 10 Jan 2018 00:24:44 -0600 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 06:50:45 -0000 > About 18 months ago, there was a thread that talked about retiring older > storage devices and some ISA devices that have outlived their usefulness. > Here's the list that John Baldwin posted. > - Older storage adapters: > - aha (ISA) > - adv (ISA / PCI) > - adw (PCI)? > - bt (ISA / PCI) > - aic (ISA / PCCard) > - dpt (ISA / PCI) > - ncv (PCCard / PCI) > - nsp (PCCard) > - stg (ISA / PCCard / PCI) > - mse(4) (ISA-only non-PS-2, non-serial mouse) > - joy(4) (ISA-only, was on various Sound Cards, etc., but I haven't seen > a "game port" on a modern box in a long while) > I plan on marking these drivers with a new API ( > https://reviews.freebsd.org/D13818) in the coming days saying they will be > removed in 12.x (meaning not in 12.0 or later). Man pages will be updated > as well. I then plan to MFC all this code to 11 shortly after. Current > users will start to receive warnings. I get that we should have done this > sooner given our loosely observed deprecation practices in the past, but > we're doing it now (which beats how we've done this in the past, which is > why I'm adding the gone_in() to allow things we plan on retiring in 13 > before 12-stable is branched). In about a month (sometime after Feb 15th), > I plan on removing these from FreeBSD, absent any actual users reporting > that recent FreeBSD 12 systems work. There are a number of special cases in > CAM that be removed once these are gone, as well as code that's only used > by these drivers. > I've spoken to Scott Long about these drivers, and he's 100% on-board with > retiring them. As far as he knows, the only parallel SCSI driver that works > is mpt with LSI 2x320 cards, so these cards are (a) too slow to even be > remotely relevant (b) have all kinds of problems that haven't been resolved > in 5-ish years of reports of malfunction. mse/joy are also at least 15 > years past their even marginal relevance. > Comments? > Warner Agreed. But for the future, note that mpt is used in VMware ESXi emulation when FreeBSD/amd64 is selected (I think also for i386). VMware tends to emulate rather old hardware. We should retain drivers commonly used in virtual environments too. Mike From owner-freebsd-arch@freebsd.org Wed Jan 10 06:54:02 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D66C6E6CE75 for ; Wed, 10 Jan 2018 06:54:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 672EE1FF3 for ; Wed, 10 Jan 2018 06:54:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-wm0-x22d.google.com with SMTP id b141so24734556wme.1 for ; Tue, 09 Jan 2018 22:54:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=DZ8mtZqeS/ZYuNzFRcyD/WmPfA8zlYRSo+uD0X4ZxJg=; b=go81duZp+hBha0Qmub+Qq/Cw1Rm5YCsvFdCt2pA6llk+1MbAqKPZictU4nHREOP0DK FVhJRN6s9QYZMXerpfHyi7OQzEWmesDHFSiyNF1K8rMfha9BFPFzjz1mEPFdO4q7iGX4 RzrUFBO1yVsKbxqZ5QioEyekKhAcIj2lWyVE1Bc67zfImYpJY0JUBBGhnNmE2z0ecBL/ m/Lv6KamLkByJTERZWXO7HX75QG6MT7t1Z5kfWcJG2lk+cvJotm8b72XPZ+bD1/tspAw LyMJeudF6hqCEw7IJWucmuQKLEiAOuW3MsVKkOWqHOlq+POvvjTDuS8Vh9syxWp31pLc dKMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=DZ8mtZqeS/ZYuNzFRcyD/WmPfA8zlYRSo+uD0X4ZxJg=; b=nZzqO8RuF1Igp5NMlXjfGyGN7vMQ0qWtb90iK/6nxJUp4tmXEvSn0MWPfyNYPQPgBM yXPcb9P2n7WOVqrLuaPabq3TJ9CA2HpLrEmH9Emmjrd4E9nuFbPqJKxYKua10Uiw4L/P U3YS0lKJ+eOh7ihnCKxUG8x6pb+DLHyAWQbhxlZ64o9piFh+Z1dRQIntcIi9ayV0VYq4 J6emCetSGQAJO10qYW+JSSx3GXulx6UKnvVTQkaeq7SHuz02IK3b0JXcWlkUckKiFqe2 SAtVgQWjN4tIo8F8rR+U3v8Lhyty6dyp0TIS0mWgAaEvSizy0+6QhX6fTy4wJDTryKMI E4hw== X-Gm-Message-State: AKwxytdJfHyWSF5RUAMRu0tgRnc58t9cPD2rr6RIBTqOhNsQ9+4j8o5G UZf+8/yt4+VDXbkxC5EldK5rl1tkNF2PGX+QmFgNlg== X-Google-Smtp-Source: ACJfBovJzfq/JTRV7ogejMtmtrpB0Otd5yvB1jiMFmjnMUGWjFnyjiebW2NZcNkUts2MDOXnrqTLiJL0bzWbJgYyY7U= X-Received: by 10.80.151.22 with SMTP id c22mr326724edb.225.1515567240639; Tue, 09 Jan 2018 22:54:00 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.80.195.88 with HTTP; Tue, 9 Jan 2018 22:53:59 -0800 (PST) X-Originating-IP: [2603:300b:6:5100:a84a:1e5c:a3af:9943] Received: by 10.80.195.88 with HTTP; Tue, 9 Jan 2018 22:53:59 -0800 (PST) In-Reply-To: <201801100624.w0A6OikT079235@mail.karels.net> References: <201801100624.w0A6OikT079235@mail.karels.net> From: Warner Losh Date: Tue, 9 Jan 2018 23:53:59 -0700 X-Google-Sender-Auth: xDNvo28-BhCaEonJKzU8zEPW9B4 Message-ID: Subject: Re: To be retired soon... To: Mike Karels Cc: freebsd-arch@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 06:54:02 -0000 On Jan 9, 2018 11:24 PM, "Mike Karels" wrote: > About 18 months ago, there was a thread that talked about retiring older > storage devices and some ISA devices that have outlived their usefulness. > Here's the list that John Baldwin posted. > - Older storage adapters: > - aha (ISA) > - adv (ISA / PCI) > - adw (PCI)? > - bt (ISA / PCI) > - aic (ISA / PCCard) > - dpt (ISA / PCI) > - ncv (PCCard / PCI) > - nsp (PCCard) > - stg (ISA / PCCard / PCI) > - mse(4) (ISA-only non-PS-2, non-serial mouse) > - joy(4) (ISA-only, was on various Sound Cards, etc., but I haven't seen > a "game port" on a modern box in a long while) > I plan on marking these drivers with a new API ( > https://reviews.freebsd.org/D13818) in the coming days saying they will be > removed in 12.x (meaning not in 12.0 or later). Man pages will be updated > as well. I then plan to MFC all this code to 11 shortly after. Current > users will start to receive warnings. I get that we should have done this > sooner given our loosely observed deprecation practices in the past, but > we're doing it now (which beats how we've done this in the past, which is > why I'm adding the gone_in() to allow things we plan on retiring in 13 > before 12-stable is branched). In about a month (sometime after Feb 15th), > I plan on removing these from FreeBSD, absent any actual users reporting > that recent FreeBSD 12 systems work. There are a number of special cases in > CAM that be removed once these are gone, as well as code that's only used > by these drivers. > I've spoken to Scott Long about these drivers, and he's 100% on-board with > retiring them. As far as he knows, the only parallel SCSI driver that works > is mpt with LSI 2x320 cards, so these cards are (a) too slow to even be > remotely relevant (b) have all kinds of problems that haven't been resolved > in 5-ish years of reports of malfunction. mse/joy are also at least 15 > years past their even marginal relevance. > Comments? > Warner Agreed. But for the future, note that mpt is used in VMware ESXi emulation when FreeBSD/amd64 is selected (I think also for i386). VMware tends to emulate rather old hardware. We should retain drivers commonly used in virtual environments too. Agreed. mpt is not in danger. Nothing on this list goes that description. Warner From owner-freebsd-arch@freebsd.org Wed Jan 10 08:26:25 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01700E71AF4 for ; Wed, 10 Jan 2018 08:26:25 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C793D688A5 for ; Wed, 10 Jan 2018 08:26:24 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with ESMTPA id ZBiIepqa4Z8gBZBiJeVQV3; Wed, 10 Jan 2018 01:26:24 -0700 X-Authority-Analysis: v=2.2 cv=M/g9E24s c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=kj9zAlcOel0A:10 a=RgaUWeydRksA:10 a=-FGs326eAAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=0LxWCHhTRZgdS47yclAA:9 a=CjuIK1q_8ugA:10 a=7Nw9HX5Nqxt2AnyyOhBr:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id D565218BDF; Wed, 10 Jan 2018 00:26:21 -0800 (PST) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id w0A8QLvc010525; Wed, 10 Jan 2018 00:26:21 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id w0A8QKDD010471; Wed, 10 Jan 2018 00:26:21 -0800 (PST) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201801100826.w0A8QKDD010471@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Mark Linimon cc: Warner Losh , "freebsd-arch@freebsd.org" Subject: Re: To be retired soon... In-Reply-To: Message from Mark Linimon of "Tue, 09 Jan 2018 22:06:59 -0600." <20180110040659.GA18065@lonesome.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 10 Jan 2018 00:26:20 -0800 X-CMAE-Envelope: MS4wfDErxtVN9q0rw3cVPpmi+JCavYPscOaiVz2k602gEgEiqMG0WaiNSVnFu2jBFZwgHCSIYdkxxpnF6vKEsA4pE8Kfs1CC8HIH67cxcRthPcyAl39UCw8x zY7hqCIiw4rir9Hw222nT5xqcAyi+Kt6kK+PGNMBi6TwvCf4ah+VMKlNIdlD2bDNZKOYsvWH/ibhEQv0cVyea5JxEE+HWxeAmroxCMqstJ0iml+MXkzWSPrt 4R1Xfn7cwuPIANW+1wztsA== X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 08:26:25 -0000 In message <20180110040659.GA18065@lonesome.com>, Mark Linimon writes: > On Tue, Jan 09, 2018 at 08:52:30PM -0700, Warner Losh wrote: > > - aha (ISA) > > - bt (ISA / PCI) > > Oh no, how am I ever going to go back to the 1990s now? > > Uh, wait. Actually, I hated that decade's music. We were so young then. Cocky too. > > So, forget it. > > mcl > > (yes, there are probably one of each in the junkbox still ...) I have an AHA-1542 in storage. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-arch@freebsd.org Wed Jan 10 13:04:01 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BBD8E7F3EB for ; Wed, 10 Jan 2018 13:04:01 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:300:2185:a:dead:bad:faff]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47B70739C8; Wed, 10 Jan 2018 13:04:00 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [104.207.135.49]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id w0AD3xSc043083; Wed, 10 Jan 2018 13:03:59 GMT (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id w0AD3w2h043082; Wed, 10 Jan 2018 13:03:58 GMT (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <201801101303.w0AD3w2h043082@donotpassgo.dyslexicfish.net> Date: Wed, 10 Jan 2018 13:03:58 +0000 Organization: Dyslexic Fish To: imp@bsdimp.com, grog@freebsd.org Cc: freebsd-arch@freebsd.org Subject: Re: To be retired soon... References: <20180110064504.GC27612@eureka.lemis.com> In-Reply-To: <20180110064504.GC27612@eureka.lemis.com> User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [104.207.135.49]); Wed, 10 Jan 2018 13:03:59 +0000 (GMT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 13:04:01 -0000 "Greg 'groggy' Lehey" wrote: > On Tuesday, 9 January 2018 at 20:52:30 -0700, Warner Losh wrote: > > I plan on marking these drivers with a new API ( > > https://reviews.freebsd.org/D13818) in the coming days saying they > > will be removed in 12.x (meaning not in 12.0 or later). > > Is that "not" intended? 12.x sounds very much like 12.0 or later. Or > is it maybe "not in 12.0, but later in 12"? Ah, the wonders of the English language! I read it in another alternative way: that they'd be removed whilst 12 was still CURRENT i.e.: ".... will be removed in 12.x (meaning they will not exist in 12.0 or later)" cheers, Jamie From owner-freebsd-arch@freebsd.org Thu Jan 11 00:19:04 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 758EDE7FCD1 for ; Thu, 11 Jan 2018 00:19:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 621EB74547 for ; Thu, 11 Jan 2018 00:19:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 5E3BAE7FCD0; Thu, 11 Jan 2018 00:19:04 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DD24E7FCCF for ; Thu, 11 Jan 2018 00:19:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 355F874546 for ; Thu, 11 Jan 2018 00:19:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (astound-66-234-199-215.ca.astound.net [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 8A0C610A7DB for ; Wed, 10 Jan 2018 19:19:01 -0500 (EST) From: John Baldwin To: arch@freebsd.org Subject: Ranting about OCF / crypto(9) Date: Wed, 10 Jan 2018 16:18:52 -0800 Message-ID: <3790717.UIxaijsHl3@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Wed, 10 Jan 2018 19:19:01 -0500 (EST) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 00:19:04 -0000 While working on hooking the ccr(4) driver into our in-kernel crypto framework (along with some out-of-tree patches to extend OpenSSL's /dev/crypto engine to support AES-CTR/XTS/GCM and some further changes to do zero-copy), I've run into several bumps / oddities in OCF. I'm probably going to miss several of them, but here's at least a start of a list of things. In some cases I have some suggestions on improvements. I will try to start with more broad / higher-level items first before diving into minutiae: - OCF is over flexible and overly broad. Rather than supporting arbitrary stacking of transforms (and arbitrary depths), I think we should probably aim to support more specific workloads and support them well. To my mind the classes of things we should support are probably: - Simple block cipher requests. - Simple "hash a buffer" requests. (Both HMAC and non-HMAC) - IPSec-style requests (combined auth and encryption using "encrypt-then-mac" with an optional AAD region before the ciphertext). Note that geli requests fall into this type. - TLS-style requests (using TLS's different methods of combining auth and encryption methods when those are separate) - Simple compression / decompression requests. While this isn't "crypto", per se, I do think it is probably still simpler to manage this via OCF than a completely separate interface. In terms of algorithms, I suspect there are some older algorithms we could drop. Modern hardware doesn't offload DES for example. Both ccr(4) and aesni(4) only support AES for encryption. We do need to keep algorithms required for IPSec in the kernel, but we could probably drop some others? - To better support OpenSSL's engine, the /dev/crypto hash interface should not require monotonic buffers, but support requests for large buffers that span multiple requests (so you can do something akin to the 'Init' / 'Update' (N times) / 'Final' model existing software hashing APIs use). In particular, the bigger win for hashing in hardware is when you can offload the hashing of a large thing rather than small requests. - To better support OpenSSL's engine, the /dev/crypto hash interface should support "plain" hash algorithms such as SHA* without an HMAC. By default OpenSSL's engine interface does the HMAC-specific bits (generating pads, etc.) in software and only defers to the engine for the raw hash (e.g. if you use the HMAC() function from libcrypto it will only ask the engine interface for a raw hash, not for an HMAC hash). - To better support OpenSSL's engine, the /dev/crypto cipher interface should also support non-monolithic buffers. The existing engine does this now by copying the last block of the output data out as a saved IV to use for a subsequent request, but it might be nicer to be more formal here and return the IV to userland for non-"final" cipher requests. - The interface between the crypto layer and backend drivers should _not_ use integer session IDs. This is rediculously dumb and inefficient. All the drivers have silly algorithms to try to manage growable arrays that can be indexed by the returned session ID. Instead, drivers should be able to return a 'void *' cookie when creating a session and get that cookie pointer as an argument to the 'process' and 'freesession' callbacks. Imagine if vnodes used an i-node number rather than 'v_data' and you'd have the model OCF uses. I don't mind if we have a kind of generic 'session' structure that we export to drivers and pass in the callbacks and the drivers get to use a 'foo_data' member of. - The interface to describe crypto requests needs to move away from arbitrary linked lists of descriptors. We should just have a single "session" structure that assumes you have one cipher and one auth with a "mode" member to indicate the particular direction / combination. Likewise, the description of a request needs to have a similar assumption. The structures used by the /dev/crypto ioctl's are a bit closer to what I think we should use compared to the linked-list thing we have now. Related is that we should be able to get rid of having the three separate "algorithms" for GCM hashes. For AES-GCM one would just say they are using AES-GCM and both the hash/tag and ciphertext would be valid inputs / outputs with a single key. - To support non-monolithic buffers from the OpenSSL engine, crypto requests to drivers also have to support non-monolithic buffers. This means having a notion of a buffer that may be at the start, middle, or end of a larger transformation (e.g. for hash only the start gets the IPAD, only the end gets the OPAD and returns a valid hash, etc., whereas for ciphers any non-end requests would return the IV to use for the next request). For drivers that have buffer size limits, it would be nice to expose those limits in the driver capabilities and depend on the upper layer to "split" requests such as happens now for disk drivers. - For hashing algorithms we should support a "verify" mode in addition to the current "compute" mode. The verify mode would accept a block of data to hash along with an expected mac and return a success / failure rather than an computed hash value. AES-GCM already works this way for decryption, but this would extend that mode for other hash algorithms (e.g. AES-CBC+SHA2-256-HMAC). Existing crypto co-processors (e.g. ccr(4)) already support these types of requests. Related is that we need to fix IPSec to treat EBADMSG errors from descryption as auth failure rather than encryption failure (right now AES-GCM auth failures are reported incorrectly in netstat -s due to this). - Sessions for a combined cipher + hash should also be tied to a specific way of combining the algorithms. Right now you can create a session for AES-CBC with a SHA hash and the driver has no way to know if you are going to do encrypt-then-mac or one of the other variants. We should include this in the session (so a given session can only be used for one type which is normally true anyway), and drivers can then only claim to support combinations they support. - The CRD_F_IV_PRESENT flag should be removed and replaced with a CRD_F_IV_INJECT flag which means "inject the IV". Right now the _lack_ of CRD_F_IV_PRESENT for encryption (but not decryption!) requests means "inject the IV". It would be clearer to just have a flag that is only set when you want the driver to take the action. - Speaking of IV handling, drivers have to do some extra handling for IVs including possibly generating them. I think the idea is that some co-processors might support generating IVs, but most of the drivers I've looked at just end up duplicating the same block of code to call arc4rand() for encryption requests without CRD_F_IV_EXPLICIT. I don't believe Linux tries to support this and instead always supplies an IV to the driver. I'd rather we do that and only depend on a flag to indicate where the IV is (crd_iv vs in the buffer). - The API for copying data to/from crypto buffers is a bit obtuse and limiting. Rather than accepting the crypto operation ('crp') as a parameter to describe the crypto buffer, the crypto_copyback() and crypto_copydata() functions accept various members of that function explicitly (e.g. crp_flags and crp_buf). However, in my experiments with zero-copy AES-GCM via /dev/crypto and OpenSSL it was convenient to store the AAD in a KVA buffer in the 'crp' and the payload to transform in an array of VM pages. However, for this model 'crp_buf' is useless. I ended up adding a wrapper API 'crypto_copyto' and 'crypto_copyfrom' which accept a 'crp' directly. Linux's API actually passes something akin to sglist as the description of the buffers in a crypto request. - We need to not treat accelerated software (e.g. AES-NI) as a hardware interface. Right now OCF's model of priorities when trying to choose a backend driver for a session only has two "levels" software vs hardware and aesni(4) (and the ARMv8 variant) are lumped into the hardware bucket so that they have precedence over the "dumb" software implementation. However, the accelerated software algorithms do need some of the same support features of the "dumb" software implementation (such as being scheduled on a thread pool to use CPU cycles) that are not needed by other "hardware" engines. OCF needs to understand this distinction. - Somewhat related, we should try to use accelerated software when possible (e.g. AES-CBC with SHA) doesn't use AES-NI unless the CPU supports accelerated SHA. Ideally for this case we'd still use AES-NI for the AES portion along with the software SHA implementation (and we'd do it one pass over the data rather than two when possible). - Sometimes a crypto driver might need to defer certain requests to software (e.g. ccr(4) has to do this for some GCM requests). In addition, there are some other cases when we might want requests from a single session to be sent to different backends (e.g. you may want to use accelerated software for requests below a certain size, and a crypto engine for larger requests. You might also want to take NUMA into account when choosing which backend crypto engine to dispatch a request to.) To that end, I think we want to have the ability for a single OCF session to support multiple backend sessions. One use case is that if I as a driver can't handle a request I'd like to be able to fail it with a special error code and have the crypto later fall back to software for me (and to use accelerated software if possible). Right now ccr(4) duplicates the "dumb" software for GCM requests it can't handle explicitly. Another use case might be failover if a hardware engine experiences a hardware failure. In theory it should be possible to fail over to a different driver at that point including resubmitting pending requests that weren't completed, and it should be possible (I think) to manage this in the crypto framework rather than in consumers like IPSec and GELI. Load distribution among backends might be another case to consider (e.g. GELI or ZFS encryption once that lands) if you have long- running sessions that spawn lots of self-contained requests. Note that if we want to spawn additional backend sessions on the fly (e.g. only create a software fallback session on demand if a driver fails a request with the "use software" magic error code), we will have to keep per-session state such as keys around. We probably already do that now, but this would definitely require doing that. One concern with some of these changes is that there are several drivers in the tree for older hardware that I'm not sure is really used anymore. That is an impediment to making changes to the crypto <-> driver interface if we can't find folks willing to at least test changes to those drivers if not maintain them. This is all I could think of today. What do other folks think? -- John Baldwin From owner-freebsd-arch@freebsd.org Thu Jan 11 06:01:44 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5791BE70094 for ; Thu, 11 Jan 2018 06:01:44 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2ECF9806F9 for ; Thu, 11 Jan 2018 06:01:44 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: by mailman.ysv.freebsd.org (Postfix) id 2D58FE70093; Thu, 11 Jan 2018 06:01:44 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CB3EE70091 for ; Thu, 11 Jan 2018 06:01:44 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-8.mit.edu (dmz-mailsec-scanner-8.mit.edu [18.7.68.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4720A806F0; Thu, 11 Jan 2018 06:01:42 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074425-ed9ff70000000aeb-41-5a56fc8d41e0 Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-8.mit.edu (Symantec Messaging Gateway) with SMTP id CD.41.02795.E8CF65A5; Thu, 11 Jan 2018 00:56:31 -0500 (EST) Received: from outgoing.mit.edu (OUTGOING-AUTH-1.MIT.EDU [18.9.28.11]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id w0B5uPl7017517; Thu, 11 Jan 2018 00:56:27 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id w0B5uK8U006391 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 11 Jan 2018 00:56:23 -0500 Date: Wed, 10 Jan 2018 23:56:21 -0600 From: Benjamin Kaduk To: John Baldwin Cc: arch@freebsd.org Subject: Re: Ranting about OCF / crypto(9) Message-ID: <20180111055620.GO72574@kduck.kaduk.org> References: <3790717.UIxaijsHl3@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3790717.UIxaijsHl3@ralph.baldwin.cx> User-Agent: Mutt/1.9.1 (2017-09-22) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrKIsWRmVeSWpSXmKPExsUixG6notv/JyzK4PFnPYslM+YxW+w9cp3Z gcljxqf5LAGMUVw2Kak5mWWpRfp2CVwZvzu2MxUszqhYuG8xewPj1sAuRk4OCQETiWfrNrJ1 MXJxCAksZpLY/fMSE4SzkVFi29mDrBDOVSaJvzNWsYC0sAioSjzcsIMRxGYTUJN4vLcZqIiD Q0RASWLqNzUQk1lAROLKIzaQCmEBLYknvdeYQWxeoGXz7k4B6xQSMJR48ns1I0RcUOLkzCdg 05mB6m/8e8kEMUZaYvk/DhCTU8BIYv1Kf5AKUQFlib19h9gnMArMQtI8C0nzLITmBYzMqxhl U3KrdHMTM3OKU5N1i5MT8/JSi3Qt9HIzS/RSU0o3MYLD0kV1B+Ocv16HGAU4GJV4eBmEw6KE WBPLiitzDzFKcjApifIGcoZGCfEl5adUZiQWZ8QXleakFh9ilOBgVhLhXRwIVM6bklhZlVqU D5OS5mBREuf1MNGOEhJITyxJzU5NLUgtgsnKcHAoSfA2/QZqFCxKTU+tSMvMKUFIM3Fwggzn ARqu9AdkeHFBYm5xZjpE/hSjMceNF6/bmDmmLXvTxizEkpeflyolzisLMk4ApDSjNA9uGii1 SGTvr3nFKA70nDBvP0gVDzAtwc17BbSKCWjV+Y2hIKtKEhFSUg2MmtwrO6rUl28UZmOfvvzY KpVrXBL2i+wjFxz57nwmmFXjq4HB59U59r6PvG1/VVv0+nyIW8MpZMls/+9m85ImiWPd+haS p1xOn2RIPf3l7/anB57e2DJ3h+jmZ0uOSxwKSL406XMSI+fkwtKiDVXth7W29set9zusqx3o Z337vUTfNEnV1rOPlViKMxINtZiLihMB8LSpvAgDAAA= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 06:01:44 -0000 Replying mostly with my upstream OpenSSL hat on, not least to note my regret that OpenSSL's historic poor API design choices reflect so heavily herein... On Wed, Jan 10, 2018 at 04:18:52PM -0800, John Baldwin wrote: > While working on hooking the ccr(4) driver into our in-kernel crypto > framework (along with some out-of-tree patches to extend OpenSSL's > /dev/crypto engine to support AES-CTR/XTS/GCM and some further changes to Note that on master/proto-1.1.1, the cryptodev engine got swapped out for the one from https://github.com/openssl/openssl/pull/3744, since the OpenBSD folks would not agree to relicense. (Apparently the move to the Apache license is probably actually going to happen.) But I'd be happy to help review patches to add additional functionality. > do zero-copy), I've run into several bumps / oddities in OCF. I'm probably > going to miss several of them, but here's at least a start of a list of > things. In some cases I have some suggestions on improvements. > > I will try to start with more broad / higher-level items first before > diving into minutiae: > > - OCF is over flexible and overly broad. Rather than supporting > arbitrary stacking of transforms (and arbitrary depths), I think we > should probably aim to support more specific workloads and support > them well. To my mind the classes of things we should support are > probably: > > - Simple block cipher requests. > - Simple "hash a buffer" requests. (Both HMAC and non-HMAC) > - IPSec-style requests (combined auth and encryption using > "encrypt-then-mac" with an optional AAD region before the > ciphertext). Note that geli requests fall into this type. > - TLS-style requests (using TLS's different methods of > combining auth and encryption methods when those are > separate) My brain is trying to ask "is that really a good idea?" about putting the old (bad/broken) TLS mac+encrypt schemes in the kernel, from the vantage point of making it easier to do insecure things. > - Simple compression / decompression requests. While this isn't > "crypto", per se, I do think it is probably still simpler to > manage this via OCF than a completely separate interface. Probably, though perhaps less so after the removal of arbitrary stacking depths. And mixing compression with encryption has its own risks, of course. > In terms of algorithms, I suspect there are some older algorithms > we could drop. Modern hardware doesn't offload DES for example. > Both ccr(4) and aesni(4) only support AES for encryption. We > do need to keep algorithms required for IPSec in the kernel, but > we could probably drop some others? Yes, it's probably time for DES to go. Maybe others as well. > - To better support OpenSSL's engine, the /dev/crypto hash interface > should not require monotonic buffers, but support requests for > large buffers that span multiple requests (so you can do something > akin to the 'Init' / 'Update' (N times) / 'Final' model existing > software hashing APIs use). In particular, the bigger win for > hashing in hardware is when you can offload the hashing of a large > thing rather than small requests. Something of an aside, but we are growing some support for "one-shot" stuff as a result of Ed25519 support landing, but the Init/Update/Final mindset is still pretty baked in, for now. > - To better support OpenSSL's engine, the /dev/crypto hash interface > should support "plain" hash algorithms such as SHA* without an > HMAC. By default OpenSSL's engine interface does the HMAC-specific > bits (generating pads, etc.) in software and only defers to the > engine for the raw hash (e.g. if you use the HMAC() function from > libcrypto it will only ask the engine interface for a raw hash, > not for an HMAC hash). (https://github.com/openssl/openssl/issues/977; patches welcome for improving the ENGINE interface) > - To better support OpenSSL's engine, the /dev/crypto cipher > interface should also support non-monolithic buffers. The existing > engine does this now by copying the last block of the output data > out as a saved IV to use for a subsequent request, but it might be > nicer to be more formal here and return the IV to userland for > non-"final" cipher requests. > > - The interface between the crypto layer and backend drivers should > _not_ use integer session IDs. This is rediculously dumb and > inefficient. All the drivers have silly algorithms to try to manage > growable arrays that can be indexed by the returned session ID. > Instead, drivers should be able to return a 'void *' cookie when > creating a session and get that cookie pointer as an argument to > the 'process' and 'freesession' callbacks. Imagine if vnodes used > an i-node number rather than 'v_data' and you'd have the model OCF > uses. I don't mind if we have a kind of generic 'session' structure > that we export to drivers and pass in the callbacks and the drivers > get to use a 'foo_data' member of. > > - The interface to describe crypto requests needs to move away from > arbitrary linked lists of descriptors. We should just have a > single "session" structure that assumes you have one cipher and > one auth with a "mode" member to indicate the particular direction > / combination. Likewise, the description of a request needs to > have a similar assumption. The structures used by the /dev/crypto > ioctl's are a bit closer to what I think we should use compared to > the linked-list thing we have now. Related is that we should be > able to get rid of having the three separate "algorithms" for GCM > hashes. For AES-GCM one would just say they are using AES-GCM > and both the hash/tag and ciphertext would be valid inputs / outputs > with a single key. > > - To support non-monolithic buffers from the OpenSSL engine, crypto > requests to drivers also have to support non-monolithic buffers. > This means having a notion of a buffer that may be at the start, > middle, or end of a larger transformation (e.g. for hash only the > start gets the IPAD, only the end gets the OPAD and returns a > valid hash, etc., whereas for ciphers any non-end requests would > return the IV to use for the next request). > > For drivers that have buffer size limits, it would be nice to expose > those limits in the driver capabilities and depend on the upper layer > to "split" requests such as happens now for disk drivers. > > - For hashing algorithms we should support a "verify" mode in addition > to the current "compute" mode. The verify mode would accept a block > of data to hash along with an expected mac and return a success > / failure rather than an computed hash value. AES-GCM already works > this way for decryption, but this would extend that mode for other > hash algorithms (e.g. AES-CBC+SHA2-256-HMAC). Existing crypto > co-processors (e.g. ccr(4)) already support these types of requests. > > Related is that we need to fix IPSec to treat EBADMSG errors from > descryption as auth failure rather than encryption failure (right > now AES-GCM auth failures are reported incorrectly in netstat -s > due to this). > > - Sessions for a combined cipher + hash should also be tied to a > specific way of combining the algorithms. Right now you can > create a session for AES-CBC with a SHA hash and the driver has no > way to know if you are going to do encrypt-then-mac or one of the > other variants. We should include this in the session (so a given > session can only be used for one type which is normally true anyway), > and drivers can then only claim to support combinations they > support. > > - The CRD_F_IV_PRESENT flag should be removed and replaced with > a CRD_F_IV_INJECT flag which means "inject the IV". Right now > the _lack_ of CRD_F_IV_PRESENT for encryption (but not decryption!) > requests means "inject the IV". It would be clearer to just have > a flag that is only set when you want the driver to take the > action. > > - Speaking of IV handling, drivers have to do some extra handling for > IVs including possibly generating them. I think the idea is that > some co-processors might support generating IVs, but most of the > drivers I've looked at just end up duplicating the same block of > code to call arc4rand() for encryption requests without > CRD_F_IV_EXPLICIT. I don't believe Linux tries to support this and > instead always supplies an IV to the driver. I'd rather we do that > and only depend on a flag to indicate where the IV is (crd_iv vs > in the buffer). > > - The API for copying data to/from crypto buffers is a bit obtuse and > limiting. Rather than accepting the crypto operation ('crp') as > a parameter to describe the crypto buffer, the crypto_copyback() > and crypto_copydata() functions accept various members of that > function explicitly (e.g. crp_flags and crp_buf). However, in my > experiments with zero-copy AES-GCM via /dev/crypto and OpenSSL it > was convenient to store the AAD in a KVA buffer in the 'crp' and > the payload to transform in an array of VM pages. However, for > this model 'crp_buf' is useless. I ended up adding a wrapper API > 'crypto_copyto' and 'crypto_copyfrom' which accept a 'crp' directly. > Linux's API actually passes something akin to sglist as the > description of the buffers in a crypto request. > > - We need to not treat accelerated software (e.g. AES-NI) as a > hardware interface. Right now OCF's model of priorities when > trying to choose a backend driver for a session only has two > "levels" software vs hardware and aesni(4) (and the ARMv8 variant) > are lumped into the hardware bucket so that they have precedence > over the "dumb" software implementation. However, the accelerated > software algorithms do need some of the same support features of > the "dumb" software implementation (such as being scheduled on a > thread pool to use CPU cycles) that are not needed by other "hardware" > engines. OCF needs to understand this distinction. > > - Somewhat related, we should try to use accelerated software when > possible (e.g. AES-CBC with SHA) doesn't use AES-NI unless the > CPU supports accelerated SHA. Ideally for this case we'd still > use AES-NI for the AES portion along with the software SHA > implementation (and we'd do it one pass over the data rather than > two when possible). > > - Sometimes a crypto driver might need to defer certain requests to > software (e.g. ccr(4) has to do this for some GCM requests). In > addition, there are some other cases when we might want requests > from a single session to be sent to different backends (e.g. you > may want to use accelerated software for requests below a certain > size, and a crypto engine for larger requests. You might also want > to take NUMA into account when choosing which backend crypto engine > to dispatch a request to.) To that end, I think we want to have the > ability for a single OCF session to support multiple backend > sessions. > > One use case is that if I as a driver can't handle a request I'd like > to be able to fail it with a special error code and have the crypto > later fall back to software for me (and to use accelerated software if > possible). Right now ccr(4) duplicates the "dumb" software for GCM > requests it can't handle explicitly. > > Another use case might be failover if a hardware engine experiences > a hardware failure. In theory it should be possible to fail over > to a different driver at that point including resubmitting pending > requests that weren't completed, and it should be possible (I think) > to manage this in the crypto framework rather than in consumers like > IPSec and GELI. > > Load distribution among backends might be another case to consider > (e.g. GELI or ZFS encryption once that lands) if you have long- > running sessions that spawn lots of self-contained requests. > > Note that if we want to spawn additional backend sessions on the fly > (e.g. only create a software fallback session on demand if a driver > fails a request with the "use software" magic error code), we will > have to keep per-session state such as keys around. We probably > already do that now, but this would definitely require doing that. > > One concern with some of these changes is that there are several drivers > in the tree for older hardware that I'm not sure is really used anymore. > That is an impediment to making changes to the crypto <-> driver interface > if we can't find folks willing to at least test changes to those drivers > if not maintain them. That does seem like a relevant concern, as some of this stuff seems pretty obscure now. I expect that some of it will have to go since no one can be found to test it. > This is all I could think of today. What do other folks think? This generally seems unobjectionable and nice to have. -Ben From owner-freebsd-arch@freebsd.org Thu Jan 11 07:46:48 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76917E750BA for ; Thu, 11 Jan 2018 07:46:48 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 61E7B83C28 for ; Thu, 11 Jan 2018 07:46:48 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: by mailman.ysv.freebsd.org (Postfix) id 61438E750B9; Thu, 11 Jan 2018 07:46:48 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60F50E750B8 for ; Thu, 11 Jan 2018 07:46:48 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 271F983C26; Thu, 11 Jan 2018 07:46:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id 5C23927396; Thu, 11 Jan 2018 07:46:40 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w0B7kOoL051885 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 11 Jan 2018 07:46:24 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w0B7kOM1051884; Thu, 11 Jan 2018 07:46:24 GMT (envelope-from phk) To: John Baldwin cc: arch@freebsd.org Subject: Re: Ranting about OCF / crypto(9) In-reply-to: <3790717.UIxaijsHl3@ralph.baldwin.cx> From: "Poul-Henning Kamp" References: <3790717.UIxaijsHl3@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <51882.1515656784.1@critter.freebsd.dk> Date: Thu, 11 Jan 2018 07:46:24 +0000 Message-ID: <51883.1515656784@critter.freebsd.dk> X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 07:46:48 -0000 -------- In message <3790717.UIxaijsHl3@ralph.baldwin.cx>, John Baldwin writes: >- OCF is over flexible and overly broad. I would actually argue that it is neithe, quite the contrary. With the kernel-userland transition becoming more expensive, what we need is a DSL where you can put entire processing steps into the kernel, sort of like BPF but more general. Ideally, you should be able to push something like this into the kernel and have it executed in a single syscall: h = hash:sha256() b = file_buffer() f = open("/tmp/foo", "r") while f.read(b): h.input(b) return h.hex() BPF is the existence proof that stuff like this is both feasible and profitable, now we just need to take it to the next level. If we had a language like this, accept-filters whouldn't be necessary. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@freebsd.org Thu Jan 11 07:54:28 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0085DE7579D for ; Thu, 11 Jan 2018 07:54:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D0F8B1E3 for ; Thu, 11 Jan 2018 07:54:27 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id CD514E7579C; Thu, 11 Jan 2018 07:54:27 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCFB4E7579B for ; Thu, 11 Jan 2018 07:54:27 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-io0-f180.google.com (mail-io0-f180.google.com [209.85.223.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D6151E1 for ; Thu, 11 Jan 2018 07:54:26 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-io0-f180.google.com with SMTP id w188so2304992iod.10 for ; Wed, 10 Jan 2018 23:54:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=CF6OZ5J/k7pYhpx792TcOz3uPAYJxYP7Pbzfr85zifo=; b=NFn3+DYtiOyQ8hqEY4IGiVWWBA8ct8hj9mka79cCJ4ZEVf1XmBBY883n0jC8oTk6VM NTX16HGu3XhcG6EJjaDJozLQVaK71ltOSIKNLOPaiMxaEXE6WXBhHZzdtJCmkIHC2z1Z fGFrTai6eufGdVg5rxq/2MoYG4zkvphIA78hxID5fUlcRarB+iyq4+XxLJTLGE+nCoAQ OTySWFFUTara3tBKI7eiytzYaag4HARrHTHihcHvkk3P5v1EiBOa9yK5QMXv8gtqroYC izuwkm+XICfjZcm6epmljM0okJK4C0Iy3QlchuFgbHKLlAcJs8/17eiwBZnJGy2uVYW3 CH8A== X-Gm-Message-State: AKwxytd3LLBusTge/qKpHtTmWsZyv7slw9IA+CmtJFX5OMNDGnwnRMIX 47/GbDPivcw+L0rQlXfVVCPl8cxO X-Google-Smtp-Source: ACJfBovbS72zqyrsLg9sakHud44SH/2JeTd87Zbui0ZqQDy4Q9NNdEeWmFGhjDRidVr6s8+kJJcRmA== X-Received: by 10.107.131.200 with SMTP id n69mr20888411ioi.76.1515657260415; Wed, 10 Jan 2018 23:54:20 -0800 (PST) Received: from mail-io0-f182.google.com (mail-io0-f182.google.com. [209.85.223.182]) by smtp.gmail.com with ESMTPSA id o138sm11356777ioo.21.2018.01.10.23.54.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 Jan 2018 23:54:20 -0800 (PST) Received: by mail-io0-f182.google.com with SMTP id t63so2330310iod.0 for ; Wed, 10 Jan 2018 23:54:20 -0800 (PST) X-Received: by 10.107.143.8 with SMTP id r8mr20290643iod.215.1515657259966; Wed, 10 Jan 2018 23:54:19 -0800 (PST) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.74.80 with HTTP; Wed, 10 Jan 2018 23:54:19 -0800 (PST) In-Reply-To: <51883.1515656784@critter.freebsd.dk> References: <3790717.UIxaijsHl3@ralph.baldwin.cx> <51883.1515656784@critter.freebsd.dk> From: Conrad Meyer Date: Wed, 10 Jan 2018 23:54:19 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Ranting about OCF / crypto(9) To: Poul-Henning Kamp Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 07:54:28 -0000 On Wed, Jan 10, 2018 at 11:46 PM, Poul-Henning Kamp wrote: > -------- > In message <3790717.UIxaijsHl3@ralph.baldwin.cx>, John Baldwin writes: > >>- OCF is over flexible and overly broad. > > I would actually argue that it is neithe, quite the contrary. > > With the kernel-userland transition becoming more expensive, what > we need is a DSL where you can put entire processing steps into the > kernel, sort of like BPF but more general. > > Ideally, you should be able to push something like this into > the kernel and have it executed in a single syscall: > > h = hash:sha256() > b = file_buffer() > f = open("/tmp/foo", "r") > while f.read(b): > h.input(b) > return h.hex() > > BPF is the existence proof that stuff like this is both > feasible and profitable, now we just need to take it to > the next level. > > If we had a language like this, accept-filters whouldn't be > necessary. Sure, that's a great idea (well, aside from introducing a large attack surface that the Linux folks have repeatedly discovered with eBPF). But, embedding lua or something like lua in the kernel is completely tangential to the problem of providing a good generic interface for crypto hardware. Please don't hijack this thread with that discussion. Conrad From owner-freebsd-arch@freebsd.org Thu Jan 11 08:24:30 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4A46E7714E for ; Thu, 11 Jan 2018 08:24:30 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id AE8AC15AD for ; Thu, 11 Jan 2018 08:24:30 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: by mailman.ysv.freebsd.org (Postfix) id ACBF5E7714D; Thu, 11 Jan 2018 08:24:30 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABC09E7714C for ; Thu, 11 Jan 2018 08:24:30 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 6DC3E15AC; Thu, 11 Jan 2018 08:24:30 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id E17522739E; Thu, 11 Jan 2018 08:24:27 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w0B8OC7v052055 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 11 Jan 2018 08:24:12 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w0B8OBfA052054; Thu, 11 Jan 2018 08:24:11 GMT (envelope-from phk) To: cem@freebsd.org cc: "freebsd-arch@freebsd.org" Subject: Re: Ranting about OCF / crypto(9) In-reply-to: From: "Poul-Henning Kamp" References: <3790717.UIxaijsHl3@ralph.baldwin.cx> <51883.1515656784@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <52052.1515659051.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Thu, 11 Jan 2018 08:24:11 +0000 Message-ID: <52053.1515659051@critter.freebsd.dk> X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 08:24:30 -0000 -------- In message , Conrad Meyer writes: >But, embedding lua or something like lua in the kernel is completely >tangential to the problem of providing a good generic interface for >crypto hardware. Please don't hijack this thread with that >discussion. The problem is not the interface to the crypto hardware, but getting the data to and from the crypto hardware in the first place. You can either drown yourself in special cases (IPSEC, HTTPS, ...) with a "good generic interface for crypto hardware", or you can solve the actual problem, with a A Good Generic Interface For Data Streams. But don't let me distract you with my experience here, I only spent years on it... -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-arch@freebsd.org Thu Jan 11 13:08:14 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 145AEE6019B for ; Thu, 11 Jan 2018 13:08:14 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E8F216F2B1 for ; Thu, 11 Jan 2018 13:08:13 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: by mailman.ysv.freebsd.org (Postfix) id E84A7E60182; Thu, 11 Jan 2018 13:08:13 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7E11E60181 for ; Thu, 11 Jan 2018 13:08:13 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AB9DC6F2AF; Thu, 11 Jan 2018 13:08:13 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 411C325D3A7C; Thu, 11 Jan 2018 13:08:02 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 8BE03D1F8D6; Thu, 11 Jan 2018 13:08:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id c1YTGYMW1gdG; Thu, 11 Jan 2018 13:08:00 +0000 (UTC) Received: from [10.248.105.126] (fresh-ayiya.sbone.de [IPv6:fde9:577b:c1a9:f001::2]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 04448D1F8D5; Thu, 11 Jan 2018 13:07:59 +0000 (UTC) From: "Bjoern A. Zeeb" To: "Benjamin Kaduk" Cc: "John Baldwin" , arch@freebsd.org Subject: Re: Ranting about OCF / crypto(9) Date: Thu, 11 Jan 2018 13:07:58 +0000 X-Mailer: MailMate (2.0BETAr6102) Message-ID: <8C6BFBB0-3323-4DC8-BF23-B27D0235256D@lists.zabbadoz.net> In-Reply-To: <20180111055620.GO72574@kduck.kaduk.org> References: <3790717.UIxaijsHl3@ralph.baldwin.cx> <20180111055620.GO72574@kduck.kaduk.org> MIME-Version: 1.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 13:08:14 -0000 On 11 Jan 2018, at 5:56, Benjamin Kaduk wrote: >> In terms of algorithms, I suspect there are some older algorithms >> we could drop. Modern hardware doesn't offload DES for example. >> Both ccr(4) and aesni(4) only support AES for encryption. We >> do need to keep algorithms required for IPSec in the kernel, but >> we could probably drop some others? > > Yes, it's probably time for DES to go. Maybe others as well. There sadly still is a lot of commercial gear out there that still requires single-DES. >> One concern with some of these changes is that there are several drivers >> in the tree for older hardware that I'm not sure is really used anymore. >> That is an impediment to making changes to the crypto <-> driver interface >> if we can't find folks willing to at least test changes to those drivers >> if not maintain them. > > That does seem like a relevant concern, as some of this stuff seems > pretty obscure now. I expect that some of it will have to go since > no one can be found to test it. I am sure I have old soekris boxes in use with a hifn(4) in them. /bz From owner-freebsd-arch@freebsd.org Thu Jan 11 17:57:34 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 406D2E722D3 for ; Thu, 11 Jan 2018 17:57:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 2BCDA7CB23 for ; Thu, 11 Jan 2018 17:57:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 284E5E722D2; Thu, 11 Jan 2018 17:57:34 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27FD6E722D1 for ; Thu, 11 Jan 2018 17:57:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 07FF67CB20 for ; Thu, 11 Jan 2018 17:57:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (astound-66-234-199-215.ca.astound.net [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 884B110A8BC; Thu, 11 Jan 2018 12:57:26 -0500 (EST) From: John Baldwin To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: Ranting about OCF / crypto(9) Date: Thu, 11 Jan 2018 09:44:47 -0800 Message-ID: <3684730.T7Zgydtq6O@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <51883.1515656784@critter.freebsd.dk> References: <3790717.UIxaijsHl3@ralph.baldwin.cx> <51883.1515656784@critter.freebsd.dk> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 11 Jan 2018 12:57:26 -0500 (EST) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 17:57:34 -0000 On Thursday, January 11, 2018 07:46:24 AM Poul-Henning Kamp wrote: > -------- > In message <3790717.UIxaijsHl3@ralph.baldwin.cx>, John Baldwin writes: > > >- OCF is over flexible and overly broad. > > I would actually argue that it is neithe, quite the contrary. >From a device driver's perspective it is overly broad. The linked-list of descriptors in theory allows arbitrary data arrangments, but all of the recent crypto engines I'm familiar with basically cater to the layout of IPSec and TLS. They assume exactly one region of cipher text with an optional AAD region that is before (not after), etc. They don't support arbitrary combinations of alorithms, and they make certain assumptions about how combined auth+enc actually works. > With the kernel-userland transition becoming more expensive, what > we need is a DSL where you can put entire processing steps into the > kernel, sort of like BPF but more general. > > Ideally, you should be able to push something like this into > the kernel and have it executed in a single syscall: > > h = hash:sha256() > b = file_buffer() > f = open("/tmp/foo", "r") > while f.read(b): > h.input(b) > return h.hex() > > BPF is the existence proof that stuff like this is both > feasible and profitable, now we just need to take it to > the next level. > > If we had a language like this, accept-filters whouldn't be > necessary. While I think this is not a bad idea, I don't think it has any bearing on the crypto <-> driver interface which is where most of my beef lies, but rather a different method to allow construction of in-kernel requests to the crypto layer. -- John Baldwin From owner-freebsd-arch@freebsd.org Thu Jan 11 17:57:36 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F240E722D9 for ; Thu, 11 Jan 2018 17:57:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EEA867CB25 for ; Thu, 11 Jan 2018 17:57:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id EDF6AE722D8; Thu, 11 Jan 2018 17:57:35 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDA08E722D6 for ; Thu, 11 Jan 2018 17:57:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF3807CB24; Thu, 11 Jan 2018 17:57:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (astound-66-234-199-215.ca.astound.net [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id EC96710A8BE; Thu, 11 Jan 2018 12:57:27 -0500 (EST) From: John Baldwin To: Benjamin Kaduk Cc: arch@freebsd.org Subject: Re: Ranting about OCF / crypto(9) Date: Thu, 11 Jan 2018 09:41:15 -0800 Message-ID: <1848677.SMV3i9kbhA@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <20180111055620.GO72574@kduck.kaduk.org> References: <3790717.UIxaijsHl3@ralph.baldwin.cx> <20180111055620.GO72574@kduck.kaduk.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 11 Jan 2018 12:57:28 -0500 (EST) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 17:57:36 -0000 On Wednesday, January 10, 2018 11:56:21 PM Benjamin Kaduk wrote: > Replying mostly with my upstream OpenSSL hat on, not least to note > my regret that OpenSSL's historic poor API design choices reflect so > heavily herein... > > On Wed, Jan 10, 2018 at 04:18:52PM -0800, John Baldwin wrote: > > While working on hooking the ccr(4) driver into our in-kernel crypto > > framework (along with some out-of-tree patches to extend OpenSSL's > > /dev/crypto engine to support AES-CTR/XTS/GCM and some further changes to > > Note that on master/proto-1.1.1, the cryptodev engine got swapped > out for the one from https://github.com/openssl/openssl/pull/3744, > since the OpenBSD folks would not agree to relicense. (Apparently > the move to the Apache license is probably actually going to > happen.) But I'd be happy to help review patches to add additional > functionality. Hmm, I tried mailing the person who worked on that new engine a while ago to see if they were open to accepting upstream patches and never got a reply. If you are able to work with upstream then I will talk to you more offline about CTR/XTS/GCM support for 1.1.1. > > - TLS-style requests (using TLS's different methods of > > combining auth and encryption methods when those are > > separate) > > My brain is trying to ask "is that really a good idea?" about > putting the old (bad/broken) TLS mac+encrypt schemes in the kernel, > from the vantage point of making it easier to do insecure things. It's more about being able to offload TLS encryption. Hardware engines support combined auth+enc operations, so we need to have requests at that granularity. When I last looked, OpenSSL 1.0.x at least didn't really support CBC+SHA as a real NID (there's a special NID only used by the userland aesni bits, but no engine implements it). Not all of the TLS world is GCM yet, so offloading non-GCM TLS is still desirable. My understanding is that TLS now supports IPSec-style EtM via some RFC I can't recall, but trying to read the code in OpenSSL 1.0.x at least I couldn't find anything that seemed to support that. > > - Simple compression / decompression requests. While this isn't > > "crypto", per se, I do think it is probably still simpler to > > manage this via OCF than a completely separate interface. > > Probably, though perhaps less so after the removal of arbitrary > stacking depths. And mixing compression with encryption has its own > risks, of course. I probably think you wouldn't mix but would either do compression, auth, hash, or auth+enc. NetBSD's /dev/crypto does support stacking compression + auth + enc in a single ioctl, but it doesn't provide any way to control the ordering so in practice I think it was just a way to permit offloading compression alone. > > In terms of algorithms, I suspect there are some older algorithms > > we could drop. Modern hardware doesn't offload DES for example. > > Both ccr(4) and aesni(4) only support AES for encryption. We > > do need to keep algorithms required for IPSec in the kernel, but > > we could probably drop some others? > > Yes, it's probably time for DES to go. Maybe others as well. > > > - To better support OpenSSL's engine, the /dev/crypto hash interface > > should not require monotonic buffers, but support requests for > > large buffers that span multiple requests (so you can do something > > akin to the 'Init' / 'Update' (N times) / 'Final' model existing > > software hashing APIs use). In particular, the bigger win for > > hashing in hardware is when you can offload the hashing of a large > > thing rather than small requests. > > Something of an aside, but we are growing some support for > "one-shot" stuff as a result of Ed25519 support landing, but the > Init/Update/Final mindset is still pretty baked in, for now. Also, the new /dev/crypto engine for 1.1.x supports a Linux API for hashes that assumes Init/Update/Final. My thinking is to implement the Linux API (really just a new flag IIRC) for this so that hopefully the 1.1.x engine doesn't need any changes to support this. It would be nice if OpenSSL would use the HMAC nids for HMAC requests instead of the plain hashes perhaps? (That seems relevant to the 977 issue you quoted and right now FreeBSD's /dev/crypto is an existing implementation that would benefit from HMAC using HMAC NIDs instead of digest NIDs.) I don't think it would require a change in the engine interface, btw. I think it means that HMAC() needs to first try to find an engine that supports the HMAC NID, and if that fails it could then use the current code which generates PADs in software and looks for an engine that implements the digest NID. -- John Baldwin From owner-freebsd-arch@freebsd.org Fri Jan 12 17:55:03 2018 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3297E72FF8 for ; Fri, 12 Jan 2018 17:55:02 +0000 (UTC) (envelope-from michelle.stern@corporatedigiworld.com) Received: from IND01-MA1-obe.outbound.protection.outlook.com (mail-ma1ind01on0061.outbound.protection.outlook.com [104.47.100.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BA8A776DC for ; Fri, 12 Jan 2018 17:55:01 +0000 (UTC) (envelope-from michelle.stern@corporatedigiworld.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT2800955.onmicrosoft.com; s=selector1-corporatedigiworld-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Ls6VOzRZPfySocFLBmvvpPJQrn+0WJJ+13hG5dBGoxo=; b=COx7MlAl4BqHVfTIlx16RmyXbKlDShFlB31hwlRnIKKs8aN42iZDkh7E1GmG2S6d5sFNdADARNEkJHftYE2afac6NFf6OKDtcLELA73kKiUxvH2Ts+57mBy5oSXvR35SZOl/DZbTnWN4mC3xhhbAlTatW3HoxiSb0FbTWQZRDCU= Received: from MA1PR0101MB1927.INDPRD01.PROD.OUTLOOK.COM (52.134.143.138) by MA1PR0101MB1926.INDPRD01.PROD.OUTLOOK.COM (52.134.143.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Fri, 12 Jan 2018 17:54:58 +0000 Received: from MA1PR0101MB1927.INDPRD01.PROD.OUTLOOK.COM ([fe80::f0a7:bcf2:ccba:2696]) by MA1PR0101MB1927.INDPRD01.PROD.OUTLOOK.COM ([fe80::f0a7:bcf2:ccba:2696%18]) with mapi id 15.20.0386.009; Fri, 12 Jan 2018 17:54:58 +0000 From: Michelle Stern To: "freebsd-arch@freebsd.org" Subject: Cisco IOS Users List Thread-Topic: Cisco IOS Users List Thread-Index: AdOLzEqxpvh9VzjQTpKGURi4jEAndA== Date: Fri, 12 Jan 2018 17:39:30 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=michelle.stern@corporatedigiworld.com; x-originating-ip: [183.82.22.5] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; MA1PR0101MB1926; 7:35JXieCZcOKJRRH+UDiMhZAwlGbhV7u5b9nDkuhramQOS9lD8kXdknTBrQHU5cjVQvID2c92jpQefj/b9EoC9puyRTGxlJ5zNMofSMcFe3SgxZIx7hVpcFMpyUT48D74ft4cztDvqEbpo0hSkwP5y8td/F47kooSwmscymuTcL4IHEphCHE+5WzqN3PZymcVf4X0m3ytBzdMKdhF4dQSlEyPxzFAKxYsNmnECu6ewZ00cadhzeZ3vwNqUErIKRPl x-ms-office365-filtering-correlation-id: 22e2b09f-c7cd-4a5c-ff76-08d559e598de x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020085)(4652020)(7021115)(5600026)(4604075)(3008032)(2017052603307)(7153060)(7193020); SRVR:MA1PR0101MB1926; x-ms-traffictypediagnostic: MA1PR0101MB1926: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(21748063052155); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(3231023)(944501146)(10201501046)(6041268)(20161123558120)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(2016111802025)(6043046)(6072148)(201708071742011); SRVR:MA1PR0101MB1926; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:MA1PR0101MB1926; x-forefront-prvs: 0550778858 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(346002)(376002)(39380400002)(396003)(39860400002)(199004)(189003)(790700001)(6116002)(2906002)(3846002)(3480700004)(7696005)(86362001)(236005)(54896002)(9686003)(14454004)(7520500002)(3280700002)(6506007)(6666003)(99286004)(478600001)(97736004)(66066001)(5660300001)(68736007)(53936002)(33656002)(5250100002)(316002)(4743002)(3660700001)(606006)(2351001)(6436002)(2900100001)(2501003)(6916009)(102836004)(106356001)(5630700001)(8936002)(105586002)(74316002)(9326002)(7736002)(81156014)(8676002)(25786009)(6306002)(81166006)(55016002)(5640700003)(19870200002); DIR:OUT; SFP:1101; SCL:1; SRVR:MA1PR0101MB1926; H:MA1PR0101MB1927.INDPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: corporatedigiworld.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: t7PrmbuEq2EMXEfm9/YJFQHGL2x7+MJ0IchhqxftTCpOa3GKh7SIDWZzICOdYr0y4KJfXtkFQWHzhNH8p9Z6Xg== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: corporatedigiworld.com X-MS-Exchange-CrossTenant-Network-Message-Id: 22e2b09f-c7cd-4a5c-ff76-08d559e598de X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jan 2018 17:39:30.7205 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: c7204af0-2c8f-42ca-b117-921e6e07b230 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MA1PR0101MB1926 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 17:55:03 -0000 Hi, I would like to know if you are interested in acquiring Cisco IOS Users List for your marketing campaigns. These are the information fields that we provide for each contacts: Names, = Title, Email, Phone, Company Name, Company URL, and Company physical addres= s, SIC Code, Industry and Company Size (Revenue and Employee). Let me know if you are interested and I will get back to you with the count= s and pricing. Regards, Michelle Stern Marketing Executive to opt out, please reply with Leave Out in the Subject Line.