Date: Wed, 3 May 2023 20:38:00 -0700 From: Enji Cooper <yaneurabeya@gmail.com> To: John Baldwin <jhb@FreeBSD.org> Cc: Pierre Pronchery <pierre@freebsdfoundation.org>, freebsd-arch@freebsd.org Subject: Re: OpenSSL 3.0 for 14.0-RELEASE: issues with 1.x/3.x symbol clashing, ports linking against base OpenSSL, ports that don't compile/link against OpenSSL 3, etc Message-ID: <0CA43F8D-E320-4537-AD89-5D10D21D31D8@gmail.com> In-Reply-To: <b2ea0517-e2ac-0c71-3d5c-cc32624d9b0f@FreeBSD.org> References: <C6F8DD52-348E-42D8-84DE-B3A399D2606F@gmail.com> <CAALwa8m7P2daUd9%2BS4oBXqexBrczcXnmL6sGJ8fR4gwJDPDbcg@mail.gmail.com> <12f8559c-d696-5344-98d5-1751d04088af@FreeBSD.org> <u2up6s$mio$1@ciao.gmane.io> <b2ea0517-e2ac-0c71-3d5c-cc32624d9b0f@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_137487B1-2513-48EF-B3CA-C852D95A0D13 Content-Type: multipart/alternative; boundary="Apple-Mail=_C598F332-56AE-4CA6-A5BD-E944CA65F837" --Apple-Mail=_C598F332-56AE-4CA6-A5BD-E944CA65F837 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On May 3, 2023, at 4:54 PM, John Baldwin <jhb@FreeBSD.org> wrote: >=20 > On 5/3/23 4:02 PM, Pierre Pronchery wrote: >> Hi everyone, >> On 5/2/23 23:24, John Baldwin wrote: >>> On 5/2/23 2:59 AM, Antoine Brodin wrote: >>>> On Tue, May 2, 2023 at 1:55=E2=80=AFAM Enji Cooper = <yaneurabeya@gmail.com> wrote: >>>>>=20 >>>>> Hello, >>>>> One of the must-haves for 14.0-RELEASE is the introduction of = OpenSSL >>>>> 3.0 into the base system. This is a must because, in short, = OpenSSL >>>>> 1.1 is no longer supported as of 09/26/2023 [1]. >>>>>=20 >>>>> I am proposing OpenSSL be made private along with all dependent >>>>> libraries, for the following reasons: >>>>> 1. More than a handful of core ports, e.g., = security/py-cryptography >>>>> [2] [3], still do not support OpenSSL 3.0. >>>>> i. If other dependent ports (like lang/python38, etc) move to = OpenSSL >>>>> 3, the distributed modules would break on load due to clashing >>>>> symbols if the right mix of modules were dlopen=E2=80=99ed in a = specific >>>>> order (importing ssl, then importing hazmat=E2=80=99s crypto would = fail). >>>>> ii. Such ports should be deprecated/marked broken as I=E2=80=99ve = recommended >>>>> on the 3.0 exp-run PR [4]. >>>>> 2. OpenSSL 1.1 and 3.0 have clashing symbols, which makes linking = in >>>>> both libraries at runtime impossible without resorting to a number = of >>>>> linker tricks hiding the namespaces using symbol prefixing of = public >>>>> symbols, etc. >>>>>=20 >>>>> The libraries which would need to be made private are as follows: >>>>> - kerberos >>>>> - libarchive >>>>> - libbsnmp >>>>> - libfetch [5] >>>>> - libgeli >>>>> - libldns >>>>> - libmp >>>>> - libradius >>>>> - libunbound >>>>=20 >>>> In my opinion this is a huge amount of work a few weeks before the >>>> release. Focusing on updating OpenSSL and those core ports may be >>>> simpler. >>>=20 >>> This is my view. I think making OpenSSL private is a very huge = task, and >>> fraught with peril in ways that haven't been thought about yet (e.g. = PAM) >>> and that we can't hold up OpenSSL 3 while we wait for this. = Instead, I >>> think >>> we need to be moving forward with OpenSSL 3 in base as-is. We will = have to >>> fix ports to work with OpenSSL 3 regardless (though this does make = that >>> pain >>> in ports happen sooner). Moving libraries private can happen = orthogonally >>> with getting base to work with OpensSL 3. >> I have started to look at updating OpenSSL to version 3.0.8 in base, >> using the existing vendor/openssl-3.0 branch. >> My progress can be found at >> https://github.com/khorben/freebsd-src/tree/khorben/openssl-3.0. I >> regularly force-push to keep a consistent and nice commit history, >> before possibly applying for a merge. >> So far the status is: >> - libssl, libcrypto build on amd64, i386, less sure about aarch64, = other >> architectures not tested >> - libfetch builds, uses libmd in addition to OpenSSL >> - libradius builds, same thing >> - libarchive builds >> - libunbound builds, but not unbound >> - libmp builds >> I used libmd to reach a buildable status faster, since the equivalent >> MD5_*() API is now deprecated in OpenSSL 3. If MD5 is still allowed = in >> OpenSSL 3, we can avoid the dependency on libmd again. (anyone got >> sample code for this?) >=20 > You can use the EVP_* API if desired. tools/cryto/cryptocheck.c has = examples > of using the EVP_* APIs for both "plain" hashes and HMAC constructions I'll echo this as well. This is what the library maintainers recommend = for crypto primitive algorithm =E2=80=9Cagility=E2=80=9D. Cheers, -Enji --Apple-Mail=_C598F332-56AE-4CA6-A5BD-E944CA65F837 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On = May 3, 2023, at 4:54 PM, John Baldwin <<a = href=3D"mailto:jhb@FreeBSD.org" class=3D"">jhb@FreeBSD.org</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><meta = charset=3D"UTF-8" class=3D""><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">On 5/3/23 4:02 PM, Pierre Pronchery wrote:</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" = class=3D""><blockquote type=3D"cite" style=3D"font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; orphans: auto; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span>Hi = everyone,<br class=3D"">On 5/2/23 23:24, John Baldwin wrote:<br = class=3D""><blockquote type=3D"cite" class=3D"">On 5/2/23 2:59 AM, = Antoine Brodin wrote:<br class=3D""><blockquote type=3D"cite" = class=3D"">On Tue, May 2, 2023 at 1:55=E2=80=AFAM Enji Cooper <<a = href=3D"mailto:yaneurabeya@gmail.com" = class=3D"">yaneurabeya@gmail.com</a>> wrote:<br class=3D""><blockquote = type=3D"cite" class=3D""><br class=3D"">Hello,<br class=3D"">One of the = must-haves for 14.0-RELEASE is the introduction of OpenSSL<br = class=3D"">3.0 into the base system. This is a must because, in short, = OpenSSL<br class=3D"">1.1 is no longer supported as of 09/26/2023 = [1].<br class=3D""><br class=3D"">I am proposing OpenSSL be made private = along with all dependent<br class=3D"">libraries, for the following = reasons:<br class=3D"">1. More than a handful of core ports, e.g., = security/py-cryptography<br class=3D"">[2] [3], still do not support = OpenSSL 3.0.<br class=3D"">i. If other dependent ports (like = lang/python38, etc) move to OpenSSL<br class=3D"">3, the distributed = modules would break on load due to clashing<br class=3D"">symbols if the = right mix of modules were dlopen=E2=80=99ed in a specific<br = class=3D"">order (importing ssl, then importing hazmat=E2=80=99s crypto = would fail).<br class=3D"">ii. Such ports should be deprecated/marked = broken as I=E2=80=99ve recommended<br class=3D"">on the 3.0 exp-run PR = [4].<br class=3D"">2. OpenSSL 1.1 and 3.0 have clashing symbols, which = makes linking in<br class=3D"">both libraries at runtime impossible = without resorting to a number of<br class=3D"">linker tricks hiding the = namespaces using symbol prefixing of public<br class=3D"">symbols, = etc.<br class=3D""><br class=3D"">The libraries which would need to be = made private are as follows:<br class=3D"">- kerberos<br class=3D"">- = libarchive<br class=3D"">- libbsnmp<br class=3D"">- libfetch [5]<br = class=3D"">- libgeli<br class=3D"">- libldns<br class=3D"">- libmp<br = class=3D"">- libradius<br class=3D"">- libunbound<br = class=3D""></blockquote><br class=3D"">In my opinion this is a huge = amount of work a few weeks before the<br class=3D"">release. = Focusing on updating OpenSSL and those core ports may be<br = class=3D"">simpler.<br class=3D""></blockquote><br class=3D"">This is my = view. I think making OpenSSL private is a very huge task, and<br = class=3D"">fraught with peril in ways that haven't been thought about = yet (e.g. PAM)<br class=3D"">and that we can't hold up OpenSSL 3 while = we wait for this. Instead, I<br class=3D"">think<br class=3D"">we = need to be moving forward with OpenSSL 3 in base as-is. We will = have to<br class=3D"">fix ports to work with OpenSSL 3 regardless = (though this does make that<br class=3D"">pain<br class=3D"">in ports = happen sooner). Moving libraries private can happen = orthogonally<br class=3D"">with getting base to work with OpensSL 3.<br = class=3D""></blockquote>I have started to look at updating OpenSSL to = version 3.0.8 in base,<br class=3D"">using the existing = vendor/openssl-3.0 branch.<br class=3D"">My progress can be found at<br = class=3D""><a = href=3D"https://github.com/khorben/freebsd-src/tree/khorben/openssl-3.0" = class=3D"">https://github.com/khorben/freebsd-src/tree/khorben/openssl-3.0= </a>. I<br class=3D"">regularly force-push to keep a consistent and nice = commit history,<br class=3D"">before possibly applying for a merge.<br = class=3D"">So far the status is:<br class=3D"">- libssl, libcrypto build = on amd64, i386, less sure about aarch64, other<br class=3D"">architectures= not tested<br class=3D"">- libfetch builds, uses libmd in addition to = OpenSSL<br class=3D"">- libradius builds, same thing<br class=3D"">- = libarchive builds<br class=3D"">- libunbound builds, but not unbound<br = class=3D"">- libmp builds<br class=3D"">I used libmd to reach a = buildable status faster, since the equivalent<br class=3D"">MD5_*() API = is now deprecated in OpenSSL 3. If MD5 is still allowed in<br = class=3D"">OpenSSL 3, we can avoid the dependency on libmd again. = (anyone got<br class=3D"">sample code for this?)<br = class=3D""></blockquote><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">You can use the EVP_* API if desired. = tools/cryto/cryptocheck.c has examples</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;" class=3D"">of using the EVP_* APIs for both = "plain" hashes and HMAC constructions</span><br style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""></div></blockquote></div><br = class=3D""><div class=3D"">I'll echo this as well. This is what the = library maintainers recommend for crypto primitive algorithm = =E2=80=9Cagility=E2=80=9D.</div><div class=3D"">Cheers,</div><div = class=3D"">-Enji</div></body></html>= --Apple-Mail=_C598F332-56AE-4CA6-A5BD-E944CA65F837-- --Apple-Mail=_137487B1-2513-48EF-B3CA-C852D95A0D13 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEtvtxN6kOllEF3nmX5JFNMZeDGN4FAmRTKJkACgkQ5JFNMZeD GN71jQ/+NE9kOX+9cWIZVEAiKxdVZ819KQh1xX7f0+05aTzZD+JvyNBnytDJPOkF ORn1x04nQvVcrAS1RQSMfIfsUJuh5p4uPV7UvyLYTR8UWuz9wrLywEPrcQTq8LVb FrYNz8F2Sk887WTB1e+uaxUDcLzWNhAf0Yp3YesHV30TiX2gkQnIrmKP/ANMReTO Lt2LTtpQTNmMfug6eB418goTEIKBDuaJlynTGgeFObO/fuvfXZD4R+/JCYzVEOm7 RjnbRoMTKd9UCElWMHTaVr2BQpa3pWixk/VNJJs2xGDGnawn1RLLOvSehwF+8R2R gWYlpvLliQnG24ew5y2ctnIcb8Z6fqv2OUIhcW1VngpGDmwiPMtbuDr3jVo0r+Eh x8NyUQkONjOnsPpJEc8OEPZM4KnaN3FZ6QMMCeHv7q7WZ64KSNWPPhIRysIyeTfV i/fwZgDvgcMJOpBgTKXJQ5d61WxxHkNpj92RXTz9OmXk57adpy5kYuytoT46XOSe BJHVJrAXiogByeFaMo8OmaLlSFgUd4cBz8oZIsOjcmbMS1CvSNLVTgVxIudZNE7e wLDD4PqxmDbggGn+p4ft28fTFPbzAw6JZ6+UGsu/7YVUawX2GoTZaiwO5TdXBn8T csW8QkYB5fgUB/C82L2Aze2i18WcP6XmkzXSZ5pOFEz3Ume5yUU= =5yrL -----END PGP SIGNATURE----- --Apple-Mail=_137487B1-2513-48EF-B3CA-C852D95A0D13--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0CA43F8D-E320-4537-AD89-5D10D21D31D8>