Date: Sat, 31 Aug 2024 09:56:07 -0600 From: Warner Losh <imp@bsdimp.com> To: John Baldwin <jhb@freebsd.org> Cc: Andrew Turner <andrew@fubar.geek.nz>, Jessica Clarke <jrtc27@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org>, "dev-commits-src-all@freebsd.org" <dev-commits-src-all@freebsd.org>, "dev-commits-src-main@freebsd.org" <dev-commits-src-main@freebsd.org> Subject: Re: git: 43e8849bc294 - main - conf: Enable BTI checking in the arm64 kernel Message-ID: <CANCZdfqJFgGsw93gv6UFG-tG3ZT=T6vWL_344Kcz2J1b_VY6cw@mail.gmail.com> In-Reply-To: <8a9faa7c-1f81-4aa8-adac-a6f07d7d73ff@FreeBSD.org> References: <202408200902.47K92BsJ078096@gitrepo.freebsd.org> <77cbcd47-1778-4e71-b824-4e75e0e4b2d6@FreeBSD.org> <19DBDC11-EB0F-48C0-9AAE-9EED087F0AB6@freebsd.org> <F794FC8A-6997-4D79-A5BD-72ACC937EFEC@fubar.geek.nz> <8a9faa7c-1f81-4aa8-adac-a6f07d7d73ff@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000099ca480620fcbf30 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 30, 2024 at 9:35=E2=80=AFAM John Baldwin <jhb@freebsd.org> wrot= e: > On 8/30/24 04:55, Andrew Turner wrote: > > > > > >> On 29 Aug 2024, at 17:02, Jessica Clarke <jrtc27@freebsd.org> wrote: > >> > >> On 21 Aug 2024, at 15:28, John Baldwin <jhb@FreeBSD.org> wrote: > >>> > >>> On 8/20/24 05:02, Andrew Turner wrote: > >>>> The branch main has been updated by andrew: > >>>> URL: > https://cgit.FreeBSD.org/src/commit/?id=3D43e8849bc29414036ccaef7788de95a= 07ad32ab5 > >>>> commit 43e8849bc29414036ccaef7788de95a07ad32ab5 > >>>> Author: Andrew Turner <andrew@FreeBSD.org> > >>>> AuthorDate: 2024-08-19 12:59:49 +0000 > >>>> Commit: Andrew Turner <andrew@FreeBSD.org> > >>>> CommitDate: 2024-08-20 08:49:15 +0000 > >>>> conf: Enable BTI checking in the arm64 kernel > >>>> To ensure new code has BTI support make it an error to not > have the > >>>> BTI ELF note when linking the kernel and kernel modules. > >>>> Reviewed by: kib, emaste > >>>> Sponsored by: Arm Ltd > >>>> Differential Revision: https://reviews.freebsd.org/D45469 > >>> > >>> This has broken two of the GitHub CI actions using clang 12 and clang > 13. > >>> Please fix this to be conditional on a supported linker version (or > perhaps > >>> add a new linker feature to bsd.linker.mk). > >> > >> This is still broken. If it=E2=80=99s not fixed promptly I will just r= evert > >> this change; we can=E2=80=99t leave CI broken, especially when it gets= used to > >> test GitHub PRs. Please stop breaking building with older toolchains, > >> this isn=E2=80=99t the first time it=E2=80=99s happened. > > > > See https://github.com/freebsd/freebsd-src/pull/1393 and > https://github.com/freebsd/freebsd-src/pull/1399 > > I do think we probably want to flesh out a bit more what kind of policy > we want for the range of compiler versions to support. E.g. in the GDB > project the policy is something like "will not use a version of C++ newer > than a compiler from 10 years ago" (IIRC). There they will also look bac= k > to whatever LTS distros are active and what is the newest compiler that > can be installed on those LTS via the equivalent of ports. > QEMU does similar things. > Traditionally we used to only support compiling FreeBSD N on FreeBSD N-1 > (though we have often supported older versions of FreeBSD). However, we > now also support cross-compiling from Linux and macOS, so we probably wan= t > to widen the support base a bit. I would say a first stab perhaps is tha= t > main and any supported stable and release branches should be buildable on > a host running any of those versions. That is, FreeBSD 13 is still > supported so we should keep main building on it directly without requirin= g > a jump to FreeBSD 14 first. Once 13 EOLs then we can stop supporting 13, > and that would gives folks on 13 a way to step up to the supported versio= n > of 14 at the time of the EOL. That said, presuambly fairly recent versio= ns > of LLVM (and GCC for that matter) will be available in ports for supporte= d > versions, so that doesn't necessarily require a very wide range of > supported > compiler versions. Supporting the "native" compiler on those releases > would be a nice goal that I think is probably achievable with minimal > effort. > Recent history has been such that all supported branches can jump to head. It's been more like N-2 or N-3 (12 -> head still works last I tried it, but it's likely to break at any random llvm import and we've had once since I (accidentally) tried it). In general, it's good policy if we can do it. In the earlier days of the project, there was considerably more API churn that required more compatibility shim= s than we need these days. It's usually just a matter of having a new enough C++ compiler for the latest language features used by llvm. I think the last breakage I looked into was like that (11 didn't have a new enough llvm for 14-current). As you note, we have ports/pkgs that can do that, but only if we keep the older recently not supported branches final packages around... Also to clarify one point: It's only the last supported release on the stable branch or newer that can jump to -current, not any point since the branch (though it often works, we dropped that effort some time ago when its main champion moved on from contributing to the FreeBSD project). I think stable-11 got a 'final' llvm update late in the branche's life because we hadn't had a release in too long, but I may be misremembering. But having said all that, I agree: this isn't going to widen things all that much. Today our oldest supported release is 14.0 which shipped with clang 16, and > 13.3 shipped with clang 17. Both of those are quite a bit newer than > clang 12 > that is current oldest version in CI. I have no idea what version range > might be sensible for Linux and macOS. Presumably we'd want to settle on > some range of ubuntu LTS versions and whatever is the newest version you > can > install on the oldest LTS as the minimum. > > clang 12 might have been used in 13.1 (not mentioned in the release notes= , > 13.0 used clang 11, 13.2 used clang 14) and 12.3 (12.4 used clang 13, > 12.2 used clang 10, 12.3 didn't mention it). > That gets back to the question of 'tip of the branch' or 'from the branch point'. I think that we should require only tip of branch (so we don't gate using new compiler features in head for something we shipped in 13.0), but encourage people to put the API compat shims into the build library for a wider range (since they will likely also be needed for Linux/MacOS too, independent of compiler version). > I know CHERI LLVM is still on LLVM 15 (though moving to LLVM 16 soon, but > Morello LLVM is 14 still I think). libc++ is also making this more > complicated as they are providing very little compatability at all. > They've > already ripped out support for GCC 14 I believe, so you can't build curre= nt > libc++ with a released version of GCC or something crazy. > Upstream will, perhaps sadly, drive a lot of our policy in this area. When upstream de-orbits support for something, it's hard for us to keep maintaining it and keep updating to upstream without some kind of compelling need. Though I recently spend a fair amount of time coming up with a fairly minimal tweak to our cdefs.h that makes it more compatible with the crazy things that libc++ does to still support compiling older C++ standards. Which brings up another possible issue. We don't support building FreeBSD, itself, with anything but a fairly up-to-date wrt standards compiler. However, we support building binaries from sources that compile with C89 and newer, and C++98 and newer (though really this is likely C++11 for most things, given the upstream de-orbiting of support for older standard binaries). We may need to carefully call this out as well... The older C++ standards don't matter too much (most of the C++ plorts actually want the newer C++), but there's still a lot of packages that compile with a strict C89 feature set that break in subtle ways when we break support for that... Warner --00000000000099ca480620fcbf30 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, Aug 30, 2024 at 9:35=E2=80=AF= AM John Baldwin <<a href=3D"mailto:jhb@freebsd.org">jhb@freebsd.org</a>&= gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0= px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 8/= 30/24 04:55, Andrew Turner wrote:<br> > <br> > <br> >> On 29 Aug 2024, at 17:02, Jessica Clarke <<a href=3D"mailto:jrt= c27@freebsd.org" target=3D"_blank">jrtc27@freebsd.org</a>> wrote:<br> >><br> >> On 21 Aug 2024, at 15:28, John Baldwin <jhb@FreeBSD.org> wro= te:<br> >>><br> >>> On 8/20/24 05:02, Andrew Turner wrote:<br> >>>> The branch main has been updated by andrew:<br> >>>> URL: <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D= 43e8849bc29414036ccaef7788de95a07ad32ab5" rel=3D"noreferrer" target=3D"_bla= nk">https://cgit.FreeBSD.org/src/commit/?id=3D43e8849bc29414036ccaef7788de9= 5a07ad32ab5</a><br> >>>> commit 43e8849bc29414036ccaef7788de95a07ad32ab5<br> >>>> Author:=C2=A0 =C2=A0 =C2=A0Andrew Turner <andrew@FreeBS= D.org><br> >>>> AuthorDate: 2024-08-19 12:59:49 +0000<br> >>>> Commit:=C2=A0 =C2=A0 =C2=A0Andrew Turner <andrew@FreeBS= D.org><br> >>>> CommitDate: 2024-08-20 08:49:15 +0000<br> >>>>=C2=A0 =C2=A0 =C2=A0conf: Enable BTI checking in the arm64 = kernel<br> >>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 To ensure new code has B= TI support make it an error to not have the<br> >>>>=C2=A0 =C2=A0 =C2=A0BTI ELF note when linking the kernel an= d kernel modules.<br> >>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Reviewed by:=C2=A0 =C2= =A0 kib, emaste<br> >>>>=C2=A0 =C2=A0 =C2=A0Sponsored by:=C2=A0 =C2=A0Arm Ltd<br> >>>>=C2=A0 =C2=A0 =C2=A0Differential Revision:=C2=A0 <a href=3D= "https://reviews.freebsd.org/D45469" rel=3D"noreferrer" target=3D"_blank">h= ttps://reviews.freebsd.org/D45469</a><br> >>><br> >>> This has broken two of the GitHub CI actions using clang 12 an= d clang 13.<br> >>> Please fix this to be conditional on a supported linker versio= n (or perhaps<br> >>> add a new linker feature to <a href=3D"http://bsd.linker.mk" r= el=3D"noreferrer" target=3D"_blank">bsd.linker.mk</a>).<br> >><br> >> This is still broken. If it=E2=80=99s not fixed promptly I will ju= st revert<br> >> this change; we can=E2=80=99t leave CI broken, especially when it = gets used to<br> >> test GitHub PRs. Please stop breaking building with older toolchai= ns,<br> >> this isn=E2=80=99t the first time it=E2=80=99s happened.<br> > <br> > See <a href=3D"https://github.com/freebsd/freebsd-src/pull/1393" rel= =3D"noreferrer" target=3D"_blank">https://github.com/freebsd/freebsd-src/pu= ll/1393</a> and <a href=3D"https://github.com/freebsd/freebsd-src/pull/1399= " rel=3D"noreferrer" target=3D"_blank">https://github.com/freebsd/freebsd-s= rc/pull/1399</a><br> <br> I do think we probably want to flesh out a bit more what kind of policy<br> we want for the range of compiler versions to support.=C2=A0 E.g. in the GD= B<br> project the policy is something like "will not use a version of C++ ne= wer<br> than a compiler from 10 years ago" (IIRC).=C2=A0 There they will also = look back<br> to whatever LTS distros are active and what is the newest compiler that<br> can be installed on those LTS via the equivalent of ports.<br></blockquote>= <div><br></div><div>QEMU does similar things.</div><div>=C2=A0</div><blockq= uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p= x solid rgb(204,204,204);padding-left:1ex"> Traditionally we used to only support compiling FreeBSD N on FreeBSD N-1<br= > (though we have often supported older versions of FreeBSD).=C2=A0 However, = we<br> now also support cross-compiling from Linux and macOS, so we probably want<= br> to widen the support base a bit.=C2=A0 I would say a first stab perhaps is = that<br> main and any supported stable and release branches should be buildable on<b= r> a host running any of those versions.=C2=A0 That is, FreeBSD 13 is still<br= > supported so we should keep main building on it directly without requiring<= br> a jump to FreeBSD 14 first.=C2=A0 Once 13 EOLs then we can stop supporting = 13,<br> and that would gives folks on 13 a way to step up to the supported version<= br> of 14 at the time of the EOL.=C2=A0 That said, presuambly fairly recent ver= sions<br> of LLVM (and GCC for that matter) will be available in ports for supported<= br> versions, so that doesn't necessarily require a very wide range of supp= orted<br> compiler versions.=C2=A0 Supporting the "native" compiler on thos= e releases<br> would be a nice goal that I think is probably achievable with minimal<br> effort.<br></blockquote><div><br></div><div>Recent history has been such th= at all supported branches can jump to head. It's</div><div>been more li= ke N-2 or N-3 (12 -> head still works last I tried it, but it's like= ly to</div><div>break at any random llvm import and we've had once sinc= e I (accidentally) tried</div><div>it). In general, it's good policy if= we can do it. In the earlier days of the project,</div><div>there was cons= iderably more API churn that required more compatibility shims</div><div>th= an we need these days. It's usually just a matter of having a new enoug= h C++</div><div>compiler for the latest language features used by llvm. I t= hink the last breakage</div><div>I looked into was like that (11 didn't= have a new enough llvm for 14-current). As</div><div>you note, we have por= ts/pkgs that can do that, but only if we keep the older recently</div><div>= not supported branches final packages around...=C2=A0=C2=A0</div><div><br><= /div><div>Also to clarify one point: It's only the last supported relea= se on the stable branch or newer</div><div>that can jump to -current, not a= ny point since the branch (though it often works, we</div><div>dropped that= effort some time ago when its main champion moved on from contributing</di= v><div>to the FreeBSD project). I think stable-11 got a 'final' llv= m update late in the branche's</div><div>life because we hadn't had= a release in too long, but I may be misremembering.</div><div><br></div><d= iv>But having said all that, I agree: this isn't going to widen things = all that much.</div><div><br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex">Today our oldest supported release is 14.0 which shipped with cl= ang 16, and<br> 13.3 shipped with clang 17.=C2=A0 Both of those are quite a bit newer than = clang 12<br> that is current oldest version in CI.=C2=A0 I have no idea what version ran= ge<br> might be sensible for Linux and macOS.=C2=A0 Presumably we'd want to se= ttle on<br> some range of ubuntu LTS versions and whatever is the newest version you ca= n<br> install on the oldest LTS as the minimum.<br> <br> clang 12 might have been used in 13.1 (not mentioned in the release notes,<= br> 13.0 used clang 11, 13.2 used clang 14) and 12.3 (12.4 used clang 13,<br> 12.2 used clang 10, 12.3 didn't mention it).<br></blockquote><div><br><= /div><div>That gets back to the question of 'tip of the branch' or = 'from the branch point'. I think</div><div>that we should require o= nly tip of branch (so we don't gate using new compiler features</div><d= iv>in head for something we shipped in 13.0), but encourage people to put t= he API compat</div><div>shims into the build library for a wider range (sin= ce they will likely also be needed for</div><div>Linux/MacOS too, independe= nt of compiler version).</div><div>=C2=A0</div><blockquote class=3D"gmail_q= uote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,2= 04);padding-left:1ex"> I know CHERI LLVM is still on LLVM 15 (though moving to LLVM 16 soon, but<b= r> Morello LLVM is 14 still I think).=C2=A0 libc++ is also making this more<br= > complicated as they are providing very little compatability at all.=C2=A0 T= hey've<br> already ripped out support for GCC 14 I believe, so you can't build cur= rent<br> libc++ with a released version of GCC or something crazy.<br></blockquote><= div><br></div><div>Upstream will, perhaps sadly, drive a lot of our policy = in this area. When upstream de-orbits</div><div>support for something, it&#= 39;s hard for us to keep maintaining it and keep updating to upstream</div>= <div>without some kind of compelling need. Though I recently spend a fair a= mount of time coming</div><div>up with a fairly minimal tweak to our cdefs.= h that makes it more compatible with the crazy things</div><div>that libc++= does to still support compiling older C++ standards.</div><div><br></div><= div>Which brings up another possible issue. We don't support building F= reeBSD, itself, with anything</div><div>but a fairly up-to-date wrt standar= ds compiler. However, we support building binaries from sources</div><div>t= hat compile with C89 and newer, and C++98 and newer (though really this is = likely C++11 for most</div><div>things, given the upstream de-orbiting of s= upport for older standard binaries). We may need to carefully</div><div>cal= l this out as well... The older C++ standards don't matter too much (mo= st of the C++ plorts actually</div><div>want the newer C++), but there'= s still a lot of packages that compile with a strict C89 feature set</div><= div>that break in subtle ways when we break support for that...</div><div><= br></div><div>Warner=C2=A0</div></div></div> --00000000000099ca480620fcbf30--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqJFgGsw93gv6UFG-tG3ZT=T6vWL_344Kcz2J1b_VY6cw>