Date: Thu, 29 Aug 2024 15:16:05 -0600 From: Warner Losh <imp@bsdimp.com> To: Alan Somers <asomers@freebsd.org> Cc: Shawn Webb <shawn.webb@hardenedbsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: A Demo of rust-in-base Message-ID: <CANCZdfoyzjH5gXyd_KXKWnMHh2NUEGeC8bmPUwpiA9pPDShT0Q@mail.gmail.com> In-Reply-To: <CAOtMX2hvjegNVeZhwTwM3s=6H1Z_dzSuhfOroJq3nZQCSfJwWg@mail.gmail.com> References: <CAOtMX2gdt8xYyLR3peYWhov-161-6d7%2B8L6TiHCCyw1NQyspXw@mail.gmail.com> <hraulhvnvyy5riur63iy53ed6m276prpw577qyg4xpm4xkyxmm@zpvdqgpvpsxv> <CAOtMX2hvjegNVeZhwTwM3s=6H1Z_dzSuhfOroJq3nZQCSfJwWg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000bdad180620d8fcb2
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Thu, Aug 29, 2024, 2:55=E2=80=AFPM Alan Somers <asomers@freebsd.org> wro=
te:
> On Thu, Aug 29, 2024 at 2:21=E2=80=AFPM Shawn Webb <shawn.webb@hardenedbs=
d.org>
> wrote:
> >
> > Hey Alan et al,
> >
> > I apologize for the silence on my end. It has been a busier two months
> > than anticipated. In that time, I've been entertaining some thoughts
> > on Rust in base support. ${LIFE} is starting to calm down again, and I
> > do believe I'll be able to start the research in time in September. I
> > will be splitting my free time between this and studying for my OSCP
> > cert.
> >
> > So, to those thoughts, in list form (in no particular order):
> >
> > 1. Use of Rust compiler toolchain support will be for userland
> > components in an opt-in fashion. Meaning, all userland components
> > written in Rust will be optional.
> > 2. It does not make sense to perform a vendor import of the Rust
> > compiler toolchain and standard libraries. All Rust code in the src
> > tree must be built from an external toolchain.
> > 3. I believe the notion of an external toolchain could be abstracted
> > such that we can support any optional userland component written in
> > a language supported by that external toolchain. This would imply
> > that other alternative languages could be supported with minimal
> > work (Zig, TypeScript, Python, Java, etc.)
> > 4. We could provide auto-detection mechanisms for determining which
> > external toolchains are available, their language support, etc. The
> > initial proof-of-concept would likely be limited to Rust to save on
> > time and complexity.
> > 5. As the work matures, and perhaps as a requisite for eventual
> > inclusion, we could land support for more than Rust. This might be
> > a step too far, but hey, it's one of the thoughts I had.
> > 6. So all of this wrapped up means that:
> > A. This is NOT a call to rewrite everything in Rust. This work will
> > only permit NEW, OPTIONAL components to be written.
> > B. Other languages/toolchains/ecosystems could be supported, not
> > just Rust.
> > C. Initial focus is on userland components. Rust in the kernel is
> > out of scope for this initial proof-of-concept.
> > D. I would like to see Rust in the kernel. That would be a good
> > next area of focus once userland support reaches some level of
> > maturity.
> >
> > My first goal will be to get a better understanding of
> > src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
> > study your work, Alan. I really appreciate the time you have taken. I
> > might reach out to you and Warner directly for further questions.
>
> Relying on an external toolchain and allowing for future external
> toolchains other than Rust sounds like a really good plan.
>
It's the only way it could work. Importing rust with its current level of
maturity and lag is logistically impossible or nearly so.rust doesn't have
a long enough support timeline to work well with our stable branches. Llvm
is tricky enough...
Conceivably we could even ditch our Lua import and use the same
> mechanism for that (but please, save Lua discussion for a separate
> flame war ;) ).
That can't possibly work since we build it into the loader with changes
that are unavoidable. There's no flame war, it's just impossibile. The
situation is entirely different.
But the lua import is the easiest of all the vendor imports I do.
Warner
I anticipate that the next big decisions will be
> "what components are so important that they musn't require an external
> toolchain?" and "how much cargo crate bloat is too much?". But we can
> cross those bridges when we come to them. I look forward to seeing
> your work, and please don't hesitate to ask for help.
>
Managing these things in a FreeBSD context will be challenging and i look
forward to the results we get.
Warner
-Alan
>
>
--000000000000bdad180620d8fcb2
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" =
class=3D"gmail_attr">On Thu, Aug 29, 2024, 2:55=E2=80=AFPM Alan Somers <=
<a href=3D"mailto:asomers@freebsd.org">asomers@freebsd.org</a>> wrote:<b=
r></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border=
-left:1px #ccc solid;padding-left:1ex">On Thu, Aug 29, 2024 at 2:21=E2=80=
=AFPM Shawn Webb <<a href=3D"mailto:shawn.webb@hardenedbsd.org" target=
=3D"_blank" rel=3D"noreferrer">shawn.webb@hardenedbsd.org</a>> wrote:<br=
>
><br>
> Hey Alan et al,<br>
><br>
> I apologize for the silence on my end. It has been a busier two months=
<br>
> than anticipated. In that time, I've been entertaining some though=
ts<br>
> on Rust in base support. ${LIFE} is starting to calm down again, and I=
<br>
> do believe I'll be able to start the research in time in September=
. I<br>
> will be splitting my free time between this and studying for my OSCP<b=
r>
> cert.<br>
><br>
> So, to those thoughts, in list form (in no particular order):<br>
><br>
> 1. Use of Rust compiler toolchain support will be for userland<br>
>=C2=A0 =C2=A0 components in an opt-in fashion. Meaning, all userland co=
mponents<br>
>=C2=A0 =C2=A0 written in Rust will be optional.<br>
> 2. It does not make sense to perform a vendor import of the Rust<br>
>=C2=A0 =C2=A0 compiler toolchain and standard libraries. All Rust code =
in the src<br>
>=C2=A0 =C2=A0 tree must be built from an external toolchain.<br>
> 3. I believe the notion of an external toolchain could be abstracted<b=
r>
>=C2=A0 =C2=A0 such that we can support any optional userland component =
written in<br>
>=C2=A0 =C2=A0 a language supported by that external toolchain. This wou=
ld imply<br>
>=C2=A0 =C2=A0 that other alternative languages could be supported with =
minimal<br>
>=C2=A0 =C2=A0 work (Zig, TypeScript, Python, Java, etc.)<br>
> 4. We could provide auto-detection mechanisms for determining which<br=
>
>=C2=A0 =C2=A0 external toolchains are available, their language support=
, etc. The<br>
>=C2=A0 =C2=A0 initial proof-of-concept would likely be limited to Rust =
to save on<br>
>=C2=A0 =C2=A0 time and complexity.<br>
> 5. As the work matures, and perhaps as a requisite for eventual<br>
>=C2=A0 =C2=A0 inclusion, we could land support for more than Rust. This=
might be<br>
>=C2=A0 =C2=A0 a step too far, but hey, it's one of the thoughts I h=
ad.<br>
> 6. So all of this wrapped up means that:<br>
>=C2=A0 =C2=A0 A. This is NOT a call to rewrite everything in Rust. This=
work will<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0only permit NEW, OPTIONAL components to be w=
ritten.<br>
>=C2=A0 =C2=A0 B. Other languages/toolchains/ecosystems could be support=
ed, not<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0just Rust.<br>
>=C2=A0 =C2=A0 C. Initial focus is on userland components. Rust in the k=
ernel is<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0out of scope for this initial proof-of-conce=
pt.<br>
>=C2=A0 =C2=A0 D. I would like to see Rust in the kernel. That would be =
a good<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0next area of focus once userland support rea=
ches some level of<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0maturity.<br>
><br>
> My first goal will be to get a better understanding of<br>
> src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll =
also<br>
> study your work, Alan. I really appreciate the time you have taken. I<=
br>
> might reach out to you and Warner directly for further questions.<br>
<br>
Relying on an external toolchain and allowing for future external<br>
toolchains other than Rust sounds like a really good plan.<br></blockquote>=
</div></div><div dir=3D"auto"><br></div><div dir=3D"auto">It's the only=
way it could work. Importing rust with its current level of maturity and l=
ag is logistically impossible or nearly so.rust doesn't have a long eno=
ugh support timeline to work well with our stable branches. Llvm is tricky =
enough...=C2=A0</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div cla=
ss=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Conceivably we could even ditch our Lua import and use the same<br>
mechanism for that (but please, save Lua discussion for a separate<br>
flame war ;) ).=C2=A0</blockquote></div></div><div dir=3D"auto"><br></div><=
div dir=3D"auto">That can't possibly work since we build it into the lo=
ader with changes that are unavoidable. There's no flame war, it's =
just impossibile. The situation is entirely different.=C2=A0</div><div dir=
=3D"auto"><br></div><div dir=3D"auto">But the lua import is the easiest of =
all the vendor imports I do.</div><div dir=3D"auto"><br></div><div dir=3D"a=
uto">Warner=C2=A0</div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></=
div><div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail=
_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:=
1ex"> I anticipate that the next big decisions will be<br>
"what components are so important that they musn't require an exte=
rnal<br>
toolchain?" and "how much cargo crate bloat is too much?".=
=C2=A0 But we can<br>
cross those bridges when we come to them.=C2=A0 I look forward to seeing<br=
>
your work, and please don't hesitate to ask for help.<br></blockquote><=
/div></div><div dir=3D"auto"><br></div><div dir=3D"auto">Managing these thi=
ngs in a FreeBSD context will be challenging and i look forward to the resu=
lts we get.</div><div dir=3D"auto"><br></div><div dir=3D"auto">Warner=C2=A0=
</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div class=3D"gmail_quo=
te"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef=
t:1px #ccc solid;padding-left:1ex">
-Alan<br>
<br>
</blockquote></div></div></div>
--000000000000bdad180620d8fcb2--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfoyzjH5gXyd_KXKWnMHh2NUEGeC8bmPUwpiA9pPDShT0Q>