Date: Thu, 29 Aug 2024 20:21:20 +0000 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Alan Somers <asomers@freebsd.org> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: A Demo of rust-in-base Message-ID: <hraulhvnvyy5riur63iy53ed6m276prpw577qyg4xpm4xkyxmm@zpvdqgpvpsxv> In-Reply-To: <CAOtMX2gdt8xYyLR3peYWhov-161-6d7%2B8L6TiHCCyw1NQyspXw@mail.gmail.com> References: <CAOtMX2gdt8xYyLR3peYWhov-161-6d7%2B8L6TiHCCyw1NQyspXw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--mad2rcwhuximkfrn
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hey Alan et al,
I apologize for the silence on my end. It has been a busier two months
than anticipated. In that time, I've been entertaining some thoughts
on Rust in base support. ${LIFE} is starting to calm down again, and I
do believe I'll be able to start the research in time in September. I
will be splitting my free time between this and studying for my OSCP
cert.
So, to those thoughts, in list form (in no particular order):
1. Use of Rust compiler toolchain support will be for userland
components in an opt-in fashion. Meaning, all userland components
written in Rust will be optional.
2. It does not make sense to perform a vendor import of the Rust
compiler toolchain and standard libraries. All Rust code in the src
tree must be built from an external toolchain.
3. I believe the notion of an external toolchain could be abstracted
such that we can support any optional userland component written in
a language supported by that external toolchain. This would imply
that other alternative languages could be supported with minimal
work (Zig, TypeScript, Python, Java, etc.)
4. We could provide auto-detection mechanisms for determining which
external toolchains are available, their language support, etc. The
initial proof-of-concept would likely be limited to Rust to save on
time and complexity.
5. As the work matures, and perhaps as a requisite for eventual
inclusion, we could land support for more than Rust. This might be
a step too far, but hey, it's one of the thoughts I had.
6. So all of this wrapped up means that:
A. This is NOT a call to rewrite everything in Rust. This work will
only permit NEW, OPTIONAL components to be written.
B. Other languages/toolchains/ecosystems could be supported, not
just Rust.
C. Initial focus is on userland components. Rust in the kernel is
out of scope for this initial proof-of-concept.
D. I would like to see Rust in the kernel. That would be a good
next area of focus once userland support reaches some level of
maturity.
My first goal will be to get a better understanding of
src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
study your work, Alan. I really appreciate the time you have taken. I
might reach out to you and Warner directly for further questions.
Thanks,
--=20
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A=
4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
On Sun, Aug 04, 2024 at 11:55:26AM UTC, Alan Somers wrote:
> Due to all of the recent discussion of using Rust for code in the
> FreeBSD base, I've put together a demo of what it might look like. It
> demonstrates:
>=20
> * Interspersing Rust crates through the tree (usr.bin/nfs-exporter,
> cddl/usr.bin/ztop, etc) rather than in some special directory.
> * Build integration for all Rust crates. You can build them all with
> a single "cargo build" command from the top level, and test them all
> with a single "cargo test".
> * Wholly new programs written from scratch in Rust (ztop plus three
> Prometheus exporters)
> * Old programs rewritten in Rust with substantial new features (gstat and=
fsx)
> * Libs (freebsd-libgeom and freebsd-libgeom-sys)
> * Commits that reconcile the dependencies of multiple crates, so as to
> minimize duplicate dependency versions (5764fb383d4 and 1edf2e19e50)
> * Vendoring all dependencies, direct and transitive, to ensure
> internet-independent and reproducible builds (37ef9ffb6a6). This
> process is automated and requires almost no manual effort. Note:
> don't panic if you look in the "vendor" directory and see a bunch of
> crates with "windows" in the name. They're all just empty stubs.
> * All Rust object files get stored in the "target" directory rather
> than /usr/obj. Today, if you want them to be stored in /usr/obj the
> best way is to use a symlink, though there's WIP to add
> MAKEOBJDIRPREFIX-like functionality to Cargo.
>=20
> It does NOT demonstrate:
>=20
> * Integrating the Rust build system with Make. Warner has some ideas
> about how to do that.
> * Pulling rustc into contrib. This tree requires an external Rust toolch=
ain.
> * Building any cdylib libraries with Rust. That's useful if you want
> a C program to call a Rust library, but I don't have any good examples
> for it.
> * kernel modules. As already discussed, those are hard.
> * Any Rust crates that involve private APIs, like CTL stuff. Those
> are among the most tantalizing programs to move from ports to base,
> but nobody's written any yet, because Rust-in-base doesn't exist yet.
>=20
> Also, I want to address a question that's popped up a few times:
> backwards-compatibility. There is a fear that Rust code needs to be
> updated for each new toolchain release. But that's not true. It
> hasn't been true for most crates since Rust 1.0 was released about a
> decade ago. A few exotic crates required "nightly" features after
> that, but they are very few in number these days, and none of them are
> included in this branch's vendored sources. What Rust _does_ do is it
> releases a new toolchain about every six weeks. Each new release
> typically includes a few new features in the standard library and they
> often add more compiler warnings, too. Sometimes they include wholly
> new compiler features, but they are _always_ backwards compatible with
> existing syntax. Roughly every three years, Rust publishes a new
> "Edition". Rust Editions are very similar to C++ versions. i.e. Rust
> 2018 is to Rust 2021 as C++14 is to C++17. New editions can include
> backwards-incompatible syntax changes, but each crate always knows
> which Edition it uses. Crates of different Editions can be linked
> together in the same build. This branch, for example, contains crates
> using Editions 2015, 2018, and 2021.
>=20
> If you have any questions about what Rust in Base would look like,
> please examine this branch. And if you've never used Rust before, I
> highly encourage you to try it. It really is the best new
> systems-program language for decades. IMHO, it's the only one that's
> a compelling replacement for C++ in all new applications, and C in
> most.
>=20
> https://github.com/asomers/freebsd-src/tree/rust-in-base-demo
>=20
--mad2rcwhuximkfrn
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmbQ2C8ACgkQ/y5nonf4
4foYYw/9Hn62TRrKoAiOYER/khW8WjZR+7IKgOguGHFbA6ajMKmjrlpE/2n9srJ1
5jyV4WSuV6w5C4Ghrvf8ZBBrx/CPURXsgZ/Mk7p7edqjkpbXjgGmdwwAJeaB/jLn
/VwKXe7v0KbTRxwtOeuoCNYC31kty9J1UOpWEXwdW9/qJ84k9/cdcGNOiHNlstP1
mRYUPPZQZJSk9QQ3EsjemA/TtJCZHhnyba/HlwANApaW+XO0tg0f2HAaUyALXKZw
W4dvdh8wu0IeSaG4sLItYjYNoHifl3xWvgJaSkCnyuKrhwM+ez7qU6lCkGRAZ0Oq
MnLPBMICLuxLPiMvzFKApi/2IUmKHVnbMari9lCv4dUFNQt01h8v5FyuNRdVQ140
x/N5CATqvXsKRSuN3TmHxmgcP8SY0qN/TR67QECyzIY6nWD9BihNWDU5L1WHe2Do
o0yz8hJ9p9VUgkN006Rl86fy6iP5o2C3IdNXaWk0/t6juBJMN+Hcs++q4bbjjRwQ
iArK4sCk2ehg8ygp3WEnJ/SZuZHsCC69vveOcMgfUEPFBq4K6x92WmAVJFETqc+P
NtfLKw6/FnsZRl3S3mJmyxhVPJtabCx5K8lxNjWeeOxtU4IeBKkepTS7GuYK/0u4
qJg9TYJbln939bNYYpQFE0l2UUukwbUlcrCmMkw3F88BRh1lUjc=
=GVJD
-----END PGP SIGNATURE-----
--mad2rcwhuximkfrn--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?hraulhvnvyy5riur63iy53ed6m276prpw577qyg4xpm4xkyxmm>