Date: Wed, 31 Jul 2024 18:51:39 +0000 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Warner Losh <imp@bsdimp.com> Cc: Alan Somers <asomers@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org>, Scott Long <scottl@freebsd.org>, Goran Meki?? <meka@tilda.center> Subject: Re: The Case for Rust (in the base system) Message-ID: <knnsh327gxyvaajwrymvflnivf3tsnigyqw2d6etfhb4irft3x@ydkh3zmb6uch> In-Reply-To: <CANCZdfqB1%2B-8BkpKwKoCM%2BzM4mCOFy63yHr1Pco7MnT1DFkb4w@mail.gmail.com> References: <CAOtMX2hAUiWdGPtpaCJLPZB%2Bj2yzNw5DSjUmkwTi%2B%2BmyemehCA@mail.gmail.com> <vdmg5zocd6wqcwc2bvzvzqn4bii2pwdc2r4mgnisukfkboj6nf@f7lv5quu4fjx> <CAOtMX2iDK3uN_oQgzzZAdoOZCfNsnvpefeZvKoTCRmPBhZywzA@mail.gmail.com> <CANCZdfqB1%2B-8BkpKwKoCM%2BzM4mCOFy63yHr1Pco7MnT1DFkb4w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--l2ox7s2au2w6utkz Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 31, 2024 at 11:01:17AM -0600, Warner Losh wrote: > On Wed, Jul 31, 2024, 9:40=E2=80=AFAM Alan Somers <asomers@freebsd.org> w= rote: >=20 > > On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb <shawn.webb@hardened= bsd.org> > > wrote: > > > > > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > > > > In a recent thread on src-committers, we discussed the costs and > > > > benefits of including Rust code in the FreeBSD base system. To > > > > summarize, the cost is that it would double our build times. imp > > > > suggested adding an additional step after buildworld for stuff that > > > > requires an external toolchain. That would ease the build time pai= n. > > > > The benefit is that some tools would become easier to write, or even > > > > become possible. Here is a list of actual and potential Rust proje= cts > > > > that could benefit from being in-tree. If anybody else has items to > > > > add, I suggest moving this into the project wiki: > > > > > > > > Stuff that could only be written in Rust if it were in base > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > * ctl-exporter (I started this, but discovered that the CTL stats A= PI > > is > > > > unstable, so it can't live in ports. Instead, I had to do it in = C). > > > > > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a864= 01469181a67ec34 > > > > > > > > * fusefs tests. Absolutely impossible to do in C. I considered Ru= st, > > but went > > > > with C++ so they could live in base. They are too closely couple= d to > > > > fusefs(5) to live out-of-tree. > > > > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fus= efs > > > > > > > > * devd. Currently C++, but imp suggested a rewrite. > > > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > > > > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using Ru= st > > would > > > > make it more testable. > > > > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zf= sd > > > > > > > > * nscd. Currently C, but confusing and with no test coverage. I've > > > > contemplated a rewrite myself, but I don't want to do it in C. > > > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > > > > > > > * The userland portion of the 802.11ac and Lightning stacks. scottl > > suggested > > > > that these were good candidates for Rust. > > > > > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > > > > > > > Stuff that can live in ports, but would be nicer in base > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > * gstat-rs https://crates.io/crates/gstat > > > > > > > > * geom-exporter (I've started this, but haven't published it) > > > > > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > > > > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But = if > > the > > > > connection to bhyve(8) is too intimate, it might be hard to do in > > ports. > > > > https://gitlab.com/virtio-fs/virtiofsd > > > > > > > > * jail-exporter https://crates.io/crates/jail_exporter > > > > > > > > * Various jail managers have been attempted in Rust. I think these > > are fine in > > > > ports, but others like Goran Mekic have opined that they should be > > moved to > > > > base instead. > > > > > > > > * musikid's pjdfstest rewrite. I think it would be great to start > > using this > > > > to test the base system's file systems. If the tests themselves > > lived in > > > > base, they would be easier to sync with file system development. > > > > https://github.com/musikid/pjdfstest > > > > > > > > * pf-rs. I suspect that the API isn't very stable. > > > > https://crates.io/crates/pf-rs > > > > > > > > * benchpmc. The pmc counter names changes between releases. > > > > https://crates.io/crates/benchpmc > > > > > > > > FreeBSD-related applications that are just fine in ports > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those = are > > usually > > > > more stable than control path APIs, so I think there's little to = be > > gained by > > > > moving this into base. https://crates.io/crates/fsx > > > > > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty sta= ble. > > > > https://crates.io/crates/ztop > > > > > > > > * iocage-provision https://crates.io/crates/iocage-provision > > > > > > > > * rsblk https://crates.io/crates/rsblk > > > > > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > > > > > > > Other FreeBSD-related libraries in Rust > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > Just see the list at https://crates.io/keywords/freebsd > > > > > > > > > > One new data point: DARPA is looking to rewrite a significant amount > > > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > > > project: > > > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view > > > > Interesting. And since you bring it up, I have two new data points mys= elf: > > > > * ctld: while working on some bugs in ctld, I had trouble > > understanding the config file parsing. So I rewrote that part in > > Rust, just to help my understanding. Later, I rewrote the XML > > parsing, too. Then I rewrote the LUN creation and deletion, just to > > see how hard it would be. All of those parts take about 5x fewer SLOC > > in Rust than in C, and they're less buggy, too. Config file parsing > > is more consistent, no memory leaks, etc. Alas, I'm not planning to > > finish this project, since the base system doesn't allow Rust and ctld > > is too tightly coupled to ctl to live in ports. > > >=20 > Cool. Still waiting for anybody to take me up on the offer to do build > system integration. Since the Rust advocates can't get even this basic st= ep > done for review, it's going to be impossible to have Rust in the base. Th= is > isn't even integrate rust compiler like we do with llvm, but with external > Rust toolchain. >=20 > Until somebody steps up for this task, the status quo can't possibly chan= ge. Back at the FreeBSD Developer Summit at this last BSDCan, there was interest in supporting optional external toolchains in the src build framework. You had mentioned you would be happy to mentor someone, but not do the nitty gritty yourself. I could carve off some time in September to be the primary developer, doing the nitty gritty work. Would you be comfortable answering my questions, should I have any? Also: what work (or research), if any, has been done on the concept of external toolchain support for optional components in the FreeBSD source tree? Am I starting afresh or building upon existing work? Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --l2ox7s2au2w6utkz Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmaqh7MACgkQ/y5nonf4 4frCGg//fh37VVA0JqPk72OOOoPW3RmJ+AhJ2F5KHKMVScrgJGl6vrzmrbqfJe0P 0tp+6TCx/nSHZHfqq8RL+S5vimv7vq+BdZgyKoFSnprjsLODe28azIilvaj/z5tJ 2DIUMWVkIOyDHojx8uxe3B/EfSOrryVGWjx1ltwRn2AcG/lEEqedwBej999fd2jo 8zBbrM/Zi/B7A2u+Sp0WM+D4jvdQLr6GHK+hteZNkovAH1O5SxdGg7nZVFIWSP8h tQpCnt7hg8awyE4AT5FPNZmtJdHnMg9K9r8I2peOwUaFleuqiIJ/eW0FP3fZ6QVh ilda8XfPiZ5TYBM5rnhZR/3vsjz68zhKpo8AypAIeLaUqFNqDIHz4VuyYwhSPox0 HDjttwThDizqqx9Pg0j7YaStjVNH8mlIgLYT22hqH8eDFSnQfInLRX7n37HcZ8DU WsSW2Z0tfEwBG587T95xVdwxlM+jZjH2mdXCaRYhb5drsEp1jePY6AAfeK6iPjmo qXF1aOffPfAkA5wqspWlcklfbpUqMh8w2RqgzrWSw5vzE3mK7gU1dRYI1Z7s0Eii lz/3dxstAYJzcwoN/Rnavehitohj9QwNX0BH5Qy2OecZAjiav8IbZ0irfxnr+8Bo ii6yD7FJiWRC7EnGq3Rg82JHlh0/+FNtbLPGzqVhxf93arxRQJ8= =oSjC -----END PGP SIGNATURE----- --l2ox7s2au2w6utkz--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?knnsh327gxyvaajwrymvflnivf3tsnigyqw2d6etfhb4irft3x>