Date: Sat, 20 Jan 2024 20:43:53 -0700 From: Warner Losh <imp@bsdimp.com> To: Alan Somers <asomers@freebsd.org> Cc: Tomoaki AOKI <junchoon@dec.sakura.ne.jp>, Aleksandr Fedorov <wigneddoom@yandex.ru>, FreeBSD Hackers <freebsd-hackers@freebsd.org>, Scott Long <scottl@freebsd.org>, "Goran Meki??" <meka@tilda.center> Subject: Re: The Case for Rust (in the base system) Message-ID: <CANCZdfpDVCP2qcz6HK1HeehBkLwyRxk2pmvP0FKGax%2BMcHmxag@mail.gmail.com> In-Reply-To: <CAOtMX2ho7b9VOnABzdRvWn_gNmz3_V1Ac1Rmo-XRC72sPTttKQ@mail.gmail.com> References: <CAOtMX2hAUiWdGPtpaCJLPZB%2Bj2yzNw5DSjUmkwTi%2B%2BmyemehCA@mail.gmail.com> <1673801705774097@mail.yandex.ru> <CANCZdfpqWgvV_RCvVO_pvTrmajQFspW%2BQ9TM_Ok3JrXZAfeAfA@mail.gmail.com> <20240121110611.af567b0ac3a8fd8593ffcb7f@dec.sakura.ne.jp> <CAOtMX2ho7b9VOnABzdRvWn_gNmz3_V1Ac1Rmo-XRC72sPTttKQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Sat, Jan 20, 2024, 7:20 PM Alan Somers <asomers@freebsd.org> wrote: > On Sat, Jan 20, 2024 at 7:06 PM Tomoaki AOKI <junchoon@dec.sakura.ne.jp> > wrote: > > > > On Sat, 20 Jan 2024 15:31:23 -0700 > > Warner Losh <imp@bsdimp.com> wrote: > > > > > On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov < > wigneddoom@yandex.ru> > > > wrote: > > > > > > > What about external dependencies? > > > > > > > > > https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19 > > > > > https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20 > > > > > > > > Is there any plan for which crates we should take into the base > system? > > > > > > > > We have had C++ in base for many years, but I don’t see any good > libraries > > > > for CLI, logging, JSON, etc. > > > > > > > > > > > > > https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools > > > > > > > > Where is the support for Freebsd as a primary platform? ARM, RISC-V, > > > > Power? Should we rewrite devd? > > > > > > > > I think we need to start by providing official repositories (e.g > > > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git) > > > > for different languages that include stable bindings to the system > API: > > > > - sysctl > > > > - libgeom > > > > - libifconfig > > > > - netgraph > > > > - jail > > > > - etc. > > > > > > > > So that it’s not just some anonymous on crates.io that represents > these > > > > bindings, but our community. > > > > Officially, with support for a stable ABI for releases, security > patches, > > > > etc. > > > > > > > > After this, it will be possible to think about which components to > include > > > > in the base system. > > > > > > > > I would be glad to see a more modern language than C in the > database, but > > > > I’m afraid that it will be like with C++, > > > > that we will get a couple of daemons and utilities and that’s all. > > > > > > > > > > These are all good questions that need good answers, though > necessarily to > > > get started. > > > > > > But the other question that occured to me after my last posting was > "What > > > about build integration?" > > > How much of the rust automation do we take in vs how much do we drive > from > > > a future bsd.rust.mk. > > > I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely > need > > > one for what we traditionally > > > think of as libraries (which may or may not map 1:1 onto crates: we > could > > > have c callable libraries > > > written in rust in the future, for example) and one for binaries. > > > Initially, though, if we go with the > > > 'make rust tests possible' then we'd likely need the appropriate > packages > > > installed for whatever > > > dependencies we'd have in the tests. This would give us a taste for > what > > > we'd need to do for > > > base, I'd think. Once we had that notion, I can easily see there > needing to > > > be some sort of > > > rust bindings for ATF/kyua as one of the first libraries / crates that > > > would test that aspect of > > > the build system. That all would be up to the people writing the tests > in > > > rust, I'd imagine. > > > > > > While I could jot out the basics of this integration (so one could > just add > > > the rust > > > tools to a subdir or subdirs, include the bsd.rust.mk or whatever and > then > > > it would build > > > if rust is enabled, and would emit a warning it was skipped because > rust > > > was disabled). > > > We'd find out if this is workable or not and iterate from there. But > that > > > would also require > > > active participation from the rust advocates to make it a reality: I > can > > > put together the > > > build infrastructure for the disabled case, but likely can't on my own > do > > > the rust enabled > > > case. I'd be happy to work with someone to do that, but I'm not going > to be > > > able to do > > > that myself: my need for rust is slight, my knowledge of rust is weak, > etc. > > > Working with > > > someone (or ideally several someones), though it could become reality. > So > > > please contact > > > me if you'd like to work on this. > > > > > > Warner > > > > One way to go could be moving programs rewritten with rust to ports. > > There are some programs (not in rust, though) moved to ports, like rcs. > > I've already done this with a few, though I didn't delete the C > versions from base. > usr.bin/gstat => sysutils/gstat-rs > tools/regression/fsx => devel/fsx > So % size `which gstat-rs` `which gstat` text data bss dec hex filename 2094442 176472 568 2271482 0x22a8fa /usr/local/sbin/gstat-rs 19350 1180 41 20571 0x505b /usr/sbin/gstat % file `which gstat-rs` `which gstat` /usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, stripped /usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 15.0 (1500008), FreeBSD-style, stripped 8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat` /usr/local/sbin/gstat-rs: libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000) libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000) libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000) libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000) libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000) /usr/sbin/gstat: libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000) libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000) libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000) libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000) libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000) libc.so.7 => /lib/libc.so.7 (0x4488aaf4000) libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000) libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000) libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000) libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000) So that looks scary, like rust is 100x larger binaries... But at runtime it's about the same: USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND imp 14735 0.0 0.0 14140 4828 0 S+ 20:38 0:00.04 gstat imp 14766 1.3 0.0 15772 6256 0 S+ 20:39 0:00.02 gstat-rs So the runtime size is at least in the same ballpark (still larger, but not crazy larger). More CPU too, but that's just a polling artifact I think (other times gstat had some, and gstat-rs didn't). Why is the rust binary so much larger? Are the rust runtime and dependencies statically linked? Warner > > > Currently, it would not be so realistic, but once we completely switch > > to pkgbase, IIUC, programs in base can sanely depemd on ports programs, > > excluding kernel and fundamental libraries. > > > > As non-rust consumers of graphics/librsvg2-rust can sanely link with > > it, I assume kmods in ports written in rust can kldload'ed sanely. > > This could be a good starting point. > > > > And would be not all, but test for rust libraries could be implemented > > with C/C++ or any other language suitable, if the rust libraries can > > sanely linked with test codes. > > Yes, if the Rust library implements a C interface, which most don't. > > > > > Am I wrong? > [-- Attachment #2 --] <div dir="ltr"><div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jan 20, 2024, 7:20 PM Alan Somers <<a href="mailto:asomers@freebsd.org" target="_blank">asomers@freebsd.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sat, Jan 20, 2024 at 7:06 PM Tomoaki AOKI <<a href="mailto:junchoon@dec.sakura.ne.jp" rel="noreferrer" target="_blank">junchoon@dec.sakura.ne.jp</a>> wrote:<br> ><br> > On Sat, 20 Jan 2024 15:31:23 -0700<br> > Warner Losh <<a href="mailto:imp@bsdimp.com" rel="noreferrer" target="_blank">imp@bsdimp.com</a>> wrote:<br> ><br> > > On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <<a href="mailto:wigneddoom@yandex.ru" rel="noreferrer" target="_blank">wigneddoom@yandex.ru</a>><br> > > wrote:<br> > ><br> > > > What about external dependencies?<br> > > ><br> > > > <a href="https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19" rel="noreferrer noreferrer" target="_blank">https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19</a><br>; > > > <a href="https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20" rel="noreferrer noreferrer" target="_blank">https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20</a><br>; > > ><br> > > > Is there any plan for which crates we should take into the base system?<br> > > ><br> > > > We have had C++ in base for many years, but I don’t see any good libraries<br> > > > for CLI, logging, JSON, etc.<br> > > ><br> > > ><br> > > > <a href="https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools" rel="noreferrer noreferrer" target="_blank">https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools</a><br>; > > ><br> > > > Where is the support for Freebsd as a primary platform? ARM, RISC-V,<br> > > > Power? Should we rewrite devd?<br> > > ><br> > > > I think we need to start by providing official repositories (e.g<br> > > > <a href="http://git.FreeBSD.org/rust.git" rel="noreferrer noreferrer" target="_blank">git.FreeBSD.org/rust.git</a> or <a href="http://git.FreeBSD.org/go.git" rel="noreferrer noreferrer" target="_blank">git.FreeBSD.org/go.git</a>)<br> > > > for different languages that include stable bindings to the system API:<br> > > > - sysctl<br> > > > - libgeom<br> > > > - libifconfig<br> > > > - netgraph<br> > > > - jail<br> > > > - etc.<br> > > ><br> > > > So that it’s not just some anonymous on <a href="http://crates.io" rel="noreferrer noreferrer" target="_blank">crates.io</a> that represents these<br> > > > bindings, but our community.<br> > > > Officially, with support for a stable ABI for releases, security patches,<br> > > > etc.<br> > > ><br> > > > After this, it will be possible to think about which components to include<br> > > > in the base system.<br> > > ><br> > > > I would be glad to see a more modern language than C in the database, but<br> > > > I’m afraid that it will be like with C++,<br> > > > that we will get a couple of daemons and utilities and that’s all.<br> > > ><br> > ><br> > > These are all good questions that need good answers, though necessarily to<br> > > get started.<br> > ><br> > > But the other question that occured to me after my last posting was "What<br> > > about build integration?"<br> > > How much of the rust automation do we take in vs how much do we drive from<br> > > a future <a href="http://bsd.rust.mk" rel="noreferrer noreferrer" target="_blank">bsd.rust.mk</a>.<br> > > I can sketch out <a href="http://bsd.rust.mk" rel="noreferrer noreferrer" target="_blank">bsd.rust.mk</a> (to pick an arbitrary name, we'd likely need<br> > > one for what we traditionally<br> > > think of as libraries (which may or may not map 1:1 onto crates: we could<br> > > have c callable libraries<br> > > written in rust in the future, for example) and one for binaries.<br> > > Initially, though, if we go with the<br> > > 'make rust tests possible' then we'd likely need the appropriate packages<br> > > installed for whatever<br> > > dependencies we'd have in the tests. This would give us a taste for what<br> > > we'd need to do for<br> > > base, I'd think. Once we had that notion, I can easily see there needing to<br> > > be some sort of<br> > > rust bindings for ATF/kyua as one of the first libraries / crates that<br> > > would test that aspect of<br> > > the build system. That all would be up to the people writing the tests in<br> > > rust, I'd imagine.<br> > ><br> > > While I could jot out the basics of this integration (so one could just add<br> > > the rust<br> > > tools to a subdir or subdirs, include the <a href="http://bsd.rust.mk" rel="noreferrer noreferrer" target="_blank">bsd.rust.mk</a> or whatever and then<br> > > it would build<br> > > if rust is enabled, and would emit a warning it was skipped because rust<br> > > was disabled).<br> > > We'd find out if this is workable or not and iterate from there. But that<br> > > would also require<br> > > active participation from the rust advocates to make it a reality: I can<br> > > put together the<br> > > build infrastructure for the disabled case, but likely can't on my own do<br> > > the rust enabled<br> > > case. I'd be happy to work with someone to do that, but I'm not going to be<br> > > able to do<br> > > that myself: my need for rust is slight, my knowledge of rust is weak, etc.<br> > > Working with<br> > > someone (or ideally several someones), though it could become reality. So<br> > > please contact<br> > > me if you'd like to work on this.<br> > ><br> > > Warner<br> ><br> > One way to go could be moving programs rewritten with rust to ports.<br> > There are some programs (not in rust, though) moved to ports, like rcs.<br> <br> I've already done this with a few, though I didn't delete the C<br> versions from base.<br> usr.bin/gstat => sysutils/gstat-rs<br> tools/regression/fsx => devel/fsx<br></blockquote></div></div><div dir="auto"><br></div><div>So</div><div>% size `which gstat-rs` `which gstat`<br> text data bss dec hex filename<br> 2094442 176472 568 2271482 0x22a8fa /usr/local/sbin/gstat-rs<br> 19350 1180 41 20571 0x505b /usr/sbin/gstat<br></div><div>% file `which gstat-rs` `which gstat`<br>/usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, FreeBSD-style, stripped<br>/usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 15.0 (1500008), FreeBSD-style, stripped<br>8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`<br>/usr/local/sbin/gstat-rs:<br> libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)<br> libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)<br> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)<br> libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)<br> libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)<br> libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)<br>/usr/sbin/gstat:<br> libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)<br> libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)<br> libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)<br> libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)<br> libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)<br> libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)<br> libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)<br> libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)<br> libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)<br> libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)</div><div><br></div><div>So that looks scary, like rust is 100x larger binaries... But at runtime it's about the same:</div><div>USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND</div><div>imp 14735 0.0 0.0 14140 4828 0 S+ 20:38 0:00.04 gstat</div><div>imp 14766 1.3 0.0 15772 6256 0 S+ 20:39 0:00.02 gstat-rs</div><div><br></div><div>So the runtime size is at least in the same ballpark (still larger, but not crazy larger). More CPU too,</div><div>but that's just a polling artifact I think (other times gstat had some, and gstat-rs didn't).<br></div><div><br></div><div>Why is the rust binary so much larger? Are the rust runtime and dependencies statically linked?<br></div><div dir="auto"><br></div><div dir="auto">Warner</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> ><br> > Currently, it would not be so realistic, but once we completely switch<br> > to pkgbase, IIUC, programs in base can sanely depemd on ports programs,<br> > excluding kernel and fundamental libraries.<br> ><br> > As non-rust consumers of graphics/librsvg2-rust can sanely link with<br> > it, I assume kmods in ports written in rust can kldload'ed sanely.<br> > This could be a good starting point.<br> ><br> > And would be not all, but test for rust libraries could be implemented<br> > with C/C++ or any other language suitable, if the rust libraries can<br> > sanely linked with test codes.<br> <br> Yes, if the Rust library implements a C interface, which most don't.<br> <br> ><br> > Am I wrong?<br> </blockquote></div></div></div> </div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpDVCP2qcz6HK1HeehBkLwyRxk2pmvP0FKGax%2BMcHmxag>