Date: Sat, 20 Jan 2024 21:14:59 -0700 From: Alan Somers <asomers@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: Alan Somers <asomers@freebsd.org>, Tomoaki AOKI <junchoon@dec.sakura.ne.jp>, Aleksandr Fedorov <wigneddoom@yandex.ru>, FreeBSD Hackers <freebsd-hackers@freebsd.org>, Scott Long <scottl@freebsd.org>, "Goran Meki??" <meka@tilda.center> Subject: Re: The Case for Rust (in the base system) Message-ID: <CAOtMX2gMTG_p8htNa5n0cCYKnZ8LoS5-7CJsC9aQ-Z-ja7raRg@mail.gmail.com> In-Reply-To: <CANCZdfpDVCP2qcz6HK1HeehBkLwyRxk2pmvP0FKGax%2BMcHmxag@mail.gmail.com> References: <CAOtMX2hAUiWdGPtpaCJLPZB%2Bj2yzNw5DSjUmkwTi%2B%2BmyemehCA@mail.gmail.com> <1673801705774097@mail.yandex.ru> <CANCZdfpqWgvV_RCvVO_pvTrmajQFspW%2BQ9TM_Ok3JrXZAfeAfA@mail.gmail.com> <20240121110611.af567b0ac3a8fd8593ffcb7f@dec.sakura.ne.jp> <CAOtMX2ho7b9VOnABzdRvWn_gNmz3_V1Ac1Rmo-XRC72sPTttKQ@mail.gmail.com> <CANCZdfpDVCP2qcz6HK1HeehBkLwyRxk2pmvP0FKGax%2BMcHmxag@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000009c7c2c060f6cf5db Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Jan 20, 2024, 8:44=E2=80=AFPM Warner Losh <imp@bsdimp.com> wrote: > > > On Sat, Jan 20, 2024, 7:20=E2=80=AFPM Alan Somers <asomers@freebsd.org> w= rote: > >> On Sat, Jan 20, 2024 at 7:06=E2=80=AFPM Tomoaki AOKI <junchoon@dec.sakur= a.ne.jp> >> wrote: >> > >> > On Sat, 20 Jan 2024 15:31:23 -0700 >> > Warner Losh <imp@bsdimp.com> wrote: >> > >> > > On Sat, Jan 20, 2024 at 11:45=E2=80=AFAM Aleksandr Fedorov < >> wigneddoom@yandex.ru> >> > > wrote: >> > > >> > > > What about external dependencies? >> > > > >> > > > >> https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L= 19 >> > > > >> https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20 >> > > > >> > > > Is there any plan for which crates we should take into the base >> system? >> > > > >> > > > We have had C++ in base for many years, but I don=E2=80=99t see an= y good >> libraries >> > > > for CLI, logging, JSON, etc. >> > > > >> > > > >> > > > >> https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-t= ools >> > > > >> > > > Where is the support for Freebsd as a primary platform? ARM, RISC-= V, >> > > > Power? Should we rewrite devd? >> > > > >> > > > I think we need to start by providing official repositories (e.g >> > > > git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git) >> > > > for different languages that include stable bindings to the system >> API: >> > > > - sysctl >> > > > - libgeom >> > > > - libifconfig >> > > > - netgraph >> > > > - jail >> > > > - etc. >> > > > >> > > > So that it=E2=80=99s not just some anonymous on crates.io that rep= resents >> these >> > > > bindings, but our community. >> > > > Officially, with support for a stable ABI for releases, security >> patches, >> > > > etc. >> > > > >> > > > After this, it will be possible to think about which components to >> include >> > > > in the base system. >> > > > >> > > > I would be glad to see a more modern language than C in the >> database, but >> > > > I=E2=80=99m afraid that it will be like with C++, >> > > > that we will get a couple of daemons and utilities and that=E2=80= =99s all. >> > > > >> > > >> > > These are all good questions that need good answers, though >> necessarily to >> > > get started. >> > > >> > > But the other question that occured to me after my last posting was >> "What >> > > about build integration?" >> > > How much of the rust automation do we take in vs how much do we driv= e >> from >> > > a future bsd.rust.mk. >> > > I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely >> need >> > > one for what we traditionally >> > > think of as libraries (which may or may not map 1:1 onto crates: we >> could >> > > have c callable libraries >> > > written in rust in the future, for example) and one for binaries. >> > > Initially, though, if we go with the >> > > 'make rust tests possible' then we'd likely need the appropriate >> packages >> > > installed for whatever >> > > dependencies we'd have in the tests. This would give us a taste for >> what >> > > we'd need to do for >> > > base, I'd think. Once we had that notion, I can easily see there >> needing to >> > > be some sort of >> > > rust bindings for ATF/kyua as one of the first libraries / crates th= at >> > > would test that aspect of >> > > the build system. That all would be up to the people writing the >> tests in >> > > rust, I'd imagine. >> > > >> > > While I could jot out the basics of this integration (so one could >> just add >> > > the rust >> > > tools to a subdir or subdirs, include the bsd.rust.mk or whatever >> and then >> > > it would build >> > > if rust is enabled, and would emit a warning it was skipped because >> rust >> > > was disabled). >> > > We'd find out if this is workable or not and iterate from there. But >> that >> > > would also require >> > > active participation from the rust advocates to make it a reality: I >> can >> > > put together the >> > > build infrastructure for the disabled case, but likely can't on my >> own do >> > > the rust enabled >> > > case. I'd be happy to work with someone to do that, but I'm not goin= g >> to be >> > > able to do >> > > that myself: my need for rust is slight, my knowledge of rust is >> weak, etc. >> > > Working with >> > > someone (or ideally several someones), though it could become >> reality. So >> > > please contact >> > > me if you'd like to work on this. >> > > >> > > Warner >> > >> > One way to go could be moving programs rewritten with rust to ports. >> > There are some programs (not in rust, though) moved to ports, like rcs= . >> >> I've already done this with a few, though I didn't delete the C >> versions from base. >> usr.bin/gstat =3D> sysutils/gstat-rs >> tools/regression/fsx =3D> devel/fsx >> > > So > % size `which gstat-rs` `which gstat` > text data bss dec hex filename > 2094442 176472 568 2271482 0x22a8fa /usr/local/sbin/gstat-rs > 19350 1180 41 20571 0x505b /usr/sbin/gstat > % file `which gstat-rs` `which gstat` > /usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64, > version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1= , > FreeBSD-style, stripped > /usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM aarch64, > version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1= , > for FreeBSD 15.0 (1500008), FreeBSD-style, stripped > 8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat` > /usr/local/sbin/gstat-rs: > libgeom.so.5 =3D> /lib/libgeom.so.5 (0x60fd38647000) > libthr.so.3 =3D> /lib/libthr.so.3 (0x60fd38b57000) > libgcc_s.so.1 =3D> /lib/libgcc_s.so.1 (0x60fd39af1000) > libc.so.7 =3D> /lib/libc.so.7 (0x60fd3be6f000) > libbsdxml.so.4 =3D> /lib/libbsdxml.so.4 (0x60fd3a009000) > libsbuf.so.6 =3D> /lib/libsbuf.so.6 (0x60fd3a55e000) > /usr/sbin/gstat: > libdevstat.so.7 =3D> /lib/libdevstat.so.7 (0x448867cd000) > libgeom.so.5 =3D> /lib/libgeom.so.5 (0x4488710b000) > libedit.so.8 =3D> /lib/libedit.so.8 (0x44887f8d000) > libtinfow.so.9 =3D> /lib/libtinfow.so.9 (0x44888aab000) > libncursesw.so.9 =3D> /lib/libncursesw.so.9 (0x44889c60000) > libc.so.7 =3D> /lib/libc.so.7 (0x4488aaf4000) > libkvm.so.7 =3D> /lib/libkvm.so.7 (0x44888f77000) > libbsdxml.so.4 =3D> /lib/libbsdxml.so.4 (0x4488ba02000) > libsbuf.so.6 =3D> /lib/libsbuf.so.6 (0x4488c68d000) > libelf.so.2 =3D> /lib/libelf.so.2 (0x4488ca45000) > > So that looks scary, like rust is 100x larger binaries... But at runtime > it's about the same: > USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND > imp 14735 0.0 0.0 14140 4828 0 S+ 20:38 0:00.04 gstat > imp 14766 1.3 0.0 15772 6256 0 S+ 20:39 0:00.02 gstat-r= s > > So the runtime size is at least in the same ballpark (still larger, but > not crazy larger). More CPU too, > but that's just a polling artifact I think (other times gstat had some, > and gstat-rs didn't). > > Why is the rust binary so much larger? Are the rust runtime and > dependencies statically linked? > Yes, that's a large part of it. Rust libraries are usually statically linked (though they don't have to be). For example, in the output above, notice that gstat-rs does not link to ncurses. That's because the equivalent library is statically linked in instead. Also, rust programs by default include goodies like stack unwinding on panic, which takes extra code too. But that can be turned off if you really want to save space. -Alan > --0000000000009c7c2c060f6cf5db Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><br><br><div class=3D"gmail_quote" dir=3D"auto"><div dir= =3D"ltr" class=3D"gmail_attr">On Sat, Jan 20, 2024, 8:44=E2=80=AFPM Warner = Losh <<a href=3D"mailto:imp@bsdimp.com" target=3D"_blank" rel=3D"norefer= rer">imp@bsdimp.com</a>> wrote:<br></div><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">= <div dir=3D"ltr"><div dir=3D"auto"><div><br><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Sat, Jan 20, 2024, 7:20=E2=80=AFPM= Alan Somers <<a href=3D"mailto:asomers@freebsd.org" rel=3D"noreferrer n= oreferrer" target=3D"_blank">asomers@freebsd.org</a>> wrote:<br></div><b= lockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px = #ccc solid;padding-left:1ex">On Sat, Jan 20, 2024 at 7:06=E2=80=AFPM Tomoak= i AOKI <<a href=3D"mailto:junchoon@dec.sakura.ne.jp" rel=3D"noreferrer n= oreferrer noreferrer" target=3D"_blank">junchoon@dec.sakura.ne.jp</a>> w= rote:<br> ><br> > On Sat, 20 Jan 2024 15:31:23 -0700<br> > Warner Losh <<a href=3D"mailto:imp@bsdimp.com" rel=3D"noreferrer no= referrer noreferrer" target=3D"_blank">imp@bsdimp.com</a>> wrote:<br> ><br> > > On Sat, Jan 20, 2024 at 11:45=E2=80=AFAM Aleksandr Fedorov <<a= href=3D"mailto:wigneddoom@yandex.ru" rel=3D"noreferrer noreferrer noreferr= er" target=3D"_blank">wigneddoom@yandex.ru</a>><br> > > wrote:<br> > ><br> > > > What about external dependencies?<br> > > ><br> > > > <a href=3D"https://github.com/Axcient/freebsd-nfs-exporter/b= lob/master/Cargo.toml#L19" rel=3D"noreferrer noreferrer noreferrer noreferr= er" target=3D"_blank">https://github.com/Axcient/freebsd-nfs-exporter/blob/= master/Cargo.toml#L19</a><br> > > > <a href=3D"https://github.com/asomers/gstat-rs/blob/master/g= stat/src/main.rs#L20" rel=3D"noreferrer noreferrer noreferrer noreferrer" t= arget=3D"_blank">https://github.com/asomers/gstat-rs/blob/master/gstat/src/= main.rs#L20</a><br> > > ><br> > > > Is there any plan for which crates we should take into the b= ase system?<br> > > ><br> > > > We have had C++ in base for many years, but I don=E2=80=99t = see any good libraries<br> > > > for CLI, logging, JSON, etc.<br> > > ><br> > > ><br> > > > <a href=3D"https://doc.rust-lang.org/rustc/platform-support.= html#tier-1-with-host-tools" rel=3D"noreferrer noreferrer noreferrer norefe= rrer" target=3D"_blank">https://doc.rust-lang.org/rustc/platform-support.ht= ml#tier-1-with-host-tools</a><br> > > ><br> > > > Where is the support for Freebsd as a primary platform? ARM,= RISC-V,<br> > > > Power? Should we rewrite devd?<br> > > ><br> > > > I think we need to start by providing official repositories = (e.g<br> > > > <a href=3D"http://git.FreeBSD.org/rust.git" rel=3D"noreferre= r noreferrer noreferrer noreferrer" target=3D"_blank">git.FreeBSD.org/rust.= git</a> or <a href=3D"http://git.FreeBSD.org/go.git" rel=3D"noreferrer nore= ferrer noreferrer noreferrer" target=3D"_blank">git.FreeBSD.org/go.git</a>)= <br> > > > for different languages that include stable bindings to the = system API:<br> > > > - sysctl<br> > > > - libgeom<br> > > > - libifconfig<br> > > > - netgraph<br> > > > - jail<br> > > > - etc.<br> > > ><br> > > > So that it=E2=80=99s not just some anonymous on <a href=3D"h= ttp://crates.io" rel=3D"noreferrer noreferrer noreferrer noreferrer" target= =3D"_blank">crates.io</a> that represents these<br> > > > bindings, but our community.<br> > > > Officially, with support for a stable ABI for releases, secu= rity patches,<br> > > > etc.<br> > > ><br> > > > After this, it will be possible to think about which compone= nts to include<br> > > > in the base system.<br> > > ><br> > > > I would be glad to see a more modern language than C in the = database, but<br> > > > I=E2=80=99m afraid that it will be like with C++,<br> > > > that we will get a couple of daemons and utilities and that= =E2=80=99s all.<br> > > ><br> > ><br> > > These are all good questions that need good answers, though neces= sarily to<br> > > get started.<br> > ><br> > > But the other question that occured to me after my last posting w= as "What<br> > > about build integration?"<br> > > How much of the rust automation do we take in vs how much do we d= rive from<br> > > a future <a href=3D"http://bsd.rust.mk" rel=3D"noreferrer norefer= rer noreferrer noreferrer" target=3D"_blank">bsd.rust.mk</a>.<br> > > I can sketch out <a href=3D"http://bsd.rust.mk" rel=3D"noreferrer= noreferrer noreferrer noreferrer" target=3D"_blank">bsd.rust.mk</a> (to pi= ck an arbitrary name, we'd likely need<br> > > one for what we traditionally<br> > > think of as libraries (which may or may not map 1:1 onto crates: = we could<br> > > have c callable libraries<br> > > written in rust in the future, for example) and one for binaries.= <br> > > Initially, though, if we go with the<br> > > 'make rust tests possible' then we'd likely need the = appropriate packages<br> > > installed for whatever<br> > > dependencies we'd have in the tests. This would give us a tas= te for what<br> > > we'd need to do for<br> > > base, I'd think. Once we had that notion, I can easily see th= ere needing to<br> > > be some sort of<br> > > rust bindings for ATF/kyua as one of the first libraries / crates= that<br> > > would test that aspect of<br> > > the build system. That all would be up to the people writing the = tests in<br> > > rust, I'd imagine.<br> > ><br> > > While I could jot out the basics of this integration (so one coul= d just add<br> > > the rust<br> > > tools to a subdir or subdirs, include the <a href=3D"http://bsd.r= ust.mk" rel=3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blan= k">bsd.rust.mk</a> or whatever and then<br> > > it would build<br> > > if rust is enabled, and would emit a warning it was skipped becau= se rust<br> > > was disabled).<br> > > We'd find out if this is workable or not and iterate from the= re. But that<br> > > would also require<br> > > active participation from the rust advocates to make it a reality= : I can<br> > > put together the<br> > > build infrastructure for the disabled case, but likely can't = on my own do<br> > > the rust enabled<br> > > case. I'd be happy to work with someone to do that, but I'= ;m not going to be<br> > > able to do<br> > > that myself: my need for rust is slight, my knowledge of rust is = weak, etc.<br> > > Working with<br> > > someone (or ideally several someones), though it could become rea= lity. So<br> > > please contact<br> > > me if you'd like to work on this.<br> > ><br> > > Warner<br> ><br> > One way to go could be moving programs rewritten with rust to ports.<b= r> > There are some programs (not in rust, though) moved to ports, like rcs= .<br> <br> I've already done this with a few, though I didn't delete the C<br> versions from base.<br> usr.bin/gstat =3D> sysutils/gstat-rs<br> tools/regression/fsx =3D> devel/fsx<br></blockquote></div></div><div dir= =3D"auto"><br></div><div>So</div><div>% size `which gstat-rs` `which gstat`= <br>=C2=A0 =C2=A0 =C2=A0text =C2=A0 =C2=A0 data =C2=A0 bss =C2=A0 =C2=A0 = =C2=A0 dec =C2=A0 =C2=A0 =C2=A0 =C2=A0hex =C2=A0 filename<br>=C2=A0 2094442= =C2=A0 176472 =C2=A0 568 =C2=A0 2271482 =C2=A0 0x22a8fa =C2=A0 /usr/local/= sbin/gstat-rs<br>=C2=A0 =C2=A0 19350 =C2=A0 =C2=A0 1180 =C2=A0 =C2=A041 =C2= =A0 =C2=A0 20571 =C2=A0 =C2=A0 0x505b =C2=A0 /usr/sbin/gstat<br></div><div>= % file `which gstat-rs` `which gstat`<br>/usr/local/sbin/gstat-rs: ELF 64-b= it LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked= , interpreter /libexec/ld-elf.so.1, FreeBSD-style, stripped<br>/usr/sbin/gs= tat: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ELF 64-bit LSB pie executable, ARM a= arch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-el= f.so.1, for FreeBSD 15.0 (1500008), FreeBSD-style, stripped<br>8:36pm brazo= s:[3826]> ldd `which gstat-rs` `which gstat`<br>/usr/local/sbin/gstat-rs= :<br> libgeom.so.5 =3D> /lib/libgeom.so.5 (0x60fd38647000)<br> libthr.so= .3 =3D> /lib/libthr.so.3 (0x60fd38b57000)<br> libgcc_s.so.1 =3D> /lib= /libgcc_s.so.1 (0x60fd39af1000)<br> libc.so.7 =3D> /lib/libc.so.7 (0x60f= d3be6f000)<br> libbsdxml.so.4 =3D> /lib/libbsdxml.so.4 (0x60fd3a009000)<= br> libsbuf.so.6 =3D> /lib/libsbuf.so.6 (0x60fd3a55e000)<br>/usr/sbin/gs= tat:<br> libdevstat.so.7 =3D> /lib/libdevstat.so.7 (0x448867cd000)<br> l= ibgeom.so.5 =3D> /lib/libgeom.so.5 (0x4488710b000)<br> libedit.so.8 =3D&= gt; /lib/libedit.so.8 (0x44887f8d000)<br> libtinfow.so.9 =3D> /lib/libti= nfow.so.9 (0x44888aab000)<br> libncursesw.so.9 =3D> /lib/libncursesw.so.= 9 (0x44889c60000)<br> libc.so.7 =3D> /lib/libc.so.7 (0x4488aaf4000)<br> = libkvm.so.7 =3D> /lib/libkvm.so.7 (0x44888f77000)<br> libbsdxml.so.4 =3D= > /lib/libbsdxml.so.4 (0x4488ba02000)<br> libsbuf.so.6 =3D> /lib/libs= buf.so.6 (0x4488c68d000)<br> libelf.so.2 =3D> /lib/libelf.so.2 (0x4488ca= 45000)</div><div><br></div><div>So that looks scary, like rust is 100x larg= er binaries...=C2=A0 But at runtime it's about the same:</div><div>USER= =C2=A0 =C2=A0PID =C2=A0 %CPU %MEM =C2=A0 VSZ =C2=A0 RSS TT =C2=A0STAT STAR= TED =C2=A0 =C2=A0 =C2=A0 =C2=A0 TIME COMMAND</div><div>imp =C2=A0 14735 =C2= =A0 =C2=A00.0 =C2=A00.0 14140 =C2=A04828 =C2=A00 =C2=A0S+ =C2=A0 20:38 =C2= =A0 =C2=A0 =C2=A0 =C2=A00:00.04 gstat</div><div>imp =C2=A0 14766 =C2=A0 =C2= =A01.3 =C2=A00.0 15772 =C2=A06256 =C2=A00 =C2=A0S+ =C2=A0 20:39 =C2=A0 =C2= =A0 =C2=A0 =C2=A00:00.02 gstat-rs</div><div><br></div><div>So the runtime s= ize is at least in the same ballpark (still larger, but not crazy larger). = More CPU too,</div><div>but that's just a polling artifact I think (oth= er times gstat had some, and gstat-rs didn't).<br></div><div><br></div>= <div>Why is the rust binary so much larger? Are the rust runtime and depend= encies statically linked?</div></div></div></blockquote></div><div dir=3D"a= uto"><br></div><div dir=3D"auto">Yes, that's a large part of it. Rust l= ibraries are usually statically linked (though they don't have to be). = For example, in the output above, notice that gstat-rs does not link to ncu= rses. That's because the equivalent library is statically linked in ins= tead. Also, rust programs by default include goodies like stack unwinding o= n panic, which takes extra code too. But that can be turned off if you real= ly want to save space.=C2=A0</div><div dir=3D"auto">-Alan</div><div class= =3D"gmail_quote" dir=3D"auto"><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"lt= r"> </div> </blockquote></div></div> --0000000000009c7c2c060f6cf5db--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2gMTG_p8htNa5n0cCYKnZ8LoS5-7CJsC9aQ-Z-ja7raRg>