From nobody Wed Jul 31 15:40:31 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WYxBh5DP9z5S5nW for ; Wed, 31 Jul 2024 15:40:44 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f45.google.com (mail-ua1-f45.google.com [209.85.222.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WYxBh3KwDz3wtM; Wed, 31 Jul 2024 15:40:44 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f45.google.com with SMTP id a1e0cc1a2514c-8223f0614b6so1604872241.2; Wed, 31 Jul 2024 08:40:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722440443; x=1723045243; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w5ZVUkhhJHEO3QIftB/BaFGIz6i4F6Y0KvxSE98eKmE=; b=YIm3wqVODzlW7aEpqQk1yGt7KJJb9ktcDwhM3clEvb36cPFS1K1Bz+iAi9zH+9rGLU C5ZDXfkvPk0OhJRRWyudWRZQK5EGDrKpxC1l6DV3jjPfqZonHYARcgeppcZA94OFRLu+ I7C3SQFlqtZp6Ui0NHAqlRHgRApvZ6l2KEu153ma0k1ZUaFJfHszzYw5BA2QZ2hN+1kk mp23/stP8PQGZx1z5KaQ+QRcn37FAZt6kuZTpk47HncSP+iriP9FTHJzbeyyIgmKkBcJ 1u3aiCsmcfUygNZnbg2gvT1rWyEqOysPHN7qhd1t9ZRot+bzV08XE5fpl5Z+ReTdZDWf HTeQ== X-Forwarded-Encrypted: i=1; AJvYcCXvZLQbtiEdNTg1UxbcLPugwW9sYFr5AqMex7I+Z96zo8YY6f8bphL1f/SOixlKl+AklfdTf6c=@freebsd.org X-Gm-Message-State: AOJu0YxI9NAvipMWslV9bfIymsmj3BdBTbwPst+eqhDC57VRXRgGAWWH VuEBSDNs1cqH7sm1A4oOhrrOy3os+NWVUfRSqNqiZpyix5SUDqvc0UXlaZA6vBy2jIYFDi7KOk8 YZ+Jzu9ngi/symSzp2b/q2sYi8Tc= X-Google-Smtp-Source: AGHT+IGBqiVCP7PNXB55UNDJ6GyT0FaqGAxMFaiUlrbJhh/ZwSkgeOWgX1NnRE9GJG49mOnqiSGQnevKwIL8YTV6hb8= X-Received: by 2002:a05:6102:3708:b0:492:a11f:a878 with SMTP id ada2fe7eead31-493fad1fbaemr17533495137.23.1722440443147; Wed, 31 Jul 2024 08:40:43 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 31 Jul 2024 09:40:31 -0600 Message-ID: Subject: Re: The Case for Rust (in the base system) To: Shawn Webb Cc: FreeBSD Hackers , Warner Losh , Scott Long , meka@tilda.center Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4WYxBh3KwDz3wtM On Wed, Jul 31, 2024 at 8:37=E2=80=AFAM Shawn Webb wrote: > > On Sat, Jan 20, 2024 at 09:51:25AM -0700, Alan Somers wrote: > > In a recent thread on src-committers, we discussed the costs and > > benefits of including Rust code in the FreeBSD base system. To > > summarize, the cost is that it would double our build times. imp > > suggested adding an additional step after buildworld for stuff that > > requires an external toolchain. That would ease the build time pain. > > The benefit is that some tools would become easier to write, or even > > become possible. Here is a list of actual and potential Rust projects > > that could benefit from being in-tree. If anybody else has items to > > add, I suggest moving this into the project wiki: > > > > Stuff that could only be written in Rust if it were in base > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > * ctl-exporter (I started this, but discovered that the CTL stats API i= s > > unstable, so it can't live in ports. Instead, I had to do it in C). > > https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a8= 6401469181a67ec34 > > > > * fusefs tests. Absolutely impossible to do in C. I considered Rust, = but went > > with C++ so they could live in base. They are too closely coupled to > > fusefs(5) to live out-of-tree. > > https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs > > > > * devd. Currently C++, but imp suggested a rewrite. > > https://github.com/freebsd/freebsd-src/tree/main/sbin/devd > > > > * zfsd. Currently C++, but I've long pondered a rewrite. Using Rust w= ould > > make it more testable. > > https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd > > > > * nscd. Currently C, but confusing and with no test coverage. I've > > contemplated a rewrite myself, but I don't want to do it in C. > > https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd > > > > * The userland portion of the 802.11ac and Lightning stacks. scottl su= ggested > > that these were good candidates for Rust. > > > > * freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0 > > > > Stuff that can live in ports, but would be nicer in base > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D > > > > * gstat-rs https://crates.io/crates/gstat > > > > * geom-exporter (I've started this, but haven't published it) > > > > * nfs-exporter https://crates.io/crates/freebsd-nfs-exporter > > > > * virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if t= he > > connection to bhyve(8) is too intimate, it might be hard to do in por= ts. > > https://gitlab.com/virtio-fs/virtiofsd > > > > * jail-exporter https://crates.io/crates/jail_exporter > > > > * Various jail managers have been attempted in Rust. I think these are= fine in > > ports, but others like Goran Mekic have opined that they should be mo= ved to > > base instead. > > > > * musikid's pjdfstest rewrite. I think it would be great to start usin= g this > > to test the base system's file systems. If the tests themselves live= d in > > base, they would be easier to sync with file system development. > > https://github.com/musikid/pjdfstest > > > > * pf-rs. I suspect that the API isn't very stable. > > https://crates.io/crates/pf-rs > > > > * benchpmc. The pmc counter names changes between releases. > > https://crates.io/crates/benchpmc > > > > FreeBSD-related applications that are just fine in ports > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > * fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are = usually > > more stable than control path APIs, so I think there's little to be g= ained by > > moving this into base. https://crates.io/crates/fsx > > > > * ztop. It uses ZFS's kstats sysctl interface, which is pretty stable. > > https://crates.io/crates/ztop > > > > * iocage-provision https://crates.io/crates/iocage-provision > > > > * rsblk https://crates.io/crates/rsblk > > > > * xfuse https://github.com/KhaledEmaraDev/xfuse > > > > Other FreeBSD-related libraries in Rust > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Just see the list at https://crates.io/keywords/freebsd > > > > One new data point: DARPA is looking to rewrite a significant amount > of C code to Rust with their "Translating All C to Rust (TRACTOR)" > project: > https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view Interesting. And since you bring it up, I have two new data points myself: * ctld: while working on some bugs in ctld, I had trouble understanding the config file parsing. So I rewrote that part in Rust, just to help my understanding. Later, I rewrote the XML parsing, too. Then I rewrote the LUN creation and deletion, just to see how hard it would be. All of those parts take about 5x fewer SLOC in Rust than in C, and they're less buggy, too. Config file parsing is more consistent, no memory leaks, etc. Alas, I'm not planning to finish this project, since the base system doesn't allow Rust and ctld is too tightly coupled to ctl to live in ports. * geom-exporter: I mentioned this project up above. It's finished now, and you can find it in ports at net-mgmt/geom-exporter .