From nobody Thu Aug 29 20:21:20 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wvt380rp3z52YYg for ; Thu, 29 Aug 2024 20:21:24 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wvt3736D1z4K14 for ; Thu, 29 Aug 2024 20:21:23 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=U26hCDMj; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::12c as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org Received: by mail-il1-x12c.google.com with SMTP id e9e14a558f8ab-39d2256ee95so4984295ab.2 for ; Thu, 29 Aug 2024 13:21:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1724962882; x=1725567682; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=IIekfUKm5EdZNUfxQBqlFDNeFJKz1WvP2dD5vj1ZT1A=; b=U26hCDMjlW+BrK9pmL5YkTo+UAc/5kj+op04YCCY93yVoB3/T57BiXdpm9fuqXYbtt QRmerPYGgiDx8Pf3zGn5TLWt96BgXuhMUy28FJAREfOsCNTwcDi9V94WxkIHeAv1Ci1Z wuCABWcoDewhDcrSNO1feNxlYtE9dfdPkC507RNuopMP0tr03Hf8m5jYJCJJqWSCqJic ZlMwE4SfgbI8ht5Px7ZtlGySU2H7xA4dThLn6pN2Y4k33CaVbE8Up32eFc+CsCVgyeQ6 F+ypqTj49a2gp33QL34TM4G8A+YlbGehevW3yMt0gewEhIVS/VxZoD+faelisfCnBvUj /Nsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724962882; x=1725567682; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=IIekfUKm5EdZNUfxQBqlFDNeFJKz1WvP2dD5vj1ZT1A=; b=lExTCDjDisi8rAQQtR7PtQIDXvPwPEq1SvrsMHq+mB3i5xkNF1ovBs+avBq9T9sC5G dEE/pMSD8kI8RpjSCXt2ekRzOF0NKYr4gZfI30RnslZQcNj/v5GALqJYQBbSIijcG7RV nFfUZn6+LwmBgkOV/6HgLfsdYfrP2dfTWX/BS1xOeD9ENZE0dCjbdgO4ZwDvbA/nrLdh VklqPTSYGREIPK9xvmRt9LImZdXxRPK6HZ8P+zMoPMjO5Z4RlkljMijizMyVQGktiQJm V93v48XAxXMmN35nEhdkjD2P90wyGgJMu0yQKQMlecOUAb7d2At0kSrEBHva5CbQiqRu dklA== X-Gm-Message-State: AOJu0YyG9ul3SPAStx6/aQMQpNGSNb8UX+JOoqmQaVMhFAjtbe1EP9DR A9CRQpboK96VmbTPsNcKtsJtTwavJXV8pWpOJ7syEadyuUwpb5/WtqF8bHLOnw4= X-Google-Smtp-Source: AGHT+IFTQhhGMQ1y7lAr4R7j2dCqFYzkc7GNkcbVY0ln5RM7KA8w8C9C6oSphciWqHIwGISMChEJSQ== X-Received: by 2002:a05:6e02:1a0e:b0:39b:33a8:2730 with SMTP id e9e14a558f8ab-39f377df024mr51185785ab.8.1724962881865; Thu, 29 Aug 2024 13:21:21 -0700 (PDT) Received: from mutt-hbsd (174-24-73-190.clsp.qwest.net. [174.24.73.190]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-39f3af969dfsm4636585ab.14.2024.08.29.13.21.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 13:21:21 -0700 (PDT) Date: Thu, 29 Aug 2024 20:21:20 +0000 From: Shawn Webb To: Alan Somers Cc: FreeBSD Hackers Subject: Re: A Demo of rust-in-base Message-ID: X-Operating-System: FreeBSD mutt-hbsd 15.0-CURRENT-HBSD FreeBSD 15.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="mad2rcwhuximkfrn" Content-Disposition: inline In-Reply-To: X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.09 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MISSING_XM_UA(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[hardenedbsd.org]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::12c:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; DKIM_TRACE(0.00)[hardenedbsd.org:+] X-Rspamd-Queue-Id: 4Wvt3736D1z4K14 --mad2rcwhuximkfrn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey Alan et al, I apologize for the silence on my end. It has been a busier two months than anticipated. In that time, I've been entertaining some thoughts on Rust in base support. ${LIFE} is starting to calm down again, and I do believe I'll be able to start the research in time in September. I will be splitting my free time between this and studying for my OSCP cert. So, to those thoughts, in list form (in no particular order): 1. Use of Rust compiler toolchain support will be for userland components in an opt-in fashion. Meaning, all userland components written in Rust will be optional. 2. It does not make sense to perform a vendor import of the Rust compiler toolchain and standard libraries. All Rust code in the src tree must be built from an external toolchain. 3. I believe the notion of an external toolchain could be abstracted such that we can support any optional userland component written in a language supported by that external toolchain. This would imply that other alternative languages could be supported with minimal work (Zig, TypeScript, Python, Java, etc.) 4. We could provide auto-detection mechanisms for determining which external toolchains are available, their language support, etc. The initial proof-of-concept would likely be limited to Rust to save on time and complexity. 5. As the work matures, and perhaps as a requisite for eventual inclusion, we could land support for more than Rust. This might be a step too far, but hey, it's one of the thoughts I had. 6. So all of this wrapped up means that: A. This is NOT a call to rewrite everything in Rust. This work will only permit NEW, OPTIONAL components to be written. B. Other languages/toolchains/ecosystems could be supported, not just Rust. C. Initial focus is on userland components. Rust in the kernel is out of scope for this initial proof-of-concept. D. I would like to see Rust in the kernel. That would be a good next area of focus once userland support reaches some level of maturity. My first goal will be to get a better understanding of src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also study your work, Alan. I really appreciate the time you have taken. I might reach out to you and Warner directly for further questions. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc On Sun, Aug 04, 2024 at 11:55:26AM UTC, Alan Somers wrote: > Due to all of the recent discussion of using Rust for code in the > FreeBSD base, I've put together a demo of what it might look like. It > demonstrates: >=20 > * Interspersing Rust crates through the tree (usr.bin/nfs-exporter, > cddl/usr.bin/ztop, etc) rather than in some special directory. > * Build integration for all Rust crates. You can build them all with > a single "cargo build" command from the top level, and test them all > with a single "cargo test". > * Wholly new programs written from scratch in Rust (ztop plus three > Prometheus exporters) > * Old programs rewritten in Rust with substantial new features (gstat and= fsx) > * Libs (freebsd-libgeom and freebsd-libgeom-sys) > * Commits that reconcile the dependencies of multiple crates, so as to > minimize duplicate dependency versions (5764fb383d4 and 1edf2e19e50) > * Vendoring all dependencies, direct and transitive, to ensure > internet-independent and reproducible builds (37ef9ffb6a6). This > process is automated and requires almost no manual effort. Note: > don't panic if you look in the "vendor" directory and see a bunch of > crates with "windows" in the name. They're all just empty stubs. > * All Rust object files get stored in the "target" directory rather > than /usr/obj. Today, if you want them to be stored in /usr/obj the > best way is to use a symlink, though there's WIP to add > MAKEOBJDIRPREFIX-like functionality to Cargo. >=20 > It does NOT demonstrate: >=20 > * Integrating the Rust build system with Make. Warner has some ideas > about how to do that. > * Pulling rustc into contrib. This tree requires an external Rust toolch= ain. > * Building any cdylib libraries with Rust. That's useful if you want > a C program to call a Rust library, but I don't have any good examples > for it. > * kernel modules. As already discussed, those are hard. > * Any Rust crates that involve private APIs, like CTL stuff. Those > are among the most tantalizing programs to move from ports to base, > but nobody's written any yet, because Rust-in-base doesn't exist yet. >=20 > Also, I want to address a question that's popped up a few times: > backwards-compatibility. There is a fear that Rust code needs to be > updated for each new toolchain release. But that's not true. It > hasn't been true for most crates since Rust 1.0 was released about a > decade ago. A few exotic crates required "nightly" features after > that, but they are very few in number these days, and none of them are > included in this branch's vendored sources. What Rust _does_ do is it > releases a new toolchain about every six weeks. Each new release > typically includes a few new features in the standard library and they > often add more compiler warnings, too. Sometimes they include wholly > new compiler features, but they are _always_ backwards compatible with > existing syntax. Roughly every three years, Rust publishes a new > "Edition". Rust Editions are very similar to C++ versions. i.e. Rust > 2018 is to Rust 2021 as C++14 is to C++17. New editions can include > backwards-incompatible syntax changes, but each crate always knows > which Edition it uses. Crates of different Editions can be linked > together in the same build. This branch, for example, contains crates > using Editions 2015, 2018, and 2021. >=20 > If you have any questions about what Rust in Base would look like, > please examine this branch. And if you've never used Rust before, I > highly encourage you to try it. It really is the best new > systems-program language for decades. IMHO, it's the only one that's > a compelling replacement for C++ in all new applications, and C in > most. >=20 > https://github.com/asomers/freebsd-src/tree/rust-in-base-demo >=20 --mad2rcwhuximkfrn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmbQ2C8ACgkQ/y5nonf4 4foYYw/9Hn62TRrKoAiOYER/khW8WjZR+7IKgOguGHFbA6ajMKmjrlpE/2n9srJ1 5jyV4WSuV6w5C4Ghrvf8ZBBrx/CPURXsgZ/Mk7p7edqjkpbXjgGmdwwAJeaB/jLn /VwKXe7v0KbTRxwtOeuoCNYC31kty9J1UOpWEXwdW9/qJ84k9/cdcGNOiHNlstP1 mRYUPPZQZJSk9QQ3EsjemA/TtJCZHhnyba/HlwANApaW+XO0tg0f2HAaUyALXKZw W4dvdh8wu0IeSaG4sLItYjYNoHifl3xWvgJaSkCnyuKrhwM+ez7qU6lCkGRAZ0Oq MnLPBMICLuxLPiMvzFKApi/2IUmKHVnbMari9lCv4dUFNQt01h8v5FyuNRdVQ140 x/N5CATqvXsKRSuN3TmHxmgcP8SY0qN/TR67QECyzIY6nWD9BihNWDU5L1WHe2Do o0yz8hJ9p9VUgkN006Rl86fy6iP5o2C3IdNXaWk0/t6juBJMN+Hcs++q4bbjjRwQ iArK4sCk2ehg8ygp3WEnJ/SZuZHsCC69vveOcMgfUEPFBq4K6x92WmAVJFETqc+P NtfLKw6/FnsZRl3S3mJmyxhVPJtabCx5K8lxNjWeeOxtU4IeBKkepTS7GuYK/0u4 qJg9TYJbln939bNYYpQFE0l2UUukwbUlcrCmMkw3F88BRh1lUjc= =GVJD -----END PGP SIGNATURE----- --mad2rcwhuximkfrn--