From owner-freebsd-hackers@freebsd.org Mon Dec 31 16:52:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 620DB1439CF3 for ; Mon, 31 Dec 2018 16:52:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46F6B736AE; Mon, 31 Dec 2018 16:52:52 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-f171.google.com with SMTP id k15-v6so23924018ljc.8; Mon, 31 Dec 2018 08:52:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=4dKYm4jBHIPoM6b4mqf/OVRDEgVFKnrA57y3h94CNC4=; b=dV/irQPxKIgc+Y1HdaGfvm4l3rSP/Jwpf+AzrwY6y6ohB2sa797PTCQ3ku/Gw8wi2B WL0BSrSLAVXW5LGMVD3mFPGTIoZZxB7znwTuRI0AFzmi49DnV5CDYlv8so69ZYEKU/UG ykygXV56rbB2dwel9Z8T3QmHQMUCUF3G+GPAMi4kr14JOl58MOt99qPt3miSEu8Sp/UF duQ88tw7hdpqpqGKRxBetzQtwpYd7Iwlpv7i/rKBDbJqtgiyXhonhmKlMl3YxEQCUelb wOXXjC6/fmvnocx3vnLM35s/LcJKspSrFSRKyI4AjVHiRsLywfvlvdztyrIvfhhsSEvB Y62A== X-Gm-Message-State: AJcUukcf/evqwilEVVYB9C9iouaGu9I84SAjawKSaMGUK7ZbAxptpJnF TACZE2TkX2uHNqJ3oR4Ug7hZld++ZUgrqXLxF5TrgQ== X-Google-Smtp-Source: ALg8bN7x43CkzZylgLMiVZ6QGB5APjd9/nMrYSSicP8MDLbOti54XqBs+L+uEjTcPgjYkslwIDKNFdLZPLgmPjFnrzs= X-Received: by 2002:a2e:20c3:: with SMTP id g64-v6mr23858619lji.101.1546275164470; Mon, 31 Dec 2018 08:52:44 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Mon, 31 Dec 2018 09:52:32 -0700 Message-ID: Subject: Re: Speculative: Rust for base system components To: "Conrad E. Meyer" Cc: Eric McCorkle , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46F6B736AE X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.208.171 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.84 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-0.99)[-0.994,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.76)[-0.760,0]; RCVD_IN_DNSWL_NONE(0.00)[171.208.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.08)[ipnet: 209.85.128.0/17(-3.76), asn: 15169(-1.55), country: US(-0.08)]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[171.208.85.209.rep.mailspike.net : 127.0.0.17]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2018 16:52:53 -0000 On Mon, Dec 31, 2018 at 12:17 AM Conrad Meyer wrote: > > Rust has many of the exact same problems C++ has. Sure, it avoids a > few classes of issue. But it doesn't fix the classic C++ problems of > extremely slow build times (if you think C++ is bad, Rust is worse) > and having a kitchen sink of language features that make Rust and C++ > code difficult to grok. Do you have any data to support the claim of slow build times? I know it's very hard to do an apples-to-apples comparison of build times between different languages, but I would be interested to see somebody try. > > It may be a suitable replacement for C++; it is not a suitable > replacement for C. And outside of a few components (Clang and devd), > most of the base system does not use C++. > > Adding more LLVM build time to the base system is fairly objectionable > even if Rust was magical in all other aspects. Too much of the build > is LLVM today. > > I don't think this is a viable long term plan even if Rust was ported > to all of our platforms (and I don't see that happening, either). > > Best, > Conrad > > > On Sun, Dec 30, 2018 at 9:39 PM Eric McCorkle wrote: > > > > Before I begin, I want to be clear that everything here is in the realm > > of speculative, long-term discussion. My goal is to start a > > conversation, not to propose anything concrete right now. > > > > > > > > I've talked at several conferences about the possibility of bringing the > > Rust programming language into the base system, with the intent of > > making it a viable implementation language for parts of the base system. > > I believe the potential security and stability benefits of doing this > > are *substantial*. This would obviously be a serious undertaking, but I > > think the benefits are worth the cost. For the rest of this mail, I'll > > outline my position and present the important facts that support it. > > > > I of course welcome any discussion, criticism, or input. > > > > = Why Rust? = > > > > Rust is a true systems programming language that incorporates results > > from programming language research that took place in the late 90's and > > 2000's, and can be viewed as a culmination of this work. (Specifically, > > it draws on work on region types as well as substructural types, but > > that's its own survey article). > > > > This manifests most clearly in Rust's lifetime type system, which is > > able to perform quite sophisticated reasoning about memory allocation > > and ownership. Rust programs do not require you to explicitly manage > > memory, but neither do they use garbage collection. Memory management > > is automatically tracked by the lifetime types and allocations/frees are > > added during compilation. Moreover, threaded programs are able to use > > the same type system features to track ownership. This effectively > > prevents a large class of memory- and concurrency-related errors *at the > > type system level*, resulting in dramatically safer programs. Rust > > provides an "unsafe" language facility for accomplishing low-level > > tasks, so you're not prevented from getting things done; however, almost > > all programs can be implemented without relying on unsafe, and doing so > > eliminates what has probably been the two worst classes of bugs in the > > history of computing. > > > > Because this is done at the type system level, it has no cost at > > runtime. This theme of "zero-cost abstractions" runs throughout the > > language, which makes it suitable as a replacement for C in the long > > term. It also incorporates features befitting a modern programming > > language such as parameterized types (aka generics), pattern-matching, > > and trait-based polymorphism. Finally, it can interface with C/C++ > > binaries and libraries relatively easily, making it a suitable systems > > implementation language. > > > > Because of these benefits, several projects (Mozilla and Tor) have begun > > to reimplement their systems using Rust. Additionally, there is a > > master's thesis about writing FreeBSD device drivers using Rust out > > there somewhere (forgive me for not chasing it down). > > > > A secondary benefit is that Rust's packaging system, cargo, is built > > around a more modern concept of modularity. This is in line with > > FreeBSD's goals of modularizing the base system and reducing redundancy. > > > > = Downsides = > > > > There are a couple of downsides, which I'll outline briefly, and provide > > some counterargumentation for them. > > > > * It's a big component > > > > Adding the entire Rust platform would add more to the buildworld time, > > which is already quite long due to LLVM. OTOH, LLVM itself is a big > > chunk of the rust build time, and we already build that as part of the > > base system now. Moreover, it stands to reason that the compiler would > > end up being the majority of an open-source operating system's build > > time, with second place being the kernel itself. Compilers are an > > active area of research with no signs of slowing down, and more hardware > > platforms means more backends, which means the compiler ends up growing > > over time, while most of the rest of the system doesn't. > > > > * The language is still evolving > > > > While this is true, the language has gotten *way* more mature in the > > past year and a half, and I think it's ready to hold up major codebases > > and production use, or at the very least it will be by the time we'd be > > seriously looking at bringing it in. > > > > * Rust binaries are large > > > > This issue can be avoided with the proper use of build flags. The > > default settings use static linking for everything, and bring in the > > entire runtime library (which isn't necessary in most cases). See this > > article: > > https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html. > > With LTO, the size can be cut down to that of a C++ binary, and with > > dynamic linking, the executable size is comparable to a C program. If > > we were seriously using rust, we would probably change the default flags > > to use dynamic linking/LTO by default, in fact. > > > > > > = Alternatives = > > > > There have been several alternatives that come up when I mention this. > > I'll discuss the two big ones: go and C++ > > > > == C++ == > > > > C++ is already in the base system, of course. It could be brought into > > the kernel with some effort and restrictions. The biggest argument for > > Rust over C++ in my view is that Rust is a much simpler, cleaner > > language. C++ at this point is bewilderingly complex as a language, and > > has so many bells and whistles that it almost requires an effort to > > decide which subset of the language will be used. This also tends to > > slow down efforts to expand the language over time. > > > > Rust by comparison is much leaner. Its lifetime type system can be > > intimidating, but the benefits are well worth getting used to it. To > > sum it up, C++ certainly isn't going anywhere, but I think Rust has a > > great deal more vitality as a language. > > > > == Go == > > > > Go is not really a "systems language" in the sense that C or Rust are. > > It isn't a replacement for C so much as a replacement for python. > > > > What's more, it is almost universally disliked by people with a > > background in programming languages (I am one of those). There's a > > couple of reasons for this. First, it omits a number of features that > > modern languages *really* should include, parameterized types (generics) > > being one of the most glaring. Beyond that, it completely ignores > > anything produced by modern PL research (unlike Rust, which benefits > > from that work). Second, one of its main design parameters was "it > > should compile fast". This as a design constraint basically guarantees > > that you can't do anything interesting in compilation, which rules out > > both advanced type systems as well as serious optimization. > > > > Unlike Rust, which offers substantial security and stability benefits > > through its type system, Go would do little more than just change the > > syntax of a bunch of code. Moreover, it's likely that there would be > > performance consequences, given the design constraint of fast compilation. > > > > = Summary = > > > > Rust is a modern systems language with substantial benefits, and is > > well-poised to become the heir apparent to C as the dominant systems > > language going into the future. For this reason, I think we should > > seriously consider it as a language for the base system at some point in > > the future, when the time is right and after sufficient exploration and > > experimentation. > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"