From owner-freebsd-hackers@freebsd.org Mon Dec 31 07:17:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E98F1429467 for ; Mon, 31 Dec 2018 07:17:23 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it1-f180.google.com (mail-it1-f180.google.com [209.85.166.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E2E48D237 for ; Mon, 31 Dec 2018 07:17:22 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it1-f180.google.com with SMTP id m62so35052857ith.5 for ; Sun, 30 Dec 2018 23:17:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=6hgFbTzJ0gkz0mWcoJ8ABeAz5QKMtOFrb05ilZzIUWQ=; b=VPjVVjab60xCTC2Hi+/qsjXHof//iCaRrJxuZwzCRfZ8PqDYyB/17HG7Yb7G3Q60Bc Zs0LrSVnJwDDFbwkJZo8OY/PDccwOtiwZAg421WUCAu/AihQg+Mb/Y6cXW/RMaF8WQ5P EPQIxA4bGjJo9M6EPyV9oFrSI+IkS0BJ0V3Oaidg3S/9A7AaTT9MI1+azheJVyUIxeoE q8Goaml8V+OWfhalmGWFb2hh9Sx+j4sft2jkcfDxCOV/UrS5BsqkZtjlzg2zsjDCWwih rjviEEOGr5xFowxJ85aWLDJco3t8nUVbf5eTjVMF20U8zGtY5CH+ANm1dNsSdfOZtERk Cx1w== X-Gm-Message-State: AA+aEWZeYs69AQOMqC+Opc0idf+Am2HJxonL/LOIOboqvmJZNpPOZsjy HHsiep6EfeWzJfldM5nuyqAil0jT X-Google-Smtp-Source: ALg8bN49IPfGHmXDNrYyoGa2K+x7/R5qglXAggbhTPZkzTWAgJ+xh2BA1MF/QLsN67j/bH7gIA6oFQ== X-Received: by 2002:a24:414c:: with SMTP id x73mr22357477ita.129.1546239382907; Sun, 30 Dec 2018 22:56:22 -0800 (PST) Received: from mail-io1-f42.google.com (mail-io1-f42.google.com. [209.85.166.42]) by smtp.gmail.com with ESMTPSA id u3sm4747827iog.67.2018.12.30.22.56.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 30 Dec 2018 22:56:22 -0800 (PST) Received: by mail-io1-f42.google.com with SMTP id r200so20807594iod.11 for ; Sun, 30 Dec 2018 22:56:22 -0800 (PST) X-Received: by 2002:a5e:8d04:: with SMTP id m4mr20559649ioj.233.1546239382432; Sun, 30 Dec 2018 22:56:22 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: Reply-To: cem@freebsd.org From: Conrad Meyer Date: Sun, 30 Dec 2018 22:56:11 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Speculative: Rust for base system components To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 9E2E48D237 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2018 07:17:23 -0000 Rust has many of the exact same problems C++ has. Sure, it avoids a few classes of issue. But it doesn't fix the classic C++ problems of extremely slow build times (if you think C++ is bad, Rust is worse) and having a kitchen sink of language features that make Rust and C++ code difficult to grok. It may be a suitable replacement for C++; it is not a suitable replacement for C. And outside of a few components (Clang and devd), most of the base system does not use C++. Adding more LLVM build time to the base system is fairly objectionable even if Rust was magical in all other aspects. Too much of the build is LLVM today. I don't think this is a viable long term plan even if Rust was ported to all of our platforms (and I don't see that happening, either). Best, Conrad On Sun, Dec 30, 2018 at 9:39 PM Eric McCorkle wrote: > > Before I begin, I want to be clear that everything here is in the realm > of speculative, long-term discussion. My goal is to start a > conversation, not to propose anything concrete right now. > > > > I've talked at several conferences about the possibility of bringing the > Rust programming language into the base system, with the intent of > making it a viable implementation language for parts of the base system. > I believe the potential security and stability benefits of doing this > are *substantial*. This would obviously be a serious undertaking, but I > think the benefits are worth the cost. For the rest of this mail, I'll > outline my position and present the important facts that support it. > > I of course welcome any discussion, criticism, or input. > > = Why Rust? = > > Rust is a true systems programming language that incorporates results > from programming language research that took place in the late 90's and > 2000's, and can be viewed as a culmination of this work. (Specifically, > it draws on work on region types as well as substructural types, but > that's its own survey article). > > This manifests most clearly in Rust's lifetime type system, which is > able to perform quite sophisticated reasoning about memory allocation > and ownership. Rust programs do not require you to explicitly manage > memory, but neither do they use garbage collection. Memory management > is automatically tracked by the lifetime types and allocations/frees are > added during compilation. Moreover, threaded programs are able to use > the same type system features to track ownership. This effectively > prevents a large class of memory- and concurrency-related errors *at the > type system level*, resulting in dramatically safer programs. Rust > provides an "unsafe" language facility for accomplishing low-level > tasks, so you're not prevented from getting things done; however, almost > all programs can be implemented without relying on unsafe, and doing so > eliminates what has probably been the two worst classes of bugs in the > history of computing. > > Because this is done at the type system level, it has no cost at > runtime. This theme of "zero-cost abstractions" runs throughout the > language, which makes it suitable as a replacement for C in the long > term. It also incorporates features befitting a modern programming > language such as parameterized types (aka generics), pattern-matching, > and trait-based polymorphism. Finally, it can interface with C/C++ > binaries and libraries relatively easily, making it a suitable systems > implementation language. > > Because of these benefits, several projects (Mozilla and Tor) have begun > to reimplement their systems using Rust. Additionally, there is a > master's thesis about writing FreeBSD device drivers using Rust out > there somewhere (forgive me for not chasing it down). > > A secondary benefit is that Rust's packaging system, cargo, is built > around a more modern concept of modularity. This is in line with > FreeBSD's goals of modularizing the base system and reducing redundancy. > > = Downsides = > > There are a couple of downsides, which I'll outline briefly, and provide > some counterargumentation for them. > > * It's a big component > > Adding the entire Rust platform would add more to the buildworld time, > which is already quite long due to LLVM. OTOH, LLVM itself is a big > chunk of the rust build time, and we already build that as part of the > base system now. Moreover, it stands to reason that the compiler would > end up being the majority of an open-source operating system's build > time, with second place being the kernel itself. Compilers are an > active area of research with no signs of slowing down, and more hardware > platforms means more backends, which means the compiler ends up growing > over time, while most of the rest of the system doesn't. > > * The language is still evolving > > While this is true, the language has gotten *way* more mature in the > past year and a half, and I think it's ready to hold up major codebases > and production use, or at the very least it will be by the time we'd be > seriously looking at bringing it in. > > * Rust binaries are large > > This issue can be avoided with the proper use of build flags. The > default settings use static linking for everything, and bring in the > entire runtime library (which isn't necessary in most cases). See this > article: > https://lifthrasiir.github.io/rustlog/why-is-a-rust-executable-large.html. > With LTO, the size can be cut down to that of a C++ binary, and with > dynamic linking, the executable size is comparable to a C program. If > we were seriously using rust, we would probably change the default flags > to use dynamic linking/LTO by default, in fact. > > > = Alternatives = > > There have been several alternatives that come up when I mention this. > I'll discuss the two big ones: go and C++ > > == C++ == > > C++ is already in the base system, of course. It could be brought into > the kernel with some effort and restrictions. The biggest argument for > Rust over C++ in my view is that Rust is a much simpler, cleaner > language. C++ at this point is bewilderingly complex as a language, and > has so many bells and whistles that it almost requires an effort to > decide which subset of the language will be used. This also tends to > slow down efforts to expand the language over time. > > Rust by comparison is much leaner. Its lifetime type system can be > intimidating, but the benefits are well worth getting used to it. To > sum it up, C++ certainly isn't going anywhere, but I think Rust has a > great deal more vitality as a language. > > == Go == > > Go is not really a "systems language" in the sense that C or Rust are. > It isn't a replacement for C so much as a replacement for python. > > What's more, it is almost universally disliked by people with a > background in programming languages (I am one of those). There's a > couple of reasons for this. First, it omits a number of features that > modern languages *really* should include, parameterized types (generics) > being one of the most glaring. Beyond that, it completely ignores > anything produced by modern PL research (unlike Rust, which benefits > from that work). Second, one of its main design parameters was "it > should compile fast". This as a design constraint basically guarantees > that you can't do anything interesting in compilation, which rules out > both advanced type systems as well as serious optimization. > > Unlike Rust, which offers substantial security and stability benefits > through its type system, Go would do little more than just change the > syntax of a bunch of code. Moreover, it's likely that there would be > performance consequences, given the design constraint of fast compilation. > > = Summary = > > Rust is a modern systems language with substantial benefits, and is > well-poised to become the heir apparent to C as the dominant systems > language going into the future. For this reason, I think we should > seriously consider it as a language for the base system at some point in > the future, when the time is right and after sufficient exploration and > experimentation. >