From nobody Sun Sep 8 15:05:00 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X1tYj38k3z5V7Xm for ; Sun, 08 Sep 2024 15:05:13 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X1tYj2bGnz4Ckc; Sun, 8 Sep 2024 15:05:13 +0000 (UTC) (envelope-from theraven@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725807913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ykRXOAd7nXvIkwG6fZQJ3jE1H9nQvPO8slsqpXd9+vU=; b=PprwZj0AKPDFrV7TmiQ3/f5ZAUtFGYcevWm4xWkYlRvGa97IYV1oVaUico5bkrw8TrPIG0 hq+2+edrZQkLGt8vTKJjxDZVh4VjCTdPnU5rxnxHh22R5EXS8+m/pYg255JkITEtTj2/UC oKyi+CergbYmRxvX6Q2V6NMFTBBavzzqcUuEReHUFYBw3F234phkNJRu4ZO1zI8oWRj2Rx /WKv96igp95076UZuOUvJsQm3EnoBYy5gUcT2H/NwDE+Z1JwlOqWA3sVDw8TZoctQ9kiMW zaLMgl8VvlgDoJ5Ti5q4YwsXUKefVr1tDy+5YBq1zsBLKuteRqHxUSdRH+YS6A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725807913; a=rsa-sha256; cv=none; b=JtWsmeY/m4eEHkwFwl0SsMQfTq/J9UAMmM2RM4dKmqgZNqo1FXnsl87B99QvUB4iu+rhiI DSJAe8bKvXKT4Ynh9742Vh3V0m1el0ywCoblemhBuE2R5Wo22coYpvlfdHd2gXBXFBi4ei +OlAnoZSt/h3en2h/xZGA+wtt6Nhd5Zcl9wmGs3r/zUryELTIMNiXP6i688n5kAfKEuH6r gkmgH0vPBmqetHvFjYV5yzoREdC2Ccf1YM431Rdgj4S3xHIB9Fg69we5madfY4xMOXvPIe 6pINisclbrSHZEQO3+CIN3U6FGQSxcbDU4xMsVOxPzKJi+IBauWW/rCO+Noisg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725807913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ykRXOAd7nXvIkwG6fZQJ3jE1H9nQvPO8slsqpXd9+vU=; b=YiPw9y4UmU/2mKnmkgmkcRNSMTTtkY839nQEcMgAK9z5ouXoATUyjAoyFbozYUsJKaIxkr J4oFMtj0oxC6Z2+/TbSa9jAXKyPcHJ1sLD/TC2UdseNXMKwBG4bAMLHiuLnOPi9P4Sm6zK LiWnawFzkRGpbhCC+3vmSkC34+9wtVH7N0vWcvyRPqpRzia8k9qmE9QpFrCnrun7tyNNVH DZ73kxwrDJAPNPJkVFEC4kaZff/FkLpR4CS+R7BLrE1eVu0cF2VANAuWkgnIJ0Qztiv9rC wY1WFK/BMmzAaQ/QBZnk0IbUO49MTxD1FKhCJhfPjW2Ckv1Ph71ropJ52qVHNA== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4X1tYj206CzSNN; Sun, 8 Sep 2024 15:05:13 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from smtpclient.apple (host109-155-136-107.range109-155.btcentralplus.com [109.155.136.107]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 7DCF36561; Sun, 08 Sep 2024 16:05:11 +0100 (BST) From: David Chisnall Message-Id: <0BC57127-5CF9-45C5-9BE6-7E21D2313291@FreeBSD.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_60EE3B51-FED3-4D06-9928-5631210681CB" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: Re: The Case for Rust (in any system) Date: Sun, 8 Sep 2024 16:05:00 +0100 In-Reply-To: Cc: Kristof Provost , Poul-Henning Kamp , Alan Somers , Dmitry Salychev , Jan Knepper , freebsd-hackers@freebsd.org To: Warner Losh References: <202409060725.4867P3ul040678@critter.freebsd.dk> <4E4FB8CC-A974-42C4-95D5-2E1E4BF681AD@freebsd.org> X-Mailer: Apple Mail (2.3776.700.51) --Apple-Mail=_60EE3B51-FED3-4D06-9928-5631210681CB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On 8 Sep 2024, at 14:50, Warner Losh wrote: >=20 > So there's four big issues with C++ in the kernel, all surmountable if = we wanted. There are two missing from your list, which I encountered when I wrote a = kernel module for FreeBSD in C++ a few years ago: C++ relies on COMDATs quite a lot. Each inline function and each = function that=E2=80=99s instantiated as a template is a separate section = with some flags indicating that the linker / loader should keep one and = discard the rest. If you have a single C++ module, this is fine, but for = two it=E2=80=99s harder. I did a small =E2=80=98libkxx=E2=80=99 module = that provided a subset of libc++ for use by different modules, but the = kernel loader code didn=E2=80=99t have enough comments for me to = understand how to fix it. I would be tempted to approach this with a = userspace tool that runs over a set of kernel modules and pulls out = duplicated COMDATs into separate modules that other things can depend = on. Alternatively, the kernel loader could be modified to load only = referenced COMDATs, reference count them, and not load unused things = from each kernel module. The latter is a cleaner approach but is more = work. Second, between 11 and 12, someone decided to replaces a load of static = inline functions in kernel headers with macros. These conflict a lot. > There's the low-level allocation issues. Right now we know what memory = is used by what because we have malloc enhanced to track this = (oversimplifying a lot I know). So we'd need some framework to make it = easy to have 'custom allocators' that could track it as well. At a bare = minimum, we need the runtime support for new and delete... This is not technically required, but it is a good idea to think about = what the right strategy is. A C++ class can implement its own `operator = new` and `operator delete` wrapping `malloc(9)` and then subclasses will = allocate with that. Similarly, things like `std::unique_ptr` can take = an explicit deleter. This can be a bit clunky and it=E2=80=99s probably a good idea to have = some sensible defaults. > Next, there's all the other run-time support that's provided by = compiler-rt. Nothing in compiler-rt is needed for C++ except the unwinder if you want = exceptions (no one else except NT uses exceptions in a kernel). The one = bit of libcxxrt that you would probably want is the support for guard = variables, which would need modifying to use kernel locks. This is = fairly small, I wrote a custom one for CHERIoT RTOS which uses our futex = APIs. > Next, there's the issues of exceptions. They are quite useful for RAII = (since you know dtors will get run when an error happens), and there'd = need to be some sane plan for these (or we'd have to forego them). Most kernels disable exceptions. You absolutely do not want = Itanium-style exceptions in a kernel because they need to allocate to = throw exceptions and so you would only be able to throw from places = where allocation is safe. Given that the most common place you=E2=80=99d = want to throw an exception (if you had them) is if `malloc` with = `M_NOWAIT` failed, this could be a problem. NT uses SEH exceptions, which allocate all of the state on the stack and = then run funclets for cleanup. It would be possible to support this in = the kernel (the relevant patents expired over ten years ago), but a = non-trivial amount of work. If someone wanted to do the work, it would = be great: SEH is one of the very few things I really liked about the NT = kernel. > Finally, there's getting the subset of the standard library that's = useful into the kernel. There's a lot of templates for facilitating RAII = that are needed, for example, and some subset of STL, etc. You don=E2=80=99t need templates for RAII, RAII just depends on = destructors. Templates are useful, but largely orthogonal. I=E2=80=99d = personally recommend against using much of the standard library in the = kernel because it does not have good ways of handling allocation failure = without exceptions. The C++ standard defines a Freestanding profile = (similar to C) that includes things like the type traits that are useful = for compile-time metaprogramming. There are a few bits you might want = to pull in but a lot more that you=E2=80=99d want to avoid (I actually = have iostream working with the kernel=E2=80=99s printf family, but it = was a terrible idea and no one should ever use that code). For example, `std::shared_ptr` is probably not a good idea (it allocates = a separate control block to hold the ref count), but something that = wraps things that are intrusively reference counted with `refcount(9)` = in smart pointers would be valuable. Using member pointers, it=E2=80=99s = easy to build a smart-pointer template that takes a C type that contains = a refcount and a pointer to the field and automatically manipulates the = reference count when you copy the pointer. > Once you have those 'table stakes' issues out of the way, you'll need = to see how it performs, and work out a few dozen logistical issues = surrounding what compiler flags to use, what language features to = enable/disable, how to optimize and what set of warnings are sensible. -fno-exceptions and -fno-rtti is what most peopls use for kernel = programming (there are a few dozen kernels written in C++, it=E2=80=99s = not like we=E2=80=99d be the first to try). > You could start using C++ with just the second of these items. You can use it within a single kernel module now, as long as you resolve = COMDATs prior to linking and #undef a bunch of things. I was doing so = five years ago. The build system actually supports it already, though = possibly not deliberately. David --Apple-Mail=_60EE3B51-FED3-4D06-9928-5631210681CB Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 On 8 Sep 2024, = at 14:50, Warner Losh <imp@bsdimp.com> wrote:

So there's four = big issues with C++ in the kernel, all surmountable if we = wanted.

There = are two missing from your list, which I encountered when I wrote a = kernel module for FreeBSD in C++ a few years = ago:

C++ relies on COMDATs quite a lot. =  Each inline function and each function that=E2=80=99s instantiated = as a template is a separate section with some flags indicating that the = linker / loader should keep one and discard the rest. If you have a = single C++ module, this is fine, but for two it=E2=80=99s harder. =  I did a small =E2=80=98libkxx=E2=80=99 module that provided a = subset of libc++ for use by different modules, but the kernel loader = code didn=E2=80=99t have enough comments for me to understand how to fix = it. I would be tempted to approach this with a userspace tool that runs = over a set of kernel modules and pulls out duplicated COMDATs into = separate modules that other things can depend on.  Alternatively, = the kernel loader could be modified to load only referenced COMDATs, = reference count them, and not load unused things from each kernel = module.  The latter is a cleaner approach but is more = work.

Second, between 11 and 12, someone = decided to replaces a load of static inline functions in kernel headers = with macros.  These conflict a lot.

There's the = low-level allocation issues. Right now we know what memory is used by = what because we have malloc enhanced to track this (oversimplifying a = lot I know). So we'd need some framework to make it easy to have 'custom = allocators' that could track it as well. At a bare minimum, we need the = runtime support for new and = delete...

This = is not technically required, but it is a good idea to think about what = the right strategy is.  A C++ class can implement its own `operator = new` and `operator delete` wrapping `malloc(9)` and then subclasses will = allocate with that.  Similarly, things like `std::unique_ptr` can = take an explicit deleter.

This can be a bit = clunky and it=E2=80=99s probably a good idea to have some sensible = defaults.

Next, there's = all the other run-time support that's provided by = compiler-rt.

Nothi= ng in compiler-rt is needed for C++ except the unwinder if you want = exceptions (no one else except NT uses exceptions in a kernel). =  The one bit of libcxxrt that you would probably want is the = support for guard variables, which would need modifying to use kernel = locks.  This is fairly small, I wrote a custom one for CHERIoT RTOS = which uses our futex APIs.

Next, there's = the issues of exceptions. They are quite useful for RAII (since you know = dtors will get run when an error happens), and there'd need to be some = sane plan for these (or we'd have to forego = them).

Most = kernels disable exceptions.  You absolutely do not want = Itanium-style exceptions in a kernel because they need to allocate to = throw exceptions and so you would only be able to throw from places = where allocation is safe.  Given that the most common place you=E2=80= =99d want to throw an exception (if you had them) is if `malloc` with = `M_NOWAIT` failed, this could be a problem.

NT = uses SEH exceptions, which allocate all of the state on the stack and = then run funclets for cleanup.  It would be possible to support = this in the kernel (the relevant patents expired over ten years ago), = but a non-trivial amount of work.  If someone wanted to do the = work, it would be great: SEH is one of the very few things I really = liked about the NT kernel.

Finally, there's = getting the subset of the standard library that's useful into the = kernel. There's a lot of templates for facilitating RAII that are = needed, for example, and some subset of STL, = etc.

You don=E2=80= =99t need templates for RAII, RAII just depends on destructors. =  Templates are useful, but largely orthogonal.  I=E2=80=99d = personally recommend against using much of the standard library in the = kernel because it does not have good ways of handling allocation failure = without exceptions.  The C++ standard defines a Freestanding = profile (similar to C) that includes things like the type traits that = are useful for compile-time metaprogramming.  There are a few bits = you might want to pull in but a lot more that you=E2=80=99d want to = avoid (I actually have iostream working with the kernel=E2=80=99s printf = family, but it was a terrible idea and no one should ever use that = code).

For example, `std::shared_ptr` is = probably not a good idea (it allocates a separate control block to hold = the ref count), but something that wraps things that are intrusively = reference counted with `refcount(9)` in smart pointers would be = valuable.  Using member pointers, it=E2=80=99s easy to build a = smart-pointer template that takes a C type that contains a refcount and = a pointer to the field and automatically manipulates the reference count = when you copy the pointer.

Once you have = those 'table stakes' issues out of the way, you'll need to see how it = performs, and work out a few dozen logistical issues surrounding what = compiler flags to use, what language features to enable/disable, how to = optimize and what set of warnings are = sensible.

-fno-exc= eptions and -fno-rtti is what most peopls use for kernel programming = (there are a few dozen kernels written in C++, it=E2=80=99s not like = we=E2=80=99d be the first to try).

You could start = using C++ with just the second of these = items.

You can = use it within a single kernel module now, as long as you resolve COMDATs = prior to linking and #undef a bunch of things.  I was doing so five = years ago.  The build system actually supports it already, though = possibly not = deliberately.

David

= --Apple-Mail=_60EE3B51-FED3-4D06-9928-5631210681CB--