From owner-freebsd-arch@freebsd.org Tue Oct 10 13:50:51 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B806E325A0 for ; Tue, 10 Oct 2017 13:50:51 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2C80B82FDE; Tue, 10 Oct 2017 13:50:51 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id EE12010A8BC; Tue, 10 Oct 2017 09:50:49 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Cc: Nathan Whitehorn Subject: Re: Making C++11 a hard requirement for FreeBSD Date: Tue, 10 Oct 2017 06:49:44 -0700 Message-ID: <1723616.qgAo6xlX2y@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <577d3900-76f2-2c52-8ada-b8fb1fe881be@freebsd.org> References: <577d3900-76f2-2c52-8ada-b8fb1fe881be@freebsd.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Tue, 10 Oct 2017 09:50:50 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2017 13:50:51 -0000 On Monday, October 09, 2017 09:57:04 PM Nathan Whitehorn wrote: > > On 10/09/17 11:32, Warner Losh wrote: > > On Oct 9, 2017 11:47 AM, "Nathan Whitehorn" wrote: > > > > > > > > On 10/08/17 22:26, Warner Losh wrote: > > > > > > > > On Sun, Oct 8, 2017 at 11:01 PM, Nathan Whitehorn > > wrote: > > > >> > >> On 10/06/17 00:20, Baptiste Daroussin wrote: > >> > >>> On Thu, Oct 05, 2017 at 11:47:57PM +0000, John Baldwin wrote: > >>> > >>>> On Thursday, October 05, 2017 04:28:44 PM Warner Losh wrote: > >>>> > >>>>> I'd like to start a conversation about the viability of making C++11 a > >>>>> hard > >>>>> requirement for bootstrapping FreeBSD and setting a specific deadline > >>>>> for > >>>>> doing so. > >>>>> > >>>>> This discussion is motivated by an ask from the jemalloc folks to use a > >>>>> limited subset of C++11 inside of malloc in such a way that is C safe > >>>>> (so > >>>>> the C programs wouldn't bloat with a C++ runtime). That's an ongoing > >>>>> discussion in another forum, and isn't appropriate for this thread > >>>>> because > >>>>> this has become a frequent request (but if you have opinions, please > >>>>> find > >>>>> the thread in current@ about it). I don't know the timeline of their > >>>>> plans > >>>>> to do this. > >>>>> > >>>>> I'd like to take the rather vague plans we've had "before 12" and > >>>>> create a > >>>>> timeline for removal of gcc 4.2 coupled with a requirement for support > >>>>> in > >>>>> clang or a working external toolchain. This requirement would be coupled > >>>>> with the requirement that the external toolchain support C++11 > >>>>> constructs. > >>>>> > >>>>> I'd like to propose we do this 12/31/17. Any architectures that can't > >>>>> meet > >>>>> this timeline will be disconnected from universe at that time and > >>>>> deleted > >>>>> 3/31/18. > >>>>> > >>>>> It's my belief that i386, amd64, arm, aarch64, powerpc and powerpc64 are > >>>>> ready for this change and mips* would be ready for this change with an > >>>>> external clang toolchain. I'm unsure of riscv and sparc64, but suspect > >>>>> that > >>>>> a newer version of gcc as an external toolchain could work. > >>>>> > >>>> In-tree clang 5.0 for MIPS mostly works (modulo some small patches). > >>>> However, > >>>> it requires external ld.bfd. I know that there ld.lld can link a working > >>>> mips64 world with some patches (notably the multigot patch). mips works > >>>> fine > >>>> with external GCC. riscv is already using external GCC that is > >>>> C++11-capable. > >>>> > >>>> The problem with external GCC is that you can cross-build a world + > >>>> kernel > >>>> just fine and get a working system via CROSS_TOOLCHAIN=foo-gcc. However, > >>>> that system has no viable '/usr/bin/cc' once GCC 4.2 is removed. bapt@ > >>>> started on ports to cross-build binutils and gcc packages via the base/* > >>>> ports, but those are not yet finished / fully tested. I don't think > >>>> anyone > >>>> has thought about how release builds will work either with only an > >>>> external > >>>> toolchain available. (I'm pretty sure sparc64 has booted fine with > >>>> external GCC, it's just in the same boat as mips with regard to > >>>> /usr/bin/cc.) > >>>> > >>> Actually I did test those and they were working (tested in qemu) they were > >>> working fine. I have given up working on them due to the lack of > >>> interested by > >>> the community (by interest I mean people really testing, working on it, > >>> not just > >>> saying "hey nice sounds cool"). > >>> > >>> As for the boot when I initially worked on external toolchain sparc64 was > >>> my > >>> guinea pig and so yes it worked an booted just fine. > >>> > >> So far as I know, we never solved any of the infrastructural problems > >> associated with this concept: > >> 1. Providing built releases with a /usr/bin/cc > >> 2. Coversioning base and in-ports toolchain, including ensuring commit > >> atomicity between toolchains and libc > >> 3. Adding a dependency on ports for src, including out-of-tree code that > >> has to be fetched from external servers > >> 4. Getting make universe to do the right thing > >> > >> We really need to solve those. If we go the external toolchain route, > >> which it is not clear to me is the best option, #2 and #1 are quite complex > >> problems. I think that, frankly, a deadline in two months to solve this set > >> of political problems we have had for two years is probably crazy, but > >> maybe making the status quo unsustainable will finally force progress. > >> > > External toolchains have been in flight for 5 or more years now. It's time > > they land. Though the requirements for them have never included > > cross-threading between /usr/src and /usr/ports like you suggest above, and > > those sorts of things won't be sorted by my deadlines (which are closer to > > 3 months). Nor, imho, should they. > > > > > > Well, sure. But the fact remains that we cannot build usable systems with > > external toolchains right now. Those are real problems that need to be > > solved somehow. > > > > > > Sure we can. I've built a bootable i386 system with gcc 6. It is a solved > > problem. > > "Bootable" is not the same as "usable" here. External toolchain > powerpc64 *boots* fine -- and has for years -- but there isn't much you > can actual *do* with the system except admire dmesg without the ability > to install ports. > > > Let's focus on #1, the largest if not the only major problem. If I build, > > say, a ppc64 system with an external toolchain right now, it boots and runs > > fine. But the system is completely unusable since: > > - There are no binary packages built for PPC64, because of project policy > > preventing the use of native build systems > > > > > > System is still usable w/o packages. People can still fire up custom > > poudrier repos. We could also change project policy. This is is a specific > > wrinkle for powerpc64 it seems. > > How would I do that? We can't run these natively, because the system > ships without a compiler. And we can't cross-compile, because the ports > tree does not support that. The base/ ports _do_ cross-compile. See /usr/ports/base/README. You have to build a world image using the external toolchain that can then be used as a --sysroot with the external toolchain to build binutils and gcc packages. These packages should then be able to be installed. To be clear, you would build a world on an amd64 host with the foo-xtoolchain-gcc toolchain, install it to some 'rootfs' directory, then build the base/ packages on the same amd64 host but the generated packages can be installed on the target via pkg install. In particular, we could automate building of the base/ packages in the cluster and publish repositories that contain just binutils and gcc. -- John Baldwin