From owner-freebsd-sparc64@freebsd.org Sun Nov 8 15:55:12 2015 Return-Path: Delivered-To: freebsd-sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1793A29C4E for ; Sun, 8 Nov 2015 15:55:12 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 857B910AE for ; Sun, 8 Nov 2015 15:55:12 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: by mailman.ysv.freebsd.org (Postfix) id 82459A29C4B; Sun, 8 Nov 2015 15:55:12 +0000 (UTC) Delivered-To: sparc64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7FBEBA29C4A; Sun, 8 Nov 2015 15:55:12 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "alchemy.franken.de", Issuer "alchemy.franken.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D8BA10AB; Sun, 8 Nov 2015 15:55:11 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.15.2/8.15.2/ALCHEMY.FRANKEN.DE) with ESMTPS id tA8Ft1mu049757 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 8 Nov 2015 16:55:02 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.15.2/8.15.2/Submit) id tA8Ft1wx049756; Sun, 8 Nov 2015 16:55:01 +0100 (CET) (envelope-from marius) Date: Sun, 8 Nov 2015 16:55:01 +0100 From: Marius Strobl To: Warner Losh Cc: sbruno@freebsd.org, freebsd-arch , sparc64@freebsd.org Subject: Re: Sparc64 doesn't care about you, and you shouldn't care about Sparc64 Message-ID: <20151108155501.GA1901@alchemy.franken.de> References: <563A5893.1030607@freebsd.org> <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2AAC0EF3-528B-476F-BA9C-CDC3004465D0@bsdimp.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (alchemy.franken.de [0.0.0.0]); Sun, 08 Nov 2015 16:55:02 +0100 (CET) X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Nov 2015 15:55:12 -0000 On Wed, Nov 04, 2015 at 04:19:38PM -0700, Warner Losh wrote: > > > On Nov 4, 2015, at 12:12 PM, Sean Bruno wrote: > > > > So here's the thing, Sparc64 is *just* barely alive in FreeBSD. > > Has anybody actually booted it off a newish tree? Yes, just works fine there as of r290419. root@v215:~ # uname -a FreeBSD v215.zeist.de 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r290419: Thu Nov 5 23:01:37 CET 2015 marius@v215.zeist.de:/tmp/obj/usr/src/sys/GENERIC sparc64 > > There is exactly 1 Sparc64 machine as a ref box being hosted at Yahoo > > for the project. No new hardware is on the horizon. Over time, linimon@ and me independently tried to get more sparc64 machines there but failed. As far as I was involved the goal was to get hardware being offered for donation in the US there but it turned out that's it's just impossible to coordinate that from over here. However, I've been told that the primary reason for getting rid of the collocation at Yahoo is to improve the situation with on-site maintenance. So getting sparc64 hosts set up it the project cluster may actually be feasible now. I probably should check with glen@. Anyway, how many arm, mips, powerpc and powerpc64 reference machines are there in the project cluster and available for general use by developers? If there were, what would you do with them? According to my experience, the very few !x86 machines that actually are in there aren't really useful without root access. > > None of the newer > > Sparc64 processors have been tested to work on FreeBSD and nobody is > > clamoring to get them working. Unlike x86, sun4u and sun4v CPUs are only designed to be backwards compatible as far as the userland is concerned, host-PCI{,e}-bridges aren't compatible etc. So the kernel side always needs work and it simply isn't a matter of "testing" newer CPUs and models to work. Thus, the hardware is needed for developing support, see also below. I'm not sure at which point I'd speak of people "clamoring" for support of some hardware; f. e. there also isn't a request for the graphics of Haswell and later Intel CPUs to be supported on the mailing lists every other day but you'll certainly agree that many are waiting for it. Now there likely are fewer people looking forward for later sun4u and sun4v processors getting supported but there definitely are people asking for it, just have a look at the lists. > > We're moving into a post-gcc base system now, and sparc64 is the obvious > > "odd arch" here. There's activity to get MIPS moved to clang and active > > work to get powerpc moved fully to clang. Leaving Sparc64 in base, > > requires someone to either make clang DTRT or keep gcc 4.2.1-ish alive. I don't see why sparc64 would be "the obvious odd arch" in that regard. The real problem is switching an architecture for which clang might have gotten en par with GCC after clang was changed to require C++11 for bootstrap. Given that clang was only the default on arm and x86 at that point in time, we are now stuck without an in-tree upgrade path on all other architectures. Granted, that might be lesser a problem on mips as these machines typically don't have enough CPU and RAM that self-hosting would be interesting in the first place. That still puts sparc64 into the same boat as powerpc and powerpc64, though. > There was some work to get clang to do the right thing for sparc64. Last > I heard, the tree compiles with it. It didn?t boot, but at the time gcc-compiled > kernels didn?t boot either. I?m not sure how this status has moved through time. > It would be best to ask Marius Strobl, since he?s the only one committing > to sparc64 sub-tree lately non-global-sweep cleanups. Err, I'm sorry to say but pretty much all of these statements are wrong or at least don't reflect reality properly. As for clang, initial support for 64-bit SPARC v9 and FreeBSD/sparc64 was added in the late 3.5.x times (I don't remember the exact version offhand). Yes, it does compile FreeBSD userland but the majority of binaries do not work or put differently: For some strange reason a clang compiled sort(1) doesn't crash _and_ apparently does the right thing but I wasn't able to find any other program that worked correctly when compiled with clang. By installing clang-compiled system libraries one even totally hoses ones system, i. e. then also sort(1) instantly crashes. With clang 3.6.0 that got even worse; the front-end was changed to select soft-float by default but the back-end didn't have support for choosing between hard- and soft-float for 64-bit SPARC v9 so every compiler invocation just bailed out. As a side note: I couldn't spot any code that would suggest that hard-float support is implemented, which should be the actual default, though. I fixed that (the soft- float defaulting that is, not implementing hard-float support) and some other astonishing bugs (or rather: hacked some things into shape), which got some more things working and IIRC even a clang- compiled kernel got considerably further. However, I didn't get anywhere near the bottom of bugs and missing features. Generally, clang on sparc64 gives more of an impression that it never has faced a compiler test-suite. I haven't given clang 3.7.0 a try thus far but the release notes and source changes don't suggest much has improved in that regard since then. Also, for me a toolchain is just what the name implies: A tool. If it isn't sufficiently close to 100 % working I just use another one. Regarding GCC-compiled kernles not working on sparc64 at that point I'm not aware of such a problem in that time frame or generally of a sparc64-specific breakage in the last couple of years (there was one in userland a couple of months ago, though). I mean, sure, the kernel build gets broken regularly as people don't even compile-test their changes and sometimes that also results in run-time problems but that certainly isn't what you are referring to. However, I can't think of a sparc64 kernel breakage that got beyond that or even was sparc64-specific. What you _might_ be thinking of in that regard is the following: When 64-bit SPARC v9 and FreeBSD/sparc64 support initially were added, clang couldn't cope with code in the sparc64 pcpu.h. There was a patch by jmg@ floating around which changed pcpu.h so that clang could compile a kernel. However, that patch conceptually is wrong and results in a non-working kernel even with GCC. Meanwhile, clang has been taught to grok at least the code in pcpu.h (but not generally to deal with the pattern in question), so at least that situation has improved and said patch is obsolete. As for getting forward, the FreeBSD Software License Policy (https://www.freebsd.org/internal/software-license.html) specifically allows for existing GPLv2 licensed software in the FreeBSD source tree to be updated to GPLv3 licensed one. The initial, longer draft of this policy posted by brooks@ to developers@ even explicitly mentioned key technologies such as toolchains of other licenses being allowed when no mature BSD-licensed alternative exists. So I propose just that: Let's upgrade binutils and GCC in base to recent versions. Seriously. That way we 1) don't need to get external toolchain support into shape, 2) don't need to solve the chicken-and-egg problem of getting a toolchain onto a machine installed from a distribution built with an external toolchain and 3) once clang becomes mature on additional architectures, we have an upgrade path. Don't get me wrong, I'm only proposing that for !arm and !x86. As a side note: A while back I talked to grehan@ and marcel@ regarding the immaturities of clang and - as expected -, a GPL'ed toolchain just is no problem for either NetApp or Juniper as the binaries they ship don't include the toolchain itself. With the possible exception of the current incarnation of SCO which apparently sells a FreeBSD-based OS likely having a system compiler, for the same reason I can't think of why a GPLv3 licensed toolchain would matter for any of the commercial downstream consumers of FreeBSD. Thus, I really can't understand all that aggression regarding making FreeBSD 11 clang-only. > Here?s a breakdown of commits in different parts of sys. The ?Marius? column > is for commits Marius has made in sparc64 only. The rest are the different > architectures we currently support. I wrote this with mail.app, so formatting > may be dicy. > > Year Marius sparc64 mips arm powerpc i386 amd64 x86 arm64 > 2015 5 32 164 445 144 168 247 109 168 > 2014 0 39 117 672 98 125 296 108 - > 2013 14 65 235 455 217 142 235 67 - > 2012 24 55 272 343 152 188 221 76 - > 2011 78 131 205 105 172 189 182 56 - > 2010 75 127 501 103 211 274 268 75 - > 2009 58 95 269 193 137 293 258 - - > 2008 65 109 65 167 161 304 222 - - > > sparc64 rate of change has fallen way off since 2011, both in terms of the > number of commits, as well as the share of commits relative to other > platforms. While I know that not all commits are treated equally, and that > different commit styles in different parts of the tree may skew things, Not only because of that incomplete last sentence it remains unclear what you exactly would like to express with that numbers. There are quite a few reasons why they look the way they do, though. First off, FreeBSD/sparc64 is rock-solid, comparable to x86 in that regard, so it just doesn't need constant changing. Moreover, for a fair comparison you'd generally need to filter f. e. board and peripheral support from sys/arm and sys/mips as sys/sparc64 doesn't have something like the former and drivers for MACs, USB controllers etc. and even the uart(4) bus front-ends live in sys/dev for sparc64. Similarly, you'd need to exclude bhyve- and XEN-related commits from sys/{amd64,i386,x86}. As a side note: Sure, sparc64 isn't exactly en par with x86 when it comes to feature parity. However, IMO overall sparc64 competes rather well with arm and mips in that regard or actually is somewhat better. For example, neither arm nor mips have been converted to NEW_PCIB nor do they provide GET_STACK_USAGE while sparc64 has both. However, one reason certainly also is that some time ago I've decided that I had spent enough money on hardware for working on FreeBSD that I'd otherwise would not have bought (which mainly was sparc64 gear but certainly not limited to that) and rely on donations instead. That didn't turn out to work too well, though. A general problem in that regard are shipping costs and taxes when stuff is located outside the Schengen Area. Also, I've f. e. applied for the M5000 and T2000 machines that where up for donation last August in order to complete/ add support for newer CPUs (support for SPARC64-VI/VII and peripherals in models based on them is mostly there but disabled as I couldn't test it). Unfortunately, I haven't heard anything back regarding these machines. However, that also means: Whoever got that equipment could justify even better use than me, although I could point to the code I had written for that gear. > > I have asked around for help getting the Sparc64 qemu-bsd-user binary > > working so I could at a minimum build packages, and I have gotten no > > feedback from folks. So the only option here is to resurrect sparc64 > > machines somewhere and start up builds on real hardware. It would be more tempting to look at that if the 64-bit SPARC v9 support of QEMU generally would be up to speed but it isn't for years. QEMU seemed to have made some progress in that regard in current versions but it still just isn't there, yet; see the recent attempts on freebsd-sparc64@ to get some of the bugs fixed. That isn't a SPARC-specific lack of interest or problem, though. At work I do microkernel-based R&D on ARM and for that QEMU just isn't usable either; it considerably lacks emulation of viable, real ARMv7-based boards, specifically Cortex-A7-based ones, its GIC emulation is foobar etc. Not that it would help me anything, but QEMU doesn't even provide targets of something as popular as the Raspberry Pi boards. Go figure. As for sun4u system emulation, Simics worked okay for me. That was before Wind River bought it, though. Regarding real sun4u hardware, over time a considerable number of machines has been donated for ports work AFAICT. > > Let's just call it what it is, a dead end of the technology tree. > > I move that we do NOT produce 11.0 versions for Sparc64 and it should be > > dropped from the tree. > > I concur. I think sparc64 has had a nice run, but it?s time to recognize > that the run is nearing its end. > Do whatever you want, I'm tired of mere political arguments and of fighting against them. Marius