From owner-freebsd-current@freebsd.org Wed Aug 26 21:25:02 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02F859C2AE5 for ; Wed, 26 Aug 2015 21:25:02 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (mail.vlakno.cz [91.217.96.224]) by mx1.freebsd.org (Postfix) with ESMTP id A1FED141E for ; Wed, 26 Aug 2015 21:25:00 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: by vlakno.cz (Postfix, from userid 1002) id 541C91E207B3; Wed, 26 Aug 2015 23:15:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=vlakno.cz; s=mail; t=1440623713; bh=Fzf/UMyUZhCT0wgB5YgMbQ3kBrr8jf2ZK+VyP++q/dE=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=pQUSGU/YdBo8GppDppaJzCekRjq2yQ0umnELj4/TsXOs1XyQO4lb9sOBnjskKJ8oE v5AHV7s7bvCreySUx5QDxjUiZHMa7TEo6VC8qng/8yZzsspYzWjODhbTeU0Ky1+CQ9 AALNRnzGVGin17Bg5HzAcXuHylFgjukjloe7x6Rw= Date: Wed, 26 Aug 2015 23:15:13 +0200 From: Roman Divacky To: Marius Strobl Cc: Kurt Lidl , freebsd-current@freebsd.org Subject: Re: HEADS UP: sparc64 backend for llvm/clang imported Message-ID: <20150826211513.GA65424@vlakno.cz> References: <20140301011643.GV47921@funkthat.com> <55D4E4B7.1000806@pix.net> <20150826211451.GA22113@alchemy.franken.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150826211451.GA22113@alchemy.franken.de> User-Agent: Mutt/1.5.23 (2014-03-12) X-Mailman-Approved-At: Wed, 26 Aug 2015 21:58:08 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Aug 2015 21:25:02 -0000 On Wed, Aug 26, 2015 at 11:14:51PM +0200, Marius Strobl wrote: > On Wed, Aug 19, 2015 at 04:19:03PM -0400, Kurt Lidl wrote: > > > Dimitry Andric wrote this message on Fri, Feb 28, 2014 at 20:22 +0100: > > >> In r262613 I have merged the clang-sparc64 branch back to head. This > > >> imports an updated sparc64 backend for llvm and clang, allowing clang to > > >> bootstrap itself on sparc64, and to completely build world. To be able > > >> to build the GENERIC kernel, there is still one patch to be finalized, > > >> see below. > > >> > > >> If you have any sparc64 hardware, and are not afraid to encounter rough > > >> edges, please try out building and running your system with clang. To > > >> do so, update to at least r262613, and enable the following options in > > >> e.g. src.conf, or in your build environment: > > >> > > >> WITH_CLANG=y > > >> WITH_CLANG_IS_CC=y > > >> WITH_LIBCPLUSPLUS=y (optional) > > >> > > >> Alternatively, if you would rather keep gcc as /usr/bin/cc for the > > >> moment, build world using just WITH_CLANG, enabling clang to be built > > >> (by gcc) and installed. After installworld, you can then set CC=clang, > > >> CXX=clang++ and CPP=clang-cpp for building another world. > > >> > > >> For building the sparc64 kernel, there is one open issue left, which is > > >> that sys/sparc64/include/pcpu.h uses global register variables, and this > > >> is not supported by clang. A preliminary patch for this is attached, > > >> but it may or may not blow up your system, please beware! > > >> > > >> The patch changes the pcpu and curpcb global register variables into > > >> inline functions, similar to what is done on other architectures. > > >> However, the current approach is not optimal, and the emitted code is > > >> slightly different from what gcc outputs. Any improvements to this > > >> patch are greatly appreciated! > > >> > > >> Last but not least, thanks go out to Roman Divacky for his work with > > >> llvm/clang upstream in getting the sparc64 backend into shape. > > > > > > Ok, I have a new pcpu patch to try. I have only compile tested it. > > > > > > It is available here: > > > https://www.funkthat.com/~jmg/sparc64.pcpu.patch > > > > > > I've also attached it. > > > > > > Craig, do you mind testing it? > > > > > > This patch also removes curpcb as it appears to not be used by any > > > sparc64 C code. A GENERIC kernel compiles fine, and fxr only turns up > > > curpcb used in machdep code, and no references to it under sparc64. > > > > > > This is not a proper solution in that > > > it can mean counters/stats can be copied/moved to other cpus overwriting > > > the previous values if a race happens... We use > > > PCPU_SET(mem, PCPU_GET(mem) + val) for PCPU_ADD, not great, but it's > > > no worse than what we were previously using.. > > > > > > Until we get a proper fix which involves mapping all the cpu's PCPU > > > data on all CPUs, this will have to sufice.. > > > > > > This patch is based upon, I believe, a patch from Marius and possibly > > > modified by rdivacky. > > > > > > Thanks for testing.. > > > > The above message was posted a while ago, and I decided that I would > > give the patch a test run on a spare sparc that I have, now that the > > instability problem with multiprocessor sparc64 machines has been > > resolved. > > > > So, I have an up-to-date stable/10 V240 (2x1.5Ghz cpus, 8GB of memory), > > running a completely stock r286861. That all seems to work just fine. > > > > I applied the patch referenced in the email: > > > > https://www.funkthat.com/~jmg/sparc64.pcpu.patch > > > > (it applied cleanly), and then rebuilt the kernel on the machine, > > using the stock gcc 4.2.1 compiler. > > > > When rebooting with that kernel, the machine panics pretty early > > in the boot: > > > > FreeBSD 10.2-STABLE #3 r286861M: Wed Aug 19 14:28:45 EDT 2015 > > lidl@spork.pix.net:/usr/obj/usr/src/sys/GENERIC sparc64 > > gcc version 4.2.1 20070831 patched [FreeBSD] > > real memory = 8589934592 (8192 MB) > > avail memory = 8379719680 (7991 MB) > > cpu0: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU) > > cpu1: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU) > > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > > random device not loaded; using insecure entropy > > panic: trap: illegal instruction (kernel) > > cpuid = 0 > > KDB: stack backtrace: > > #0 0xc05750e0 at panic+0x20 > > #1 0xc08db9f8 at trap+0x558 > > Uptime: 1s > > Automatic reboot in 15 seconds - press a key on the console to abort > > Rebooting... > > timeout shutting down CPUs. > > > > So, the patch to get rid of the pcpu usage (as a prereq to poking > > at the clang compiler issues) does not work properly. > > > > As I pointed out when that patch was posted, the approach taken by > it assumes a CPU can access foreign PCPU data, which currently isn't > true on sparc64. So the patch is at least incomplete but also may > have further issues. > > Such a patch is no longer a prerequisite for compiling a sparc64 > kernel with clang, though, as clang meanwhile has been told to > grok at least the global registers used by the PCPU code. > > Besides some default options like the choice of code model not > being appropriate for FreeBSD, clang-compiled loader and kernel > don't work due to two major problems present in clang up to at > least version 3.6.0: a) it uses a different stack layout than > GCC so any unwinding code fails and b) it produces broken code > for recursive function calls. YMMV. Is there a bug reported upstream about the recursive function calls? Roman