From owner-freebsd-toolchain@freebsd.org Wed May 24 11:44:13 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4C02D7B51F; Wed, 24 May 2017 11:44:13 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (mail.vlakno.cz [91.217.96.224]) by mx1.freebsd.org (Postfix) with ESMTP id B2202105A; Wed, 24 May 2017 11:44:13 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: by vlakno.cz (Postfix, from userid 1002) id D842E1C7FF01; Wed, 24 May 2017 13:34:41 +0200 (CEST) Date: Wed, 24 May 2017 13:34:41 +0200 From: Roman Divacky To: freebsd-toolchain@freebsd.org Cc: freebsd-ppc@freebsd.org, markmi@dsl-only.net Subject: PowerPC64 C++ exceptions with clang/llvm Message-ID: <20170524113441.GA58092@vlakno.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.8.2 (2017-04-18) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 May 2017 11:44:14 -0000 Hi, Currently C++ exceptions don't work on PowerPC64 when compiled with clang. I looked at why and discovered the following. With the default gcc unwinder and libstdc++ this simple test program: #include int main() { try { std::cout << "Before\n"; throw std::exception(); std::cout << "After\n"; } catch (...) { std::cout << "Exception\n"; } } segfaults right after printing "Before\n". This is because the .eh_frame is corrupted and the personality relocation is wrong: < 1> version 1 cie section offset 56 0x00000038 augmentation zPLR code_alignment_factor 4 data_alignment_factor -8 return_address_register 65 eh aug data len 0xb bytes 0x94 00 00 00 01 00 01 02 ed 14 1b bytes of initial instructions 3 cie length 28 initial instructions 0 DW_CFA_def_cfa r1 0 the personality relocation is "00 00 00 01 00 01 02 ed" encoded as pc relative. The program segfaults because this address is wrong. The 33rd bit is incorrect, the real address should be "00 00 00 00 00 01 02 ed". The same problem applies for LDSA encoding. Both personality and LDSA relocations are produced by CFI instructions in the assembler source code. I verified this theory by hacking the gcc unwinder and libsupc++ &= the address with 0xffffffff; and that made the application work. (the full patch can be found at http://www.vlakno.cz/~rdivacky/ppc64.exceptions.hack.patch). I checked why this relocation is wrong and it turns out it's our old in-tree ld that (wrongly) produces this. Our old gcc does not use CFI to produce these and hardcodes the CIE manually and produces correct address. And for some reason ld is fine with that. When I compile and assemble the above source code using clang++ and only use newer ld linker from ports the resulting binary is correct and works as expected. So this is a bug in our in-tree ld linker. I don't know why ld has this bug but given Mark Millard's report of ld being broken for other stuff (kernel iirc) I am not going to bother trying to fix it. I suspect it might be related to comdats but I didn't really investigate. I didn't try the situation with the LLVM unwinder and libcxxrt, but I hope it might be the same. Thank you, Roman From owner-freebsd-toolchain@freebsd.org Thu May 25 04:03:55 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFD7FD8179B for ; Thu, 25 May 2017 04:03:55 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-15.reflexion.net [208.70.210.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8602A19D6 for ; Thu, 25 May 2017 04:03:54 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 29448 invoked from network); 25 May 2017 04:03:53 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 25 May 2017 04:03:53 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Thu, 25 May 2017 00:03:53 -0400 (EDT) Received: (qmail 8168 invoked from network); 25 May 2017 04:03:53 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 25 May 2017 04:03:53 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id CFE10EC78CC; Wed, 24 May 2017 21:03:52 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: PowerPC64 C++ exceptions with clang/llvm From: Mark Millard In-Reply-To: <20170524113441.GA58092@vlakno.cz> Date: Wed, 24 May 2017 21:03:52 -0700 Cc: FreeBSD Toolchain , freebsd-ppc@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <74D091B2-71CD-42DE-8C1E-8E0BDED1EAFD@dsl-only.net> References: <20170524113441.GA58092@vlakno.cz> To: Roman Divacky X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 May 2017 04:03:55 -0000 [So far I've been unable to replicate dwarfump reporting anything near "0x94 00 00 00 01 00 01 02 ed 14 1b". The compiled code still gets SIGSEGV: it is only the low level detail that I'm noting as different. Roman has more information from me than I show here.] On 2017-May-24, at 4:34 AM, Roman Divacky wrote: > Hi, >=20 > Currently C++ exceptions don't work on PowerPC64 when compiled with = clang. I > looked at why and discovered the following. >=20 > With the default gcc unwinder and libstdc++ this simple test program: >=20 > #include >=20 > int main() > { > try { > std::cout << "Before\n"; > throw std::exception(); > std::cout << "After\n"; > } catch (...) { > std::cout << "Exception\n"; > } > } >=20 > segfaults right after printing "Before\n". This is because the = .eh_frame is corrupted and > the personality relocation is wrong: >=20 > < 1> version 1 > cie section offset 56 0x00000038 > augmentation zPLR > code_alignment_factor 4 > data_alignment_factor -8 > return_address_register 65 > eh aug data len 0xb bytes 0x94 00 00 00 01 00 01 02 ed 14 1b=20 So far I'm unable to replicate producing anything like the "large one" as I'll call it. I get things like: # clang++ -std=3Dc++11 exception_test_dots.cpp # dwarfdump -v -v -F a.out | more .eh_frame . . . cie: < 0> version 1 . . . eh aug data len 0xb bytes 0x94 00 00 00 00 00 01 03 45 14 1b=20 . . . (No cia<1> at all.) and like: # clang++ -B /usr/local/bin/ -std=3Dc++11 exception_test_dots.cpp # dwarfdump -v -v -F a.out | more .eh_frame . . . cie: < 0> version 1 . . . < 1> version 1 . . . eh aug data len 0xb bytes 0x94 00 00 00 00 00 01 f1 1d 14 1b=20 . . . My context is head -r317820 on a system built with the system clang without building gcc 4.2.1 materials. # uname -paKU FreeBSD FBSDG5L 12.0-CURRENT FreeBSD 12.0-CURRENT r317820M powerpc = powerpc64 1200030 1200030 # ld --version GNU ld 2.17.50 [FreeBSD] 2007-07-03 Copyright 2007 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms = of the GNU General Public License. This program has absolutely no = warranty. # /usr/local/bin/ld --version GNU ld (GNU Binutils) 2.27 Copyright (C) 2016 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms = of the GNU General Public License version 3 or (at your option) a later = version. This program has absolutely no warranty. (I have avoided 2.28 so far because of the "return code" consequences on the likes of aarch64.) I've tried variations like omitting -std=3Dc++11. So far I've not found a way to get near: "0x94 00 00 00 01 00 01 02 ed 14 1b" but all the experiments get SIGSEGV when I run the produced program. (I do not have a gcc 4.2.1 build around and so do not have libstdc++ on powerpc64.) > bytes of initial instructions 3 > cie length 28 > initial instructions > 0 DW_CFA_def_cfa r1 0 >=20 > the personality relocation is "00 00 00 01 00 01 02 ed" encoded as pc = relative. The program segfaults > because this address is wrong. The 33rd bit is incorrect, the real = address should be "00 00 00 00 00 01 02 ed". > The same problem applies for LDSA encoding. Both personality and LDSA = relocations are produced by CFI instructions > in the assembler source code. >=20 > I verified this theory by hacking the gcc unwinder and libsupc++ &=3D = the address with 0xffffffff; and that made > the application work. (the full patch can be found at = http://www.vlakno.cz/~rdivacky/ppc64.exceptions.hack.patch). >=20 > I checked why this relocation is wrong and it turns out it's our old = in-tree ld that (wrongly) produces this. > Our old gcc does not use CFI to produce these and hardcodes the CIE = manually and produces correct address. And for > some reason ld is fine with that. >=20 > When I compile and assemble the above source code using clang++ and = only use newer ld linker from ports the resulting > binary is correct and works as expected. >=20 > So this is a bug in our in-tree ld linker. I don't know why ld has = this bug but given Mark Millard's report of ld > being broken for other stuff (kernel iirc) I am not going to bother = trying to fix it. I suspect it might be related > to comdats but I didn't really investigate. >=20 > I didn't try the situation with the LLVM unwinder and libcxxrt, but I = hope it might be the same. Side note: Why I'm at head -r317820 . . . For TARGET_ARCH=3Dpowerpc I've got a periodic panic after booting for which changing the memory layout even a little bit seems to stop the panic. (Something less obvious may still be wrong.) The panic can take minutes but usually is more like hours. Even attempting to add small amounts of debug code tends to make the problem invisible. Thus a binary search for versions is not in the cards for now. So I've been focusing on trying for evidence about what leads to the panic and I've not been updating any of the FreeBSD environments that I have around. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-toolchain@freebsd.org Thu May 25 22:06:41 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D69ED82EE4 for ; Thu, 25 May 2017 22:06:41 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-63.reflexion.net [208.70.210.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 220D21BB3 for ; Thu, 25 May 2017 22:06:40 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 12676 invoked from network); 25 May 2017 21:59:19 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 25 May 2017 21:59:19 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Thu, 25 May 2017 17:59:19 -0400 (EDT) Received: (qmail 10415 invoked from network); 25 May 2017 21:59:19 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 25 May 2017 21:59:19 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 11BB2EC805D; Thu, 25 May 2017 14:59:19 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: llvm FreeBSD powerpc ABI target bug fix: Re: [Bug 26519] Clang 4.0.0's "Target: powerpc-unknown-freebsd11.0" code generation is violating the SVR4 ABI (SEGV can result) From: Mark Millard In-Reply-To: <408D3509-3D62-4413-986B-6C1171FB6138@dsl-only.net> Date: Thu, 25 May 2017 14:59:18 -0700 Cc: Roman Divacky , FreeBSD Toolchain , FreeBSD PowerPC ML Content-Transfer-Encoding: quoted-printable Message-Id: References: <0103401A-CEEA-4992-A45E-E60EA151119B@dsl-only.net> <893ECA11-7C80-4D24-A496-92ADC7978A07@FreeBSD.org> <408D3509-3D62-4413-986B-6C1171FB6138@dsl-only.net> To: Dimitry Andric X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 May 2017 22:06:41 -0000 [Top post of question with older material listed after for reference.] Is llvm bugzilla's latest 26519 fix going to make it into release/11.1.0 ? This fixes the last known stack handling ABI violation for targeting powerpc FreeBSD (32-bit). (For both powerpc64 and powerpc handling thrown C++ exceptions still needs work but the above does make the rest of buildworld operational for TARGET_ARCH=3Dpowerpc --as far as my basic testing goes anyway.) =3D=3D=3D Mark Millard markmi at dsl-only.net On 2017-May-17, at 3:22 PM, Mark Millard wrote: I got a notice this morning of the latest fix for covering the TARGET_ARCH=3Dpowerpc stack-handling issues for llvm bugzilla 26519: > Comment # 33 on bug 26519 from Krzysztof Parzyszek > The fix has been committed in master in r303257. >=20 > I opened PR33070 to request merging it into 4.0.1. >=20 > You are receiving this mail because: > =E2=80=A2 You reported the bug. I've been using a version of the patch for some time and for buildworld it appears that with it powerpc and powerpc64 have a similar status: the one known area not working is handling of thrown C++ exceptions --for example the required dwarf information is incomplete so programs crash. (I have one powerpc64 patch in use that is not applied upstream or in FreeBSD that is essential for the powerpc64 status. See the later side notes for the tiny patch.) For buildkernel there is a difference for TARGET_ARCH=3Dpowerpc vs. TARGET_ARCH=3Dpowerpc64 : A) powerpc46 works for building and running the kernel and world on the old G5 PowerMacs. B) powerpc FreeBSD on the same machines fails at the /sbin/init attempt and then gets an alignment exception. (I've not tried a G4 or G3 yet.) As of yet I've no clue why (B) is an issue. Side notes: Note 0: The patch I was given a fair time ago that is required for TARGET_ARCH=3Dpowerpc64 is: # svnlite diff /usr/src/contrib/llvm/tools/ Index: /usr/src/contrib/llvm/tools/lld/ELF/Target.cpp =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/contrib/llvm/tools/lld/ELF/Target.cpp (revision = 317820) +++ /usr/src/contrib/llvm/tools/lld/ELF/Target.cpp (working copy) @@ -1070,7 +1070,8 @@ } PPC64TargetInfo::PPC64TargetInfo() { - PltRel =3D GotRel =3D R_PPC64_GLOB_DAT; + GotRel =3D R_PPC64_GLOB_DAT; + PltRel =3D R_PPC64_JMP_SLOT; RelativeRel =3D R_PPC64_RELATIVE; GotEntrySize =3D 8; (Thanks to Roman Divacky.) Note 1: While a pure gcc 4.2.1 buildworld buildkernel with a debug kernel is working for booting and using, a production style kernel gets occasional panics on the old G5 PowerMacs. (The powerpc64 builds work fine on the same machines.) (I've not tried any G4's or a G3 yet.) It also appears that small changes in memory layout details (from trying to get better evidence) change the behavior/failure-mode/ details. I do not expect to find out what is going on any time soon. The same problems existed when buildworld was via clang 4. A kernel from a clang buildkernel does not get far enough for me to see what it would do for the issue. As this issue is more fundamental to general operation it has been getting much of my FreeBSD time. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-toolchain@freebsd.org Thu May 25 23:31:40 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B50EBD81824; Thu, 25 May 2017 23:31:40 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76E0A131D; Thu, 25 May 2017 23:31:40 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:470:7a58::4034:2885:b014:e7b0] (unknown [IPv6:2001:470:7a58:0:4034:2885:b014:e7b0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id C739016B07; Fri, 26 May 2017 01:31:30 +0200 (CEST) From: Dimitry Andric Message-Id: <45696549-B3B7-4080-AB2B-7A1DC0966C81@FreeBSD.org> Content-Type: multipart/signed; boundary="Apple-Mail=_28F03236-9184-43E4-A21E-EA1248916F82"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: llvm FreeBSD powerpc ABI target bug fix: Re: [Bug 26519] Clang 4.0.0's "Target: powerpc-unknown-freebsd11.0" code generation is violating the SVR4 ABI (SEGV can result) Date: Fri, 26 May 2017 01:31:18 +0200 In-Reply-To: Cc: FreeBSD Toolchain , FreeBSD PowerPC ML To: Mark Millard References: <0103401A-CEEA-4992-A45E-E60EA151119B@dsl-only.net> <893ECA11-7C80-4D24-A496-92ADC7978A07@FreeBSD.org> <408D3509-3D62-4413-986B-6C1171FB6138@dsl-only.net> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 May 2017 23:31:40 -0000 --Apple-Mail=_28F03236-9184-43E4-A21E-EA1248916F82 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On 25 May 2017, at 23:59, Mark Millard wrote: > > Is llvm bugzilla's latest 26519 fix going to make it > into release/11.1.0 ? This fixes the last known stack > handling ABI violation for targeting powerpc FreeBSD > (32-bit). I just committed it in r318906. It should make 11.1-RELEASE. Would be nice if we can finally close PR 206990. :) -Dimitry --Apple-Mail=_28F03236-9184-43E4-A21E-EA1248916F82 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.30 iEYEARECAAYFAlknaVIACgkQsF6jCi4glqPCQwCdH7xbkCUG6e7E4I4eFINg3Qfj 5EwAoKmPAfmwW+maIJkWidS6EDfd6x1f =bvq7 -----END PGP SIGNATURE----- --Apple-Mail=_28F03236-9184-43E4-A21E-EA1248916F82-- From owner-freebsd-toolchain@freebsd.org Fri May 26 00:29:14 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B8F4D7AFA0 for ; Fri, 26 May 2017 00:29:14 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-62.reflexion.net [208.70.210.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AFBD9118C for ; Fri, 26 May 2017 00:29:13 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 15302 invoked from network); 26 May 2017 00:29:11 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 26 May 2017 00:29:11 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Thu, 25 May 2017 20:29:12 -0400 (EDT) Received: (qmail 6258 invoked from network); 26 May 2017 00:29:11 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 26 May 2017 00:29:11 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 1A3FAEC7D4D; Thu, 25 May 2017 17:29:11 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: llvm FreeBSD powerpc ABI target bug fix: Re: [Bug 26519] Clang 4.0.0's "Target: powerpc-unknown-freebsd11.0" code generation is violating the SVR4 ABI (SEGV can result) From: Mark Millard In-Reply-To: <45696549-B3B7-4080-AB2B-7A1DC0966C81@FreeBSD.org> Date: Thu, 25 May 2017 17:29:10 -0700 Cc: FreeBSD Toolchain , FreeBSD PowerPC ML Content-Transfer-Encoding: 7bit Message-Id: References: <0103401A-CEEA-4992-A45E-E60EA151119B@dsl-only.net> <893ECA11-7C80-4D24-A496-92ADC7978A07@FreeBSD.org> <408D3509-3D62-4413-986B-6C1171FB6138@dsl-only.net> <45696549-B3B7-4080-AB2B-7A1DC0966C81@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 May 2017 00:29:14 -0000 On 2017-May-25, at 4:31 PM, Dimitry Andric wrote: > On 25 May 2017, at 23:59, Mark Millard wrote: >> >> Is llvm bugzilla's latest 26519 fix going to make it >> into release/11.1.0 ? This fixes the last known stack >> handling ABI violation for targeting powerpc FreeBSD >> (32-bit). > > I just committed it in r318906. It should make 11.1-RELEASE. Would be > nice if we can finally close PR 206990. :) To my knowledge there are no more stack handling problems and use of the update should provide evidence that 206990 can be closed. But it may be a while before I update to direct use of something after head -r317820. . . I've frozen at head -317820 while trying to get evidence for a periodic powerpc kernel panic in that version (kernel built with gcc 4.2.1, worlds via both compilers). Small adjustments that change the kernel memory layout a little tend to change the behavior, usually meaning no obvious symptoms. So updating to some other version likely just hides the problem. Unfortunately it frequently takes hours for the panic to occur and so a long time to conclude the panic is not likely to happen in any test that I have running. It may be that I just will not identify anything that points at anything fairly specific to fix but I've not given up yet. What I know of for powerpc's clang status with the stack handling fixes in place is: A) Handling thrown C++ exceptions crashes for code generated by clang. (powerpc64 also has this property.) B) A clang-based kernel fails to start /sbin/init and then gets a data alignment panic. Past investigations relative to (A) suggest multiple problems are around that contribute, it is not just one simple problem. I've not looked at (B) yet. === Mark Millard markmi at dsl-only.net From owner-freebsd-toolchain@freebsd.org Fri May 26 11:00:36 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B01FCD7B66A for ; Fri, 26 May 2017 11:00:36 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-63.reflexion.net [208.70.210.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 75E271062 for ; Fri, 26 May 2017 11:00:35 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 12637 invoked from network); 26 May 2017 11:00:34 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 26 May 2017 11:00:34 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Fri, 26 May 2017 07:00:34 -0400 (EDT) Received: (qmail 1700 invoked from network); 26 May 2017 11:00:34 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 26 May 2017 11:00:34 -0000 Received: from [192.168.1.114] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 58486EC805D; Fri, 26 May 2017 04:00:33 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: PowerPC64 C++ exceptions with clang/llvm From: Mark Millard In-Reply-To: <74D091B2-71CD-42DE-8C1E-8E0BDED1EAFD@dsl-only.net> Date: Fri, 26 May 2017 04:00:32 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20170524113441.GA58092@vlakno.cz> <74D091B2-71CD-42DE-8C1E-8E0BDED1EAFD@dsl-only.net> To: FreeBSD Toolchain , FreeBSD PowerPC ML X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 May 2017 11:00:36 -0000 [Turns out Roman's working case was based on gcc 4.2.1 based system libraries, not ones built by clang.] On 2017-May-24, at 9:03 PM, Mark Millard wrote: > [So far I've been unable to replicate dwarfump reporting > anything near "0x94 00 00 00 01 00 01 02 ed 14 1b". The > compiled code still gets SIGSEGV: it is only the low > level detail that I'm noting as different. Roman has > more information from me than I show here.] >=20 > On 2017-May-24, at 4:34 AM, Roman Divacky = wrote: >=20 >> Hi, >>=20 >> Currently C++ exceptions don't work on PowerPC64 when compiled with = clang. I >> looked at why and discovered the following. >>=20 >> With the default gcc unwinder and libstdc++ this simple test program: >>=20 >> #include >>=20 >> int main() >> { >> try { >> std::cout << "Before\n"; >> throw std::exception(); >> std::cout << "After\n"; >> } catch (...) { >> std::cout << "Exception\n"; >> } >> } >>=20 >> segfaults right after printing "Before\n". This is because the = .eh_frame is corrupted and >> the personality relocation is wrong: >>=20 >> < 1> version 1 >> cie section offset 56 0x00000038 >> augmentation zPLR >> code_alignment_factor 4 >> data_alignment_factor -8 >> return_address_register 65 >> eh aug data len 0xb bytes 0x94 00 00 00 01 00 01 02 ed 14 1b=20 >=20 > So far I'm unable to replicate producing anything like > the "large one" as I'll call it. >=20 > I get things like: >=20 > # clang++ -std=3Dc++11 exception_test_dots.cpp > # dwarfdump -v -v -F a.out | more >=20 > .eh_frame > . . . > cie: > < 0> version 1 > . . . > eh aug data len 0xb bytes 0x94 00 00 00 00 00 01 03 45 14 1b=20 > . . . >=20 > (No cia<1> at all.) >=20 > and like: >=20 > # clang++ -B /usr/local/bin/ -std=3Dc++11 exception_test_dots.cpp > # dwarfdump -v -v -F a.out | more >=20 > .eh_frame > . . . > cie: > < 0> version 1 > . . . > < 1> version 1 > . . . > eh aug data len 0xb bytes 0x94 00 00 00 00 00 01 f1 1d 14 1b=20 > . . . >=20 > My context is head -r317820 on a system built with > the system clang without building gcc 4.2.1 materials. >=20 > # uname -paKU > FreeBSD FBSDG5L 12.0-CURRENT FreeBSD 12.0-CURRENT r317820M powerpc = powerpc64 1200030 1200030 >=20 > # ld --version > GNU ld 2.17.50 [FreeBSD] 2007-07-03 > Copyright 2007 Free Software Foundation, Inc. > This program is free software; you may redistribute it under the terms = of > the GNU General Public License. This program has absolutely no = warranty. >=20 > # /usr/local/bin/ld --version > GNU ld (GNU Binutils) 2.27 > Copyright (C) 2016 Free Software Foundation, Inc. > This program is free software; you may redistribute it under the terms = of > the GNU General Public License version 3 or (at your option) a later = version. > This program has absolutely no warranty. >=20 > (I have avoided 2.28 so far because of the > "return code" consequences on the likes of > aarch64.) >=20 > I've tried variations like omitting -std=3Dc++11. > So far I've not found a way to get near: >=20 > "0x94 00 00 00 01 00 01 02 ed 14 1b" >=20 > but all the experiments get SIGSEGV when > I run the produced program. >=20 > (I do not have a gcc 4.2.1 build around and so > do not have libstdc++ on powerpc64.) It is still true that I've not replicated the "large one" problems (as I called it). But my environment is not a mix of gcc 4.2.1 and clang based: just clang based, including libc++ and the like. So things are very different from Roman's context. >> bytes of initial instructions 3 >> cie length 28 >> initial instructions >> 0 DW_CFA_def_cfa r1 0 >>=20 >> the personality relocation is "00 00 00 01 00 01 02 ed" encoded as pc = relative. The program segfaults >> because this address is wrong. The 33rd bit is incorrect, the real = address should be "00 00 00 00 00 01 02 ed". >> The same problem applies for LDSA encoding. Both personality and LDSA = relocations are produced by CFI instructions >> in the assembler source code. >>=20 >> I verified this theory by hacking the gcc unwinder and libsupc++ &=3D = the address with 0xffffffff; and that made >> the application work. (the full patch can be found at = http://www.vlakno.cz/~rdivacky/ppc64.exceptions.hack.patch). Roman did not test a libstdc++ built with clang here. He was using a library to support handling thrown C++ exceptions that was built by gcc 4.2.1 . Instead the only clang code tested was in his a.out (or whatever he named the program generated). My past testing indicates that whatever library is supposed to support thrown C++ exceptions is broken when built by clang. (I've a couple of llvm submittals about messed up dwarf information and the like.) In essence the consequences of the builtin's that are involved in creating the supporting code in the library is wrong/incomplete in clang's code and/or dwarf generation as I understand. For example: __builtin_unwind_init() __builtin_eh_return_data_regno (0) __builtin_eh_return __builtin_dwarf_cfa() were involved back when I did the clang 3.8 investigation (that mostly seems to still apply in the C++ exception handling area). I'm not surprised that clang generated code can use the library code that gcc 4.2.1 generated, at least for the small examples involved (and probably far more). >> I checked why this relocation is wrong and it turns out it's our old = in-tree ld that (wrongly) produces this. >> Our old gcc does not use CFI to produce these and hardcodes the CIE = manually and produces correct address. And for >> some reason ld is fine with that. >>=20 >> When I compile and assemble the above source code using clang++ and = only use newer ld linker from ports the resulting >> binary is correct and works as expected. >>=20 >> So this is a bug in our in-tree ld linker. I don't know why ld has = this bug but given Mark Millard's report of ld >> being broken for other stuff (kernel iirc) I am not going to bother = trying to fix it. I suspect it might be related >> to comdats but I didn't really investigate. As my libc++ and the like was built with clang for my tests, the system ld does not work for buildworld for such a context, nor does lld. So I used devel/powerpc64-binutils for buildworld and buildkernel. Given that clang built it all instead of gcc 4.2.1, what the system ld does for the little test program in my context would be expected to be likely to not match Roman's results. But my context gives more information about the status of things for attempting to avoid gcc 4.2.1, since that is what I was doing (gcc 4.2.1 and related material not built or present) . >> I didn't try the situation with the LLVM unwinder and libcxxrt, but I = hope it might be the same. Last I tried the llvm unwinder could not be be built by clang. I submitted llvm bugzilla material for it at the time. lld did not work last I tried either. (In fact Roman supplied one patch that let me get far enough to report some on what else is a problem for lld.) =3D=3D=3D Mark Millard markmi at dsl-only.net