From owner-freebsd-toolchain@freebsd.org Thu Dec 1 05:57:25 2016 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B30CC6274B for ; Thu, 1 Dec 2016 05:57:25 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-57.reflexion.net [208.70.210.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FE1A1955 for ; Thu, 1 Dec 2016 05:57:24 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 13435 invoked from network); 1 Dec 2016 05:50:44 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 1 Dec 2016 05:50:44 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.20.0) with SMTP; Thu, 01 Dec 2016 00:50:50 -0500 (EST) Received: (qmail 30913 invoked from network); 1 Dec 2016 05:50:50 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 1 Dec 2016 05:50:50 -0000 Received: from [192.168.1.106] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 8012DEC8FF8; Wed, 30 Nov 2016 21:50:38 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) Subject: Re: WITH_LLVM_LIBUNWIND vs. WITHOUT_LLVM_LIBUNWIND, clang vs. gcc (such as devel/powerpc64-xtoolchain-gcc ): What is intended to be required for C++ exceptions to work? From: Mark Millard In-Reply-To: <81CBAB7A-B855-4673-B2E2-7862F441FDDA@dsl-only.net> Date: Wed, 30 Nov 2016 21:50:37 -0800 Cc: FreeBSD Toolchain , FreeBSD PowerPC ML , Dimitry Andric Content-Transfer-Encoding: quoted-printable Message-Id: <68972A8F-C8D4-4FF2-A933-ACCC79C15065@dsl-only.net> References: <750FCE4D-F25B-46E1-9383-B8A94AAA8792@dsl-only.net> <81CBAB7A-B855-4673-B2E2-7862F441FDDA@dsl-only.net> To: Ed Maste X-Mailer: Apple Mail (2.3251) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2016 05:57:25 -0000 [I add the dwarfdump -v -v -F /lib/libgcc_s.so.1 from 9.3-RELEASE -r268512 that does not agree with stable/11 -r309179's clang 3.9.0 content: it is like OpenMandriva Lx for the scratch register handling. I looked back this far because it was gcc 4.2.1 based for official amd64 releases.] On 2016-Nov-30, at 6:03 PM, Mark Millard wrote: > On 2016-Nov-29, at 2:56 PM, Ed Maste wrote: >=20 >> On 29 November 2016 at 16:46, Mark Millard = wrote: >>>=20 >>>=20 >>> Summary: Does using clang 3.9.0 as the system compiler imply one = should or >>> must (eventually?) use WITH_LLVM_LIBUNWIND to have C++ exceptions = work? >>>=20 >>> Do WITH_LLVM_LIBUNWIND and WITHOUT_LLVM_LIBUNWIND have the same = criteria >>> for what dwarfdump should show for the exception information (if the >>> information to be shown is to be correct/sufficient for libunwind)? >>=20 >> It does not. It should be possible to build a functional system both >> WITH_ and WITHOUT_LLVM_LIBUNWIND. The compiler is unaware of the >> _LLVM_LIBUNWIND setting. Both unwind libraries use the same unwind >> data. >=20 > Good to know. Thanks. >=20 > [As there are no clang-based powerpc64 linux builds to my knowledge > the following uses amd64/x86_64 examples instead for comparisons.] >=20 > What I see from "dwarfdump -v -v -F .../libgcc_s.so.1" for FreeBSD > head -r309179 vs. what I see for the clang 3.8.0 based OpenMandriva > Lx V3.0 leaves me a little worried: OpenMandriva Lx V3.0 is more > like what I would expect for: >=20 > _Unwind_RaiseException (9990 vs. c701) > _Unwind_ForcedUnwind (a330 vs. c857) > _Unwind_Resume (9fb0 vs. c927) > _Unwind_Resume_or_Rethrow (2da0 vs. c9e4) >=20 > (The numbers are just for my reference in finding the dwarfdump > material.) >=20 > By contrast FreeBSD has missing information from what I can tell. >=20 > For these 4 routines only: OpenMandriva Lx has scratch registers > listed (r0 and r1 from the set of normally volatile registers > normally not listed). No other routines in libgcc_s.so.1 have > this. FreeBSD has no such extra registers listed anywhere, > including not on the 4 where I'd expect them. >=20 > Using _Unwind_RaiseException as an example of those 4 routines > that I'm worried about. . . >=20 > [Notes: r7 is %rsp, r6 is %rbp, r3 is %rbx, r2 is %rcx, r1 is %rdx, > and r0 is %rax in amd64/x86_64 land. Only OpenMandriva Lx had > examples of the last two. I use the dwarfdump naming below.] >=20 > FreeBSD _Unwind_RaiseException: >=20 > fde section offset 5328 0x000014d0 cie offset for fde: 5332 0x000014d4 > 0 DW_CFA_advance_loc 1 (1 * 1) > 1 DW_CFA_def_cfa_offset 16 > 3 DW_CFA_offset r6 -16 (2 * -8) > 5 DW_CFA_advance_loc 3 (3 * 1) > 6 DW_CFA_def_cfa_register r6 > 8 DW_CFA_advance_loc 16 (16 * 1) > 9 DW_CFA_offset r3 -56 (7 * -8) > 11 DW_CFA_offset r12 -48 (6 * -8) > 13 DW_CFA_offset r13 -40 (5 * -8) > 15 DW_CFA_offset r14 -32 (4 * -8) > 17 DW_CFA_offset r15 -24 (3 * -8) > 19 DW_CFA_nop > 20 DW_CFA_nop > 21 DW_CFA_nop > 22 DW_CFA_nop [It may be that some sort of update after 9.x makes what I report below expected. If so I've not identified it yet. At this point that is my guess: a different amd64 ABI now than in 9.3 for C++ exception handling, at least for some details.] The amd64 9.3-RELEASE -r268512 dwarfdump output for its _Unwind_RaiseException in its /lib/libgcc_s.so.1 does not match the above from stable/11 -r309179 for the scratch register handling. Note the r0 and r1 items (20 and 22): fde section offset 2672 0x00000a70 cie offset for fde: 2676 0x00000a74 0 DW_CFA_advance_loc 1 (1 * 1) 1 DW_CFA_def_cfa_offset 16 3 DW_CFA_offset r6 -16 (2 * -8) 5 DW_CFA_advance_loc 3 (3 * 1) 6 DW_CFA_def_cfa_register r6 8 DW_CFA_advance_loc 13 (13 * 1) 9 DW_CFA_offset r3 -56 (7 * -8) 11 DW_CFA_offset r12 -48 (6 * -8) 13 DW_CFA_offset r13 -40 (5 * -8) 15 DW_CFA_offset r14 -32 (4 * -8) 17 DW_CFA_offset r15 -24 (3 * -8) 19 DW_CFA_advance_loc 9 (9 * 1) 20 DW_CFA_offset r0 -72 (9 * -8) 22 DW_CFA_offset r1 -64 (8 * -8) 24 DW_CFA_nop 25 DW_CFA_nop 26 DW_CFA_nop 27 DW_CFA_nop 28 DW_CFA_nop 29 DW_CFA_nop 30 DW_CFA_nop The 9.3 code has %rdx (r2) and %rax (r0) saved and restored: 00000000000088d0 <_Unwind_RaiseException> push %rbp 00000000000088d1 <_Unwind_RaiseException+0x1> mov %rsp,%rbp 00000000000088d4 <_Unwind_RaiseException+0x4> push %r15 00000000000088d6 <_Unwind_RaiseException+0x6> lea 0x10(%rbp),%rsi 00000000000088da <_Unwind_RaiseException+0xa> push %r14 00000000000088dc <_Unwind_RaiseException+0xc> push %r13 00000000000088de <_Unwind_RaiseException+0xe> push %r12 00000000000088e0 <_Unwind_RaiseException+0x10> push %rbx 00000000000088e1 <_Unwind_RaiseException+0x11> lea -0x3a0(%rbp),%rbx 00000000000088e8 <_Unwind_RaiseException+0x18> push %rdx 00000000000088e9 <_Unwind_RaiseException+0x19> push %rax . . . (the code here freely assigns to and uses %rdx and %rax) . . . . . . (also there are multiple return paths with differing properties) . = . . 00000000000089a3 <_Unwind_RaiseException+0xd3> mov -0x28(%rbp),%rbx 00000000000089a7 <_Unwind_RaiseException+0xd7> mov -0x20(%rbp),%r12 00000000000089ab <_Unwind_RaiseException+0xdb> mov -0x18(%rbp),%r13 00000000000089af <_Unwind_RaiseException+0xdf> mov -0x10(%rbp),%r14 00000000000089b3 <_Unwind_RaiseException+0xe3> mov -0x8(%rbp),%r15 00000000000089b7 <_Unwind_RaiseException+0xe7> leaveq=20 00000000000089b8 <_Unwind_RaiseException+0xe8> retq =20 . . . (Note the lack of restoring %rax above and the normal %rbp/%rsp restore: leaveq) . . . . . . (before the below involves _Unwind_Backtrace that the above did not) . . . 0000000000008a33 <_Unwind_RaiseException+0x163> lea = 0x8(%rbp,%rcx,1),%rcx 0000000000008a38 <_Unwind_RaiseException+0x168> mov -0x38(%rbp),%rax 0000000000008a3c <_Unwind_RaiseException+0x16c> mov -0x30(%rbp),%rdx 0000000000008a40 <_Unwind_RaiseException+0x170> mov -0x28(%rbp),%rbx 0000000000008a44 <_Unwind_RaiseException+0x174> mov -0x20(%rbp),%r12 0000000000008a48 <_Unwind_RaiseException+0x178> mov -0x18(%rbp),%r13 0000000000008a4c <_Unwind_RaiseException+0x17c> mov -0x10(%rbp),%r14 0000000000008a50 <_Unwind_RaiseException+0x180> mov -0x8(%rbp),%r15 0000000000008a54 <_Unwind_RaiseException+0x184> mov 0x0(%rbp),%rbp 0000000000008a58 <_Unwind_RaiseException+0x188> mov %rcx,%rsp 0000000000008a5b <_Unwind_RaiseException+0x18b> retq . . . (Note the abnormal %rbp/%rsp restore above) . . . No other routine in 9.3's /lib/libgcc_s.so.1 pushes or pops either of %rdx or %rax: only _Unwind_RaiseException. The code structure allows executing exception landing-pad code and such during the exception handling with the 2 registers used to provide context. The stable/11 -r309179 code for _Unwind_RaiseException does not have those scratch registers saved and restored and has some other basic differences for related aspects: 0000000000009990 <_Unwind_RaiseException> push %rbp 0000000000009991 <_Unwind_RaiseException+0x1> mov %rsp,%rbp 0000000000009994 <_Unwind_RaiseException+0x4> push %r15 0000000000009996 <_Unwind_RaiseException+0x6> push %r14 0000000000009998 <_Unwind_RaiseException+0x8> push %r13 000000000000999a <_Unwind_RaiseException+0xa> push %r12 000000000000999c <_Unwind_RaiseException+0xc> push %rbx 000000000000999d <_Unwind_RaiseException+0xd> sub $0x418,%rsp . . . (the code here freely assigns to and uses %rdx and %rax) . . . 0000000000009c61 <_Unwind_RaiseException+0x2d1> add $0x418,%rsp 0000000000009c68 <_Unwind_RaiseException+0x2d8> pop %rbx 0000000000009c69 <_Unwind_RaiseException+0x2d9> pop %r12 0000000000009c6b <_Unwind_RaiseException+0x2db> pop %r13 0000000000009c6d <_Unwind_RaiseException+0x2dd> pop %r14 0000000000009c6f <_Unwind_RaiseException+0x2df> pop %r15 0000000000009c71 <_Unwind_RaiseException+0x2e1> pop %rbp 0000000000009c72 <_Unwind_RaiseException+0x2e2> retq No code in the stable/11 libgcc_s.so.1 has a "push %rdx", but lots of code there has a "push %rax". The only means by which _Unwind_RaiseException's %rsp is restored/"popped" is also very different for stable/11's code compared to the 2nd of the 9.3 code paths for returning: stable/11 seems to not have any match analogous to that path. The differences seem too large to be the same ABI when things seem to be working. (Although I've not tried 9.3 application code that uses C++ exceptions under stable/11 : old stuff could fail if well tested for all I know.) I've not looked at amd64 10.x yet. > All the registers referenced are very common: lots of routines > having nothing to do with the Exception handling also list them. >=20 > OpenMandriva Lx _Unwind_RaiseException: >=20 > fde section offset 4280 0x000010b8 cie offset for fde: 4284 0x000010bc > 0 DW_CFA_advance_loc 1 (1 * 1) > 1 DW_CFA_def_cfa_offset 16 > 3 DW_CFA_offset r6 -16 (2 * -8) > 5 DW_CFA_advance_loc 3 (3 * 1) > 6 DW_CFA_def_cfa_register r6 > 8 DW_CFA_advance_loc 8 (8 * 1) > 9 DW_CFA_offset r15 -24 (3 * -8) > 11 DW_CFA_offset r14 -32 (4 * -8) > 13 DW_CFA_offset r13 -40 (5 * -8) > 15 DW_CFA_offset r12 -48 (6 * -8) > 17 DW_CFA_advance_loc 20 (20 * 1) > 18 DW_CFA_offset r3 -56 (7 * -8) > 20 DW_CFA_offset r1 -64 (8 * -8) > 22 DW_CFA_offset r0 -72 (9 * -8) > 24 DW_CFA_advance_loc2 284 > 27 DW_CFA_remember_state > 28 DW_CFA_restore r6 > 29 DW_CFA_def_cfa r2 8 > 32 DW_CFA_advance_loc 4 (4 * 1) > 33 DW_CFA_restore_state > 34 DW_CFA_advance_loc 21 (21 * 1) > 35 DW_CFA_def_cfa r7 8 > 38 DW_CFA_nop >=20 > Note the lines for r1 and r0 that FreeBSD has no > equivalent of: These are the Exception ABI's > scratch registers usage that only apply to exception > handling code that uses certain built-ins that > imply the context. In normal routines r1 and r0 are > volatile and so are not listed on OpenMandriva Lx. >=20 > But FreeBSD never lists any scratch registers for the > exception handing ABI, unlike the Itanium's standard > that specified there are 4 --2 with predefined > purposes. (Itanimum used GP15-GP18 for the exception > handling scratch registers or something like that if > I remember right.) >=20 > So the amd64 FreeBSD information from clang 3.9.0 looks > wrong to me unless the Itanium's standard is not the > basis for what FreeBSD is doing here. >=20 > NOTE: Analogous incorrectness with observed scratch > register usage in builds based on clang 3.8.0 and > clang 3.9.0 for TARGET_ARCH=3Dpowerpc and > TARGET_ARCH=3Dpowerpc64 are part of what I've reported > as wrong to llvm for what blocks clang for them. In > this context actual crashes from asserts or SIGSEGV > during _Unwind_GetGR or _Unwind_SetGR for a scratch > register's index is what started my investigation of > the dwarfdump reports in the first place. I found > the register information missing in the dwarfdump > output and comments in the code about being such a > missing status for any GR leading to what I had > observed for references to that GR --and that is > my interpretation of the code so commented as well. >=20 >=20 >> Eventually new features may show up in Clang and LLVM's libunwind = (and >> new versions of GNU's unwinder) that won't work with the old = unwinder. >>=20 >>> Your answer's detail might indicate that I've misdirected the llvm = folks >>> in submittals like https://llvm.org/bugs/show_bug.cgi?id=3D26844 . >>>=20 >>> There is also the question of if/when llvm's libunwind is ready to = be used >>> for powerpc64 or powerpc (or . . .) if there are architecture = specifics >>> involved. That answer might determine when C++ exceptions work (and = so >>> when devel/kyua might have a chance to work) and is sort of separate = from >>> the main question here but is still of interest overall. >>>=20 >>> Should powerpc64 and powerpc clang 3.9.0 testing be using >>> WITH_LLVM_LIBUNWIND ? WITHOUT_LLVM_LIBUNWIND ? Both? >>=20 >> For testing I think WITH_LLVM_LIBUNWIND is the interesting case. My >> eventual goal is to have a functioning Clang, LLD, LLDB, libunwind, >> and ELF Tool Chain on all of our supported architectures. =3D=3D=3D Mark Millard markmi at dsl-only.net