Date: Sun, 8 Jan 2017 02:24:17 -0800 From: Mark Millard <markmi@dsl-only.net> To: Roman Divacky <rdivacky@vlakno.cz> Cc: Ed Maste <emaste@freebsd.org>, Mark Linimon <linimon@lonesome.com>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org>, Justin Hibbits <chmeeedalf@gmail.com>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: clang 3.9.0 buildkernel on old powerpc64's vs. trying to build hwpmc_e500.o and the like. . . [Actually .__start code failure vs. working and R_PPC64_ADDR16_DS vs. R_PPC64_TOC16_DS] Message-ID: <1BBC8304-F9E8-4181-A287-11DD8E73B31F@dsl-only.net> In-Reply-To: <20170108090307.GA3140@vlakno.cz> References: <CE88C1F4-B9BD-4D45-8DF0-F1079C3257A5@dsl-only.net> <20161208185541.GA33364@vlakno.cz> <E49F7EE4-7A62-4601-98DC-4C4791B7158D@dsl-only.net> <20161208221452.GA42380@vlakno.cz> <20161212210922.GA27403@vlakno.cz> <613BB28B-46F1-4959-B576-C8AD42A21200@dsl-only.net> <20170107085126.GA82107@vlakno.cz> <2B5FDD60-4D8B-4803-B59C-3C569BA36E68@dsl-only.net> <FB7F73D7-BF8F-4477-8057-1404D27622A8@dsl-only.net> <45C94BFE-4F93-4897-A767-518DACB5D28B@dsl-only.net> <20170108090307.GA3140@vlakno.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017-Jan-8, at 1:03 AM, Roman Divacky <rdivacky at vlakno.cz> wrote:
> I think we should add the @toc notations to all the places where it's =
needed.
> Can you submit such a patch (perhaps with the one for adding 0 to the =
cmp
> instruction) so they can be committed to FreeBSD repo?
In order to test I added @toc to each of the 10 instructions instead
of adjusting the macros so that instructions would automatically
get the notation but other (what are now) TOC_REF(...) usage would
not that should not.
I suspect Nathan and Justin might prefer a more automatic
alternative so that TOC_REF(...) in an instruction would
be sufficient without an explicit @toc in the instruction.
I'll see about switching to such code before providing a
patch. I'd include the "0, " update to the cmp instruction.
But adding @toc's in those instructions (with prior workarounds
as well) did allow me to build a bootable system based on using
devel/powerpc64-binutils and clang 3.9.1 for both buildworld
and buildkernel --still using clang's internal assembler.
One issue is that clang does not support (or need) the
-mminmal-toc in sys/modules/zfs/Makefile but gcc 4.2.1
requires it. For sys/modules/zfs/Makefile my source
currently does not support gcc 4.2.1 without editing.
As I remember devel/powerpc64-gcc
(devel/powerpc64-xtoolchain-gcc) does not require the
-mminimal-toc either. But devel/powerpc64-gcc does
allow -mminimal-toc so it can be a clang vs. gcc based
choice.
I might also deal with that before providing patches.
Note: -mlongcall was also not needed nor used for clang.
(Still used for gcc.)
sys/boot/powerpc/kboot/Makefile has a -mcpu=3Dpowerpc64
and -Wa,-mppc64bridge that I eliminated (they messed up
32-bit powerpc builds) but I've no way to test kboot that
I know of. This patch might be rejected.
Remember that I got this far in part by using your partial
e500-instructions patch. I can provide my variant that
is a diff with -r311147 instead of an older place in the
history. But it would be incomplete coverage of those 2
instructions in clang.
Also I build with a workaround for PowerMac G5 boot
reliability since OpenFirmware and FreeBSD interact
badly at times on PowerMac G5's. This I would not
provide as it is PowerMac G5 specific.
> If gnu as warns and llvm assembler does the wrong thing without @toc =
and both
> do the correct thing with @toc I think it's an obvious decision.
My choice would be to supply the @toc notation in some way,
not necessarily the form I used for the initial test.
> Also, with all these changes. Does clang compiled kernel boot?
The PowerMac G5 so-called "Quad Core" that I currently have access
to is now running from buildworld and buildkernel based on
devel/powerpc64-binutils and clang 3.9.1 :
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-CURRENT #2 r311147M: Sat Jan 7 16:55:01 PST 2017
=
markmi@FreeBSDx64:/usr/obj/powerpc64vtsc_clang_altbinutils_kernel/powerpc.=
powerpc64/usr/src/sys/GENERIC64vtsc-NODBG powerpc
FreeBSD clang version 3.9.1 (tags/RELEASE_391/final 289601) (based on =
LLVM 3.9.1)
. . .
# uname -apKU
FreeBSD FBSDG5L 12.0-CURRENT FreeBSD 12.0-CURRENT #2 r311147M: Sat Jan =
7 16:55:01 PST 2017 =
markmi@FreeBSDx64:/usr/obj/powerpc64vtsc_clang_altbinutils_kernel/powerpc.=
powerpc64/usr/src/sys/GENERIC64vtsc-NODBG powerpc powerpc64 1200019 =
1200019
I've built about a dozen ports on it after booting but I've not
done a self-hosted buildworld buildkernel yet.
[Note: Anything dependent on throwing C++ exceptions in
code generated by clang++ 3.9.1 is broken.]
=3D=3D=3D
Mark Millard
markmi at dsl-only.net
On Sat, Jan 07, 2017 at 04:28:48PM -0800, Mark Millard wrote:
> [Top post about FreeBSD bugzilla 215819 and llvm bugzilla
> 31574 submittals for the issue involved.]
>=20
> My guess is that FreeBSD will view this as a kernel
> source code issue and not as a toolchain issue. But
> it is only a guess on my part.
>=20
>=20
> I have submitted llvm bugzilla 31574 for this issue. It
> includes example .S file content that shows the "problem"
> in the generated .o file (form FreeBSD's view point).
> (I've no clue how llvm views its criteria relative to this
> mismatch with gcc/binutils.)
>=20
> Because FreeBSD source code changes (being explicit about
> @toc) avoid the distinction between clang and gcc/binutils
> I've not added 31574 to the Depends on list for llvm 25780
> (the FreeBSD system compiler issues META entry in llvm).
>=20
> Someone with official status for FreeBSD could add 31574 to
> llvm's 25780 if FreeBSD wants to push llvm to match
> gcc/binutils for "@toc notation missing".
>=20
> Otherwise this is a kernel source code issue (as I would
> guess) and not a toolchain issue.
>=20
> Ed Maste or someone like that likely would make the final
> decision.
>=20
>=20
> I've added to FreeBSD Bugzilla 215819 the new information,
> including the simple example .S file content that shows the
> problem in the generated .o file. (Comments #3 and #4
> have the new material.)
>=20
> My guess is that FreeBSD Bugzilla 215819 should no longer
> be assigned to freebsd-toolchain@FreeBSD.org . Instead it
> would be a powerpc64 kernel source code issue if I'm right.
>=20
> Ed Maste or someone like that likely would make this final
> decision as well.
>=20
> =3D=3D=3D
> Mark Millard
> markmi at dsl-only.net
>=20
> On 2017-Jan-7, at 3:12 PM, Mark Millard <markmi at dsl-only.net> =
wrote:
>=20
>> [I've supplied a list of places that adding @toc notation should
>> make clang 3.9.1 targeting powerpc64 do the right thing for
>> this issue.]
>>=20
>> On 2017-Jan-7, at 2:07 PM, Mark Millard <markmi at dsl-only.net> =
wrote:
>>=20
>>> On 2017-Jan-7, at 12:51 AM, Roman Divacky <rdivacky at vlakno.cz> =
wrote:
>>>=20
>>>> That's a great progress. Can you produce minimal self contained =
test case that
>>>> exhibits this bug? And submit it to llvm bugzilla?
>>>>=20
>>>> Also, clang3.9 defaults to using it's own internal asm, what =
happens if you
>>>> add -no-integrated-as to CFLAGS and recompile the kernel? That =
should remove
>>>> this llvm assembly problem. Does it boot?
>>>>=20
>>>> Thanks Mark, really great progress.
>>>>=20
>>>> Roman
>>>=20
>>> In attempting this I found how to control the behavior based on
>>> the assembler notation @toc being missing vs. being present.
>>>=20
>>> If llvm should change is strongly tied to llvm's criteria for
>>> gcc compatibility relative to filling-in/defaulting omitted
>>> @toc's in the assembler notation.
>>>=20
>>> FreeBSD has the option of always being explicit with @toc in order
>>> to avoid differences in handling of omitted notation.
>>>=20
>>> So I've no clue if FreebSD wants to claim that a llvm change
>>> is a requirement for using clang as the powerpc64 system compiler.
>>>=20
>>> [The issue of the distinction is submittable to llvm either way.]
>>>=20
>>> Details. . .
>>>=20
>>> For:
>>>=20
>>> .section ".toc","aw"
>>> tmpstk.L: .tc tmpstk[TC],tmpstk
>>> . . .
>>> /* Set up the stack pointer */
>>> ld %r1,tmpstk.L(%r2)
>>>=20
>>> using devel/powerpc64-gcc gets:
>>>=20
>>> # /usr/local/bin/powerpc64-unknown-freebsd12.0-gcc \ =
=
=
-c \ =
=
=
=
-x assembler-with-cpp \ =
=
=
=
-pipe \ =
=
=20
>=20
>>=20
>>> =
locore64_simplified.S
>>> locore64_simplified.S: Assembler messages:
>>> locore64_simplified.S:80: Warning: assuming @toc on symbol
>>>=20
>>> and produces (with R_PPC64_TOC16_DS for .toc):
>>>=20
>>> # /usr/local/powerpc64-freebsd/bin/objdump -r locore64_simplified.o
>>>=20
>>> locore64_simplified.o: file format elf64-powerpc-freebsd
>>>=20
>>> RELOCATION RECORDS FOR [.text]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000028 R_PPC64_REL64 __tocbase+0x0000000000008000
>>> 0000000000000046 R_PPC64_TOC16_DS .toc
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.toc]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 tmpstk
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.opd]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 .__start
>>> 0000000000000008 R_PPC64_TOC *ABS*
>>>=20
>>>=20
>>> By contrast clang is silent (cross compiler used):
>>>=20
>>> # =
/usr/obj/powerpc64vtsc_clang_kernel/powerpc.powerpc64/usr/src/tmp/usr/bin/=
cc \ =
=
-target =
powerpc64-unknown-freebsd12.0 \ =
=
=
=
--sysroot=3D/usr/obj/powerpc64vtsc_clang_kernel/powerpc.powerpc64/usr/src/=
tmp \ =
=
=
-B/usr/obj/powerpc64vtsc_clang_kernel/powerpc.powerpc64/usr/src/tmp/usr/bi=
n \ =
=20
>=20
>>=20
>>> =
-c \ =
=
=
=
-x assembler-with-cpp \ =
=
=
-pipe \ =
=
=
=
locore64_simplified.S
>>>=20
>>> and produces code with R_PPC64_ADDR16_DS for the .toc instead:
>>>=20
>>> # /usr/local/powerpc64-freebsd/bin/objdump -r locore64_simplified.o =
| more =
=
=20
>>> locore64_simplified.o: file format elf64-powerpc-freebsd
>>>=20
>>> RELOCATION RECORDS FOR [.text]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000028 R_PPC64_REL64 __tocbase+0x0000000000008000
>>> 0000000000000046 R_PPC64_ADDR16_DS .toc
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.toc]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 tmpstk
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.opd]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 .__start
>>> 0000000000000008 R_PPC64_TOC *ABS*
>>>=20
>>>=20
>>>=20
>>> But for:
>>>=20
>>> .section ".toc","aw"
>>> tmpstk.L: .tc tmpstk[TC],tmpstk
>>> . . .
>>> /* Set up the stack pointer */
>>> ld %r1,tmpstk.L@toc(%r2)
>>>=20
>>> (note the @toc notation) both compilers agree and use
>>> R_PPC64_TOC16_DS for the .toc:
>>>=20
>>> # /usr/local/bin/powerpc64-unknown-freebsd12.0-gcc \ =
=
=
-c \ =
=
=
=
-x assembler-with-cpp \ =
=
=
=
-pipe \ =
=
=20
>=20
>>=20
>>> =
locore64_simplified.S
>>>=20
>>> # /usr/local/powerpc64-freebsd/bin/objdump -r locore64_simplified.o =
| more =
=
=20
>>> locore64_simplified.o: file format elf64-powerpc-freebsd
>>>=20
>>> RELOCATION RECORDS FOR [.text]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000028 R_PPC64_REL64 __tocbase+0x0000000000008000
>>> 0000000000000046 R_PPC64_TOC16_DS .toc
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.toc]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 tmpstk
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.opd]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 .__start
>>> 0000000000000008 R_PPC64_TOC *ABS*
>>>=20
>>>=20
>>> # =
/usr/obj/powerpc64vtsc_clang_kernel/powerpc.powerpc64/usr/src/tmp/usr/bin/=
cc \ =
=
-target =
powerpc64-unknown-freebsd12.0 \ =
=
=
=
--sysroot=3D/usr/obj/powerpc64vtsc_clang_kernel/powerpc.powerpc64/usr/src/=
tmp \ =
=
=
-B/usr/obj/powerpc64vtsc_clang_kernel/powerpc.powerpc64/usr/src/tmp/usr/bi=
n \ =
=20
>=20
>>=20
>>> =
-c \ =
=
=
=
-x assembler-with-cpp \ =
=
=
-pipe \ =
=
=
=
locore64_simplified.S
>>>=20
>>> # /usr/local/powerpc64-freebsd/bin/objdump -r locore64_simplified.o =
| more =
=
=20
>>> locore64_simplified.o: file format elf64-powerpc-freebsd
>>>=20
>>> RELOCATION RECORDS FOR [.text]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000028 R_PPC64_REL64 __tocbase+0x0000000000008000
>>> 0000000000000046 R_PPC64_TOC16_DS .toc
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.toc]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 tmpstk
>>>=20
>>>=20
>>> RELOCATION RECORDS FOR [.opd]:
>>> OFFSET TYPE VALUE=20
>>> 0000000000000000 R_PPC64_ADDR64 .__start
>>> 0000000000000008 R_PPC64_TOC *ABS*
>>>=20
>>>=20
>>>=20
>>> I omitted "-f -gdwarf-2" to simplify things but with such
>>> clang complains about:
>>>=20
>>> locore64_simplified.S:36:2: warning: DWARF2 only supports one =
section per compilation unit
>>> .section ".toc","aw"
>>> ^
>>> locore64_simplified.S:47:2: warning: DWARF2 only supports one =
section per compilation unit
>>> .section ".opd","aw"
>>> ^
>>>=20
>>> (buildkernel gets such messages.)
>>>=20
>>>=20
>>> I expect I can simplify the .S code more than I have so far but
>>> I figured I'd report the discovery of the choice FreeBSD needs
>>> to make for powerpc64 for if llvm changes are to be required
>>> vs. not.
>>=20
>> The following should be a list of the places that adding @toc usage
>> would fix some things for using clang 3.9.1 to target powerpc64:
>>=20
>> # grep "@toc[^b]" =
/root/sys_typescripts/typescript_make_powerpc64vtsc_nodebug_incl_clang_xto=
olchain_kernel-amd64-host-2017-01-03:23:48:41 | more
>> /usr/src/sys/powerpc/aim/locore64.S:102: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/aim/trap_subr64.S:320: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/aim/trap_subr64.S:797: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/ofw/ofwcall64.S:104: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/ofw/ofwcall64.S:108: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/ofw/ofwcall64.S:116: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/ofw/ofwcall64.S:226: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/ofw/ofwcall64.S:228: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/ofw/ofwcall64.S:235: Warning: assuming @toc on =
symbol
>> /usr/src/sys/powerpc/powerpc/swtch64.S:153: Warning: assuming @toc on =
symbol
>>=20
>> devel/powerpc64-gcc and devel/powerpc64-binutils together happens to =
report
>> on missing @toc 's.
>>=20
>> But, of course, if some sections of code are conditionally compiled =
and
>> excluded above they would not be listed.
>>=20
>> =3D=3D=3D
>> Mark Millard
>> markmi at dsl-only.net
>=20
> _______________________________________________
> freebsd-toolchain@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
> To unsubscribe, send any mail to =
"freebsd-toolchain-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1BBC8304-F9E8-4181-A287-11DD8E73B31F>