Date: Mon, 19 Jul 2021 19:22:55 +0100 From: David Chisnall <theraven@FreeBSD.org> To: marklmi@yahoo.com Cc: toolchain@freebsd.org Subject: Re: Why is main's system clang (12.0.1-rc2) using /usr/local/bin/aarch64-unknown-freebsd14.0-ld ? (such breaks things) Message-ID: <138904BA-12EA-439D-893C-BC8C4DFADB35@FreeBSD.org> In-Reply-To: <603ADFF0-7F41-4CB0-B527-CA255586570A@yahoo.com> References: <7073D16F-4505-4948-8232-A9618DF2FE5F.ref@yahoo.com> <7073D16F-4505-4948-8232-A9618DF2FE5F@yahoo.com> <77803112-7C05-40EA-9A46-8A5EB419950C@FreeBSD.org> <3B5C7A3A-344F-4CC7-A50D-56E391D04E59@yahoo.com> <0563fd33-75bd-ff16-2d37-ce62f15fa4a9@FreeBSD.org> <EE6357AE-2F7F-4139-8DC4-668619C819E3@yahoo.com> <f0baa3d8-8a26-db1f-c0d0-f40c9bb67176@FreeBSD.org> <603ADFF0-7F41-4CB0-B527-CA255586570A@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 19 Jul 2021, at 18:00, Mark Millard via toolchain = <toolchain@freebsd.org> wrote: >=20 > [Mostly this fills in some detail rather than going in > a different direction from your notes.] Thanks. >> The clang driver will provide a *default* value for the -target = argument if one is not provided. >=20 > And it happens to be aarch64-unknown-freebsd14.0 in my context. I suspect that having the base system and ports use different vendor IDs = here would prevent any related problems. > QUOTE from https://llvm.org/docs/GoldPlugin.html > Building with link time optimization requires cooperation from the > system linker. LTO support on Linux systems is available via the > gold linker which supports LTO via plugins. This is the same > mechanism used by the GCC LTO project. >=20 > . . . The same plugin can also be used by other tools such as ar > and nm. Note that ld.bfd from binutils version 2.21.51.0.2 and > above also supports LTO via plugins. However, usage of the LLVM > gold plugin with ld.bfd is not tested and therefore not officially > supported or recommended. > END QUOTE >=20 > So if it is not using its own linker, clang presumes the plugin > is appropriate, even when it was not built/installed? Possibly > an "oddity" for a FreeBSD context but possibly a messy thing > to track. >=20 > (But the wording only allows for the "system linker" as well.) The wording here predates LLD and probably needs updating. It looks as = if the last time that file was touched was in 2018 and most of the text = is from 2013, before lld was useable: = https://github.com/llvm/llvm-project/commits/main/llvm/docs/GoldPlugin.rst= It=E2=80=99s still mostly true on non-FreeBSD platforms: On Linux, the = system linker is typically provided by GNU binutils and requires the = plugin. On macOS, the system linker is ld64 and does not require a = plugin. In both cases, the linker is maintained outside of the LLVM = project and is distributed separately. FreeBSD is quite unusual in taking the compiler and linker from the LLVM = project (though I believe there are plans for macOS to migrate from ld64 = to lld at some point). The other places that ship both tend to be = cross-compile toolchains (e.g. the Playstation SDK) and so do not use = the system linker. > For reference: >=20 > QUOTE > You should produce bitcode files from clang with the option > -flto. This flag will also cause clang to look for the gold > plugin in the lib directory under its prefix and pass the > -plugin option to ld. It will not look for an alternate > linker without -fuse-ld=3Dgold, which is why you otherwise > need gold to be the installed system linker in your path. >=20 > ar and nm also accept the -plugin option and it=E2=80=99s possible > to to install LLVMgold.so to /usr/lib/bfd-plugins for a > seamless setup. If you built your own gold, be sure to > install the ar and nm-new you built to /usr/bin. > . . . > --- command lines --- > $ clang -flto a.c -c -o a.o # <-- a.o is LLVM bitcode file > $ ar q a.a a.o # <-- a.a is an archive with LLVM = bitcode > $ clang b.c -c -o b.o # <-- b.o is native object file > $ clang -flto a.a b.o -o main # <-- link with LLVMgold plugin > END QUOTE >=20 >> but we don't install it. It should probably check for the existence = of LLVMgold.so and not pass it to the linker if it doesn't exist. >=20 > Could well be. But does that get back into looking in a > bunch of places to be sure, and to know what would be > found based on which linker was used? May be too much > of a mess to be practical. I was a bit surprised that it=E2=80=99s passed, because the default for = FreeBSD should be that the linker is lld-flavoured and so clang = shouldn=E2=80=99t need to pass it. It turns out that, for = compatibility, lld silently ignores being told to load the Gold plugin = and clang passes this argument even if it knows that the linker is LLD. >=20 >>> There is another message from me where I note that I'd figured >>> out the -fuse-ld=3Dlld hack, although I omitted "ld." in the >>> example: should have been aarch64-unknown-freebsd14.0-ld.lld . >>> I'm actively using this technique because it seems to have the >>> minimal other potential consequences. >>=20 >> If you do anything that depends on a specific linker or linker = behaviour (e.g. LTO) then specifying the linker to use is good practice, = not a 'hack'. >=20 > As I understand, that other linker is supposed to be able > to support LLVM's lto via the plugin. "This is the same > mechanism used by the GCC LTO project.=E2=80=9D Note that there is no guarantee that the linker in your path is lld or a = bfd ld, especially for Arm targets. The Arm linker, for example, does = not support LTO (I think - if it does, it doesn=E2=80=99t, neither do = some other proprietary linkers). > But: >=20 > QUOTE > You should produce bitcode files from clang with the > option -flto. This flag will also cause clang to look > for the gold plugin in the lib directory under its > prefix and pass the -plugin option to ld. It will not > look for an alternate linker without -fuse-ld=3Dgold, > which is why you otherwise need gold to be the > installed system linker in your path. > END QUOTE >=20 > But FreeBSD with -fuse-ld=3Dlld works instead of the > non-existent linker that would go with -fuse-ld=3Dgold . > This seems to be a FreeBSD mismatch with the > documentation at https://llvm.org/docs/GoldPlugin.html . -fuse-ld=3Dgold will cause LLVM to search for a linker whose name ends = ld.gold. The more I look at the docs, the more I realise that most of the docs = I=E2=80=99ve read about this are actually comments in the code. The = user facing docs are somewhat less than ideal. David
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?138904BA-12EA-439D-893C-BC8C4DFADB35>