From owner-svn-src-all@freebsd.org Wed Mar 20 20:57:14 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D0AA1535B4D; Wed, 20 Mar 2019 20:57:14 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD8FE8152A; Wed, 20 Mar 2019 20:57:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 914172626E; Wed, 20 Mar 2019 20:57:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x2KKvD68033018; Wed, 20 Mar 2019 20:57:13 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x2KKvBN3033008; Wed, 20 Mar 2019 20:57:11 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201903202057.x2KKvBN3033008@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 20 Mar 2019 20:57:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r345349 - in head: contrib/llvm/tools/lld/ELF contrib/llvm/tools/lld/docs lib/clang/include/lld/Common X-SVN-Group: head X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in head: contrib/llvm/tools/lld/ELF contrib/llvm/tools/lld/docs lib/clang/include/lld/Common X-SVN-Commit-Revision: 345349 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: BD8FE8152A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.964,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Mar 2019 20:57:14 -0000 Author: dim Date: Wed Mar 20 20:57:11 2019 New Revision: 345349 URL: https://svnweb.freebsd.org/changeset/base/345349 Log: Pull in r352826 from upstream lld trunk (by Fangrui Song): [ELF] Support --{,no-}allow-shlib-undefined Summary: In ld.bfd/gold, --no-allow-shlib-undefined is the default when linking an executable. This patch implements a check to error on undefined symbols in a shared object, if all of its DT_NEEDED entries are seen. Our approach resembles the one used in gold, achieves a good balance to be useful but not too smart (ld.bfd traces all DSOs and emulates the behavior of a dynamic linker to catch more cases). The error is issued based on the symbol table, different from undefined reference errors issued for relocations. It is most effective when there are DSOs that were not linked with -z defs (e.g. when static sanitizers runtime is used). gold has a comment that some system libraries on GNU/Linux may have spurious undefined references and thus system libraries should be excluded (https://sourceware.org/bugzilla/show_bug.cgi?id=6811). The story may have changed now but we make --allow-shlib-undefined the default for now. Its interaction with -shared can be discussed in the future. Reviewers: ruiu, grimar, pcc, espindola Reviewed By: ruiu Subscribers: joerg, emaste, arichardson, llvm-commits Differential Revision: https://reviews.llvm.org/D57385 Pull in r352943 from upstream lld trunk (by Fangrui Song): [ELF] Default to --no-allow-shlib-undefined for executables Summary: This follows the ld.bfd/gold behavior. The error check is useful as it captures a common type of ld.so undefined symbol errors as link-time errors: // a.cc => a.so (not linked with -z defs) void f(); // f is undefined void g() { f(); } // b.cc => executable with a DT_NEEDED entry on a.so void g(); int main() { g(); } // ld.so errors when g() is executed (lazy binding) or when the program is started (-z now) // symbol lookup error: ... undefined symbol: f Reviewers: ruiu, grimar, pcc, espindola Reviewed By: ruiu Subscribers: llvm-commits, emaste, arichardson Tags: #llvm Differential Revision: https://reviews.llvm.org/D57569 Together, these add support for --no-allow-shlib-undefined, and make it the default for executables, so they will fail to link if any symbols from needed shared libraries are undefined. Reported by: jbeich PR: 236062, 236141 MFC after: 1 month X-MFC-With: r344779 Modified: head/contrib/llvm/tools/lld/ELF/Config.h head/contrib/llvm/tools/lld/ELF/Driver.cpp head/contrib/llvm/tools/lld/ELF/InputFiles.cpp head/contrib/llvm/tools/lld/ELF/InputFiles.h head/contrib/llvm/tools/lld/ELF/Options.td head/contrib/llvm/tools/lld/ELF/SymbolTable.cpp head/contrib/llvm/tools/lld/ELF/SymbolTable.h head/contrib/llvm/tools/lld/ELF/Writer.cpp head/contrib/llvm/tools/lld/docs/ld.lld.1 head/lib/clang/include/lld/Common/Version.inc Modified: head/contrib/llvm/tools/lld/ELF/Config.h ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Config.h Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/Config.h Wed Mar 20 20:57:11 2019 (r345349) @@ -122,6 +122,7 @@ struct Configuration { uint64_t> CallGraphProfile; bool AllowMultipleDefinition; + bool AllowShlibUndefined; bool AndroidPackDynRelocs; bool ARMHasBlx = false; bool ARMHasMovtMovw = false; Modified: head/contrib/llvm/tools/lld/ELF/Driver.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Driver.cpp Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/Driver.cpp Wed Mar 20 20:57:11 2019 (r345349) @@ -758,6 +758,9 @@ void LinkerDriver::readConfigs(opt::InputArgList &Args Args.hasFlag(OPT_allow_multiple_definition, OPT_no_allow_multiple_definition, false) || hasZOption(Args, "muldefs"); + Config->AllowShlibUndefined = + Args.hasFlag(OPT_allow_shlib_undefined, OPT_no_allow_shlib_undefined, + Args.hasArg(OPT_shared)); Config->AuxiliaryList = args::getStrings(Args, OPT_auxiliary); Config->Bsymbolic = Args.hasArg(OPT_Bsymbolic); Config->BsymbolicFunctions = Args.hasArg(OPT_Bsymbolic_functions); Modified: head/contrib/llvm/tools/lld/ELF/InputFiles.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/InputFiles.cpp Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/InputFiles.cpp Wed Mar 20 20:57:11 2019 (r345349) @@ -865,7 +865,7 @@ SharedFile::SharedFile(MemoryBufferRef M, String // Partially parse the shared object file so that we can call // getSoName on this object. -template void SharedFile::parseSoName() { +template void SharedFile::parseDynamic() { const Elf_Shdr *DynamicSec = nullptr; const ELFFile Obj = this->getObj(); ArrayRef Sections = CHECK(Obj.sections(), this); @@ -902,12 +902,16 @@ template void SharedFile::parseSoNa ArrayRef Arr = CHECK(Obj.template getSectionContentsAsArray(DynamicSec), this); for (const Elf_Dyn &Dyn : Arr) { - if (Dyn.d_tag == DT_SONAME) { + if (Dyn.d_tag == DT_NEEDED) { uint64_t Val = Dyn.getVal(); if (Val >= this->StringTable.size()) + fatal(toString(this) + ": invalid DT_NEEDED entry"); + DtNeeded.push_back(this->StringTable.data() + Val); + } else if (Dyn.d_tag == DT_SONAME) { + uint64_t Val = Dyn.getVal(); + if (Val >= this->StringTable.size()) fatal(toString(this) + ": invalid DT_SONAME entry"); SoName = this->StringTable.data() + Val; - return; } } } @@ -975,7 +979,7 @@ uint32_t SharedFile::getAlignment(ArrayRef UINT32_MAX) ? 0 : Ret; } -// Fully parse the shared object file. This must be called after parseSoName(). +// Fully parse the shared object file. This must be called after parseDynamic(). // // This function parses symbol versions. If a DSO has version information, // the file has a ".gnu.version_d" section which contains symbol version Modified: head/contrib/llvm/tools/lld/ELF/InputFiles.h ============================================================================== --- head/contrib/llvm/tools/lld/ELF/InputFiles.h Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/InputFiles.h Wed Mar 20 20:57:11 2019 (r345349) @@ -323,6 +323,7 @@ template class SharedFile : public ELFFil public: std::vector Verdefs; + std::vector DtNeeded; std::string SoName; static bool classof(const InputFile *F) { @@ -331,7 +332,7 @@ template class SharedFile : public ELFFil SharedFile(MemoryBufferRef M, StringRef DefaultSoName); - void parseSoName(); + void parseDynamic(); void parseRest(); uint32_t getAlignment(ArrayRef Sections, const Elf_Sym &Sym); std::vector parseVerdefs(); @@ -348,6 +349,9 @@ template class SharedFile : public ELFFil // Mapping from Elf_Verdef data structures to information about Elf_Vernaux // data structures in the output file. std::map VerdefMap; + + // Used for --no-allow-shlib-undefined. + bool AllNeededIsKnown; // Used for --as-needed bool IsNeeded; Modified: head/contrib/llvm/tools/lld/ELF/Options.td ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Options.td Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/Options.td Wed Mar 20 20:57:11 2019 (r345349) @@ -63,6 +63,10 @@ defm allow_multiple_definition: B<"allow-multiple-defi "Allow multiple definitions", "Do not allow multiple definitions (default)">; +defm allow_shlib_undefined: B<"allow-shlib-undefined", + "Allow unresolved references in shared libraries (default when linking a shared library)", + "Do not allow unresolved references in shared libraries (default when linking an executable)">; + defm apply_dynamic_relocs: B<"apply-dynamic-relocs", "Apply link-time values for dynamic relocations", "Do not apply link-time values for dynamic relocations (default)">; @@ -492,12 +496,10 @@ def plugin_opt_thinlto: J<"plugin-opt=thinlto">; def plugin_opt_slash: J<"plugin-opt=/">; // Options listed below are silently ignored for now for compatibility. -def: F<"allow-shlib-undefined">; def: F<"detect-odr-violations">; def: Flag<["-"], "g">; def: F<"long-plt">; def: F<"no-add-needed">; -def: F<"no-allow-shlib-undefined">; def: F<"no-copy-dt-needed-entries">; def: F<"no-ctors-in-init-array">; def: F<"no-keep-memory">; Modified: head/contrib/llvm/tools/lld/ELF/SymbolTable.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/SymbolTable.cpp Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/SymbolTable.cpp Wed Mar 20 20:57:11 2019 (r345349) @@ -93,7 +93,7 @@ template void SymbolTable::addFile(InputF // .so file if (auto *F = dyn_cast>(File)) { // DSOs are uniquified not by filename but by soname. - F->parseSoName(); + F->parseDynamic(); if (errorCount()) return; Modified: head/contrib/llvm/tools/lld/ELF/SymbolTable.h ============================================================================== --- head/contrib/llvm/tools/lld/ELF/SymbolTable.h Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/SymbolTable.h Wed Mar 20 20:57:11 2019 (r345349) @@ -80,6 +80,9 @@ class SymbolTable { (public) void handleDynamicList(); + // Set of .so files to not link the same shared object file more than once. + llvm::DenseMap SoNames; + private: std::pair insertName(StringRef Name); @@ -106,9 +109,6 @@ class SymbolTable { (public) // same name, only one of them is linked, and the other is ignored. This set // is used to uniquify them. llvm::DenseSet ComdatGroups; - - // Set of .so files to not link the same shared object file more than once. - llvm::DenseMap SoNames; // A map from demangled symbol names to their symbol objects. // This mapping is 1:N because two symbols with different versions Modified: head/contrib/llvm/tools/lld/ELF/Writer.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Writer.cpp Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/ELF/Writer.cpp Wed Mar 20 20:57:11 2019 (r345349) @@ -1668,6 +1668,27 @@ template void Writer::finalizeSecti if (In.Iplt && !In.Iplt->empty()) In.Iplt->addSymbols(); + if (!Config->AllowShlibUndefined) { + // Error on undefined symbols in a shared object, if all of its DT_NEEDED + // entires are seen. These cases would otherwise lead to runtime errors + // reported by the dynamic linker. + // + // ld.bfd traces all DT_NEEDED to emulate the logic of the dynamic linker to + // catch more cases. That is too much for us. Our approach resembles the one + // used in ld.gold, achieves a good balance to be useful but not too smart. + for (InputFile *File : SharedFiles) { + SharedFile *F = cast>(File); + F->AllNeededIsKnown = llvm::all_of(F->DtNeeded, [&](StringRef Needed) { + return Symtab->SoNames.count(Needed); + }); + } + for (Symbol *Sym : Symtab->getSymbols()) + if (Sym->isUndefined() && !Sym->isWeak()) + if (auto *F = dyn_cast_or_null>(Sym->File)) + if (F->AllNeededIsKnown) + error(toString(F) + ": undefined reference to " + toString(*Sym)); + } + // Now that we have defined all possible global symbols including linker- // synthesized ones. Visit all symbols to give the finishing touches. for (Symbol *Sym : Symtab->getSymbols()) { Modified: head/contrib/llvm/tools/lld/docs/ld.lld.1 ============================================================================== --- head/contrib/llvm/tools/lld/docs/ld.lld.1 Wed Mar 20 20:42:44 2019 (r345348) +++ head/contrib/llvm/tools/lld/docs/ld.lld.1 Wed Mar 20 20:57:11 2019 (r345349) @@ -56,6 +56,9 @@ option. .It Fl -allow-multiple-definition Do not error if a symbol is defined multiple times. The first definition will be used. +.It Fl -allow-shlib-undefined +Allow unresolved references in shared libraries. +This option is enabled by default when linking a shared library. .It Fl -apply-dynamic-relocs Apply link-time values for dynamic relocations. .It Fl -as-needed @@ -252,6 +255,9 @@ Set target emulation. .It Fl -Map Ns = Ns Ar file , Fl M Ar file Print a link map to .Ar file . +.It Fl -no-allow-shlib-undefined +Do not allow unresolved references in shared libraries. +This option is enabled by default when linking an executable. .It Fl -no-as-needed Always set .Dv DT_NEEDED Modified: head/lib/clang/include/lld/Common/Version.inc ============================================================================== --- head/lib/clang/include/lld/Common/Version.inc Wed Mar 20 20:42:44 2019 (r345348) +++ head/lib/clang/include/lld/Common/Version.inc Wed Mar 20 20:57:11 2019 (r345349) @@ -7,4 +7,4 @@ #define LLD_REPOSITORY_STRING "FreeBSD" // - -#define LLD_REVISION_STRING "356365-1300002" +#define LLD_REVISION_STRING "356365-1300003"