From owner-svn-src-all@freebsd.org Wed Oct 23 17:52:49 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 26F93159C96; Wed, 23 Oct 2019 17:52:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46yydX66TGz4TCT; Wed, 23 Oct 2019 17:52:48 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 19504385; Wed, 23 Oct 2019 17:52:48 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x9NHqmh9078481; Wed, 23 Oct 2019 17:52:48 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x9NHqkRl078470; Wed, 23 Oct 2019 17:52:46 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201910231752.x9NHqkRl078470@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 23 Oct 2019 17:52:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r353950 - in vendor/lld/dist: . COFF Common ELF ELF/Arch docs include/lld/Common include/lld/Core include/lld/ReaderWriter lib/Driver lib/ReaderWriter lib/ReaderWriter/MachO lib/ReaderW... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/lld/dist: . COFF Common ELF ELF/Arch docs include/lld/Common include/lld/Core include/lld/ReaderWriter lib/Driver lib/ReaderWriter lib/ReaderWriter/MachO lib/ReaderWriter/YAML tools/lld X-SVN-Commit-Revision: 353950 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2019 17:52:49 -0000 Author: dim Date: Wed Oct 23 17:52:45 2019 New Revision: 353950 URL: https://svnweb.freebsd.org/changeset/base/353950 Log: Vendor import of stripped lld trunk r375505, the last commit before the upstream Subversion repository was made read-only, and the LLVM project migrated to GitHub: https://llvm.org/svn/llvm-project/lld/trunk@375505 Added: vendor/lld/dist/Common/DWARF.cpp vendor/lld/dist/ELF/ARMErrataFix.cpp vendor/lld/dist/ELF/ARMErrataFix.h vendor/lld/dist/include/lld/Common/DWARF.h Modified: vendor/lld/dist/CMakeLists.txt vendor/lld/dist/COFF/CMakeLists.txt vendor/lld/dist/COFF/Config.h vendor/lld/dist/COFF/DLL.cpp vendor/lld/dist/COFF/DebugTypes.cpp vendor/lld/dist/COFF/Driver.cpp vendor/lld/dist/COFF/Driver.h vendor/lld/dist/COFF/DriverUtils.cpp vendor/lld/dist/COFF/ICF.cpp vendor/lld/dist/COFF/InputFiles.cpp vendor/lld/dist/COFF/InputFiles.h vendor/lld/dist/COFF/LTO.cpp vendor/lld/dist/COFF/MapFile.cpp vendor/lld/dist/COFF/MinGW.cpp vendor/lld/dist/COFF/Options.td vendor/lld/dist/COFF/PDB.cpp vendor/lld/dist/COFF/PDB.h vendor/lld/dist/COFF/SymbolTable.cpp vendor/lld/dist/COFF/SymbolTable.h vendor/lld/dist/COFF/Symbols.cpp vendor/lld/dist/COFF/Symbols.h vendor/lld/dist/COFF/Writer.cpp vendor/lld/dist/Common/CMakeLists.txt vendor/lld/dist/Common/ErrorHandler.cpp vendor/lld/dist/Common/Strings.cpp vendor/lld/dist/Common/TargetOptionsCommandFlags.cpp vendor/lld/dist/ELF/AArch64ErrataFix.cpp vendor/lld/dist/ELF/Arch/AArch64.cpp vendor/lld/dist/ELF/Arch/AMDGPU.cpp vendor/lld/dist/ELF/Arch/ARM.cpp vendor/lld/dist/ELF/Arch/AVR.cpp vendor/lld/dist/ELF/Arch/Hexagon.cpp vendor/lld/dist/ELF/Arch/MSP430.cpp vendor/lld/dist/ELF/Arch/Mips.cpp vendor/lld/dist/ELF/Arch/MipsArchTree.cpp vendor/lld/dist/ELF/Arch/PPC.cpp vendor/lld/dist/ELF/Arch/PPC64.cpp vendor/lld/dist/ELF/Arch/RISCV.cpp vendor/lld/dist/ELF/Arch/SPARCV9.cpp vendor/lld/dist/ELF/Arch/X86.cpp vendor/lld/dist/ELF/Arch/X86_64.cpp vendor/lld/dist/ELF/CMakeLists.txt vendor/lld/dist/ELF/CallGraphSort.cpp vendor/lld/dist/ELF/Config.h vendor/lld/dist/ELF/DWARF.cpp vendor/lld/dist/ELF/DWARF.h vendor/lld/dist/ELF/Driver.cpp vendor/lld/dist/ELF/DriverUtils.cpp vendor/lld/dist/ELF/EhFrame.cpp vendor/lld/dist/ELF/ICF.cpp vendor/lld/dist/ELF/InputFiles.cpp vendor/lld/dist/ELF/InputFiles.h vendor/lld/dist/ELF/InputSection.cpp vendor/lld/dist/ELF/InputSection.h vendor/lld/dist/ELF/LTO.cpp vendor/lld/dist/ELF/LinkerScript.cpp vendor/lld/dist/ELF/LinkerScript.h vendor/lld/dist/ELF/MapFile.cpp vendor/lld/dist/ELF/MarkLive.cpp vendor/lld/dist/ELF/Options.td vendor/lld/dist/ELF/OutputSections.cpp vendor/lld/dist/ELF/OutputSections.h vendor/lld/dist/ELF/Relocations.cpp vendor/lld/dist/ELF/Relocations.h vendor/lld/dist/ELF/ScriptLexer.cpp vendor/lld/dist/ELF/ScriptParser.cpp vendor/lld/dist/ELF/SymbolTable.cpp vendor/lld/dist/ELF/SymbolTable.h vendor/lld/dist/ELF/Symbols.cpp vendor/lld/dist/ELF/Symbols.h vendor/lld/dist/ELF/SyntheticSections.cpp vendor/lld/dist/ELF/SyntheticSections.h vendor/lld/dist/ELF/Target.cpp vendor/lld/dist/ELF/Writer.cpp vendor/lld/dist/ELF/Writer.h vendor/lld/dist/docs/ReleaseNotes.rst vendor/lld/dist/docs/WebAssembly.rst vendor/lld/dist/docs/conf.py vendor/lld/dist/docs/ld.lld.1 vendor/lld/dist/include/lld/Common/ErrorHandler.h vendor/lld/dist/include/lld/Common/LLVM.h vendor/lld/dist/include/lld/Common/Strings.h vendor/lld/dist/include/lld/Common/TargetOptionsCommandFlags.h vendor/lld/dist/include/lld/Core/File.h vendor/lld/dist/include/lld/ReaderWriter/MachOLinkingContext.h vendor/lld/dist/lib/Driver/DarwinLdDriver.cpp vendor/lld/dist/lib/ReaderWriter/FileArchive.cpp vendor/lld/dist/lib/ReaderWriter/MachO/ArchHandler_x86_64.cpp vendor/lld/dist/lib/ReaderWriter/MachO/CompactUnwindPass.cpp vendor/lld/dist/lib/ReaderWriter/MachO/DebugInfo.h vendor/lld/dist/lib/ReaderWriter/MachO/GOTPass.cpp vendor/lld/dist/lib/ReaderWriter/MachO/LayoutPass.cpp vendor/lld/dist/lib/ReaderWriter/MachO/MachOLinkingContext.cpp vendor/lld/dist/lib/ReaderWriter/MachO/MachONormalizedFileBinaryReader.cpp vendor/lld/dist/lib/ReaderWriter/MachO/MachONormalizedFileBinaryWriter.cpp vendor/lld/dist/lib/ReaderWriter/MachO/MachONormalizedFileToAtoms.cpp vendor/lld/dist/lib/ReaderWriter/MachO/ObjCPass.cpp vendor/lld/dist/lib/ReaderWriter/MachO/ShimPass.cpp vendor/lld/dist/lib/ReaderWriter/MachO/TLVPass.cpp vendor/lld/dist/lib/ReaderWriter/YAML/ReaderWriterYAML.cpp vendor/lld/dist/tools/lld/lld.cpp Modified: vendor/lld/dist/CMakeLists.txt ============================================================================== --- vendor/lld/dist/CMakeLists.txt Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/CMakeLists.txt Wed Oct 23 17:52:45 2019 (r353950) @@ -56,7 +56,6 @@ if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR) include(HandleLLVMOptions) if(LLVM_INCLUDE_TESTS) - set(Python_ADDITIONAL_VERSIONS 2.7) include(FindPythonInterp) if(NOT PYTHONINTERP_FOUND) message(FATAL_ERROR Modified: vendor/lld/dist/COFF/CMakeLists.txt ============================================================================== --- vendor/lld/dist/COFF/CMakeLists.txt Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/CMakeLists.txt Wed Oct 23 17:52:45 2019 (r353950) @@ -28,8 +28,10 @@ add_lld_library(lldCOFF BinaryFormat Core DebugInfoCodeView + DebugInfoDWARF DebugInfoMSF DebugInfoPDB + Demangle LibDriver LTO MC Modified: vendor/lld/dist/COFF/Config.h ============================================================================== --- vendor/lld/dist/COFF/Config.h Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/Config.h Wed Oct 23 17:52:45 2019 (r353950) @@ -122,6 +122,7 @@ struct Configuration { bool dll = false; StringRef implib; std::vector exports; + bool hadExplicitExports; std::set delayLoads; std::map dllOrder; Symbol *delayLoadHelper = nullptr; @@ -189,6 +190,10 @@ struct Configuration { // Used for /thinlto-object-suffix-replace: std::pair thinLTOObjectSuffixReplace; + // Used for /lto-obj-path: + llvm::StringRef ltoObjPath; + + uint64_t align = 4096; uint64_t imageBase = -1; uint64_t fileAlign = 512; uint64_t stackReserve = 1024 * 1024; Modified: vendor/lld/dist/COFF/DLL.cpp ============================================================================== --- vendor/lld/dist/COFF/DLL.cpp Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/DLL.cpp Wed Oct 23 17:52:45 2019 (r353950) @@ -135,7 +135,7 @@ class NullChunk : public NonSectionChunk { (private) static std::vector> binImports(const std::vector &imports) { // Group DLL-imported symbols by DLL name because that's how - // symbols are layed out in the import descriptor table. + // symbols are laid out in the import descriptor table. auto less = [](const std::string &a, const std::string &b) { return config->dllOrder[a] < config->dllOrder[b]; }; @@ -188,7 +188,7 @@ class DelayDirectoryChunk : public NonSectionChunk { ( // Initial contents for delay-loaded functions. // This code calls __delayLoadHelper2 function to resolve a symbol -// and then overwrites its jump table slot with the result +// which then overwrites its jump table slot with the result // for subsequent function calls. static const uint8_t thunkX64[] = { 0x48, 0x8D, 0x05, 0, 0, 0, 0, // lea rax, [__imp_] Modified: vendor/lld/dist/COFF/DebugTypes.cpp ============================================================================== --- vendor/lld/dist/COFF/DebugTypes.cpp Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/DebugTypes.cpp Wed Oct 23 17:52:45 2019 (r353950) @@ -17,11 +17,12 @@ #include "llvm/DebugInfo/PDB/Native/PDBFile.h" #include "llvm/Support/Path.h" -using namespace lld; -using namespace lld::coff; using namespace llvm; using namespace llvm::codeview; +namespace lld { +namespace coff { + namespace { // The TypeServerSource class represents a PDB type server, a file referenced by // OBJ files compiled with MSVC /Zi. A single PDB can be shared by several OBJ @@ -96,27 +97,25 @@ TpiSource::TpiSource(TpiKind k, const ObjFile *f) : ki GC.push_back(std::unique_ptr(this)); } -TpiSource *lld::coff::makeTpiSource(const ObjFile *f) { +TpiSource *makeTpiSource(const ObjFile *f) { return new TpiSource(TpiSource::Regular, f); } -TpiSource *lld::coff::makeUseTypeServerSource(const ObjFile *f, +TpiSource *makeUseTypeServerSource(const ObjFile *f, const TypeServer2Record *ts) { TypeServerSource::enqueue(f, *ts); return new UseTypeServerSource(f, ts); } -TpiSource *lld::coff::makePrecompSource(const ObjFile *f) { +TpiSource *makePrecompSource(const ObjFile *f) { return new PrecompSource(f); } -TpiSource *lld::coff::makeUsePrecompSource(const ObjFile *f, +TpiSource *makeUsePrecompSource(const ObjFile *f, const PrecompRecord *precomp) { return new UsePrecompSource(f, precomp); } -namespace lld { -namespace coff { template <> const PrecompRecord &retrieveDependencyInfo(const TpiSource *source) { assert(source->kind == TpiSource::UsingPCH); @@ -128,8 +127,6 @@ const TypeServer2Record &retrieveDependencyInfo(const assert(source->kind == TpiSource::UsingPDB); return ((const UseTypeServerSource *)source)->typeServerDependency; } -} // namespace coff -} // namespace lld std::map> TypeServerSource::instances; @@ -210,8 +207,7 @@ TypeServerSource::findFromFile(const ObjFile *dependen // FIXME: Temporary interface until PDBLinker::maybeMergeTypeServerPDB() is // moved here. -Expected -lld::coff::findTypeServerSource(const ObjFile *f) { +Expected findTypeServerSource(const ObjFile *f) { Expected ts = TypeServerSource::findFromFile(f); if (!ts) return ts.takeError(); @@ -231,7 +227,7 @@ void TypeServerSource::enqueue(const ObjFile *dependen if (!it.second) return; // another OBJ already scheduled this PDB for load - driver->enqueuePath(*p, false); + driver->enqueuePath(*p, false, false); } // Create an instance of TypeServerSource or an error string if the PDB couldn't @@ -239,7 +235,7 @@ void TypeServerSource::enqueue(const ObjFile *dependen // will be merged in. NOTE - a PDB load failure is not a link error: some // debug info will simply be missing from the final PDB - that is the default // accepted behavior. -void lld::coff::loadTypeServerSource(llvm::MemoryBufferRef m) { +void loadTypeServerSource(llvm::MemoryBufferRef m) { std::string path = normalizePdbPath(m.getBufferIdentifier()); Expected ts = TypeServerSource::getInstance(m); @@ -266,3 +262,6 @@ Expected TypeServerSource::getInst return info.takeError(); return new TypeServerSource(m, session.release()); } + +} // namespace coff +} // namespace lld Modified: vendor/lld/dist/COFF/Driver.cpp ============================================================================== --- vendor/lld/dist/COFF/Driver.cpp Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/Driver.cpp Wed Oct 23 17:52:45 2019 (r353950) @@ -27,6 +27,7 @@ #include "llvm/ADT/Optional.h" #include "llvm/ADT/StringSwitch.h" #include "llvm/BinaryFormat/Magic.h" +#include "llvm/LTO/LTO.h" #include "llvm/Object/ArchiveWriter.h" #include "llvm/Object/COFFImportFile.h" #include "llvm/Object/COFFModuleDefinition.h" @@ -36,6 +37,7 @@ #include "llvm/Option/Option.h" #include "llvm/Support/Debug.h" #include "llvm/Support/LEB128.h" +#include "llvm/Support/MathExtras.h" #include "llvm/Support/Path.h" #include "llvm/Support/Process.h" #include "llvm/Support/TarWriter.h" @@ -62,16 +64,16 @@ LinkerDriver *driver; bool link(ArrayRef args, bool canExitEarly, raw_ostream &diag) { errorHandler().logName = args::getFilenameWithoutExe(args[0]); errorHandler().errorOS = &diag; - errorHandler().colorDiagnostics = diag.has_colors(); errorHandler().errorLimitExceededMsg = "too many errors emitted, stopping now" " (use /errorlimit:0 to see all errors)"; errorHandler().exitEarly = canExitEarly; - config = make(); + enableColors(diag.has_colors()); + config = make(); symtab = make(); - driver = make(); + driver->link(args); // Call exit() if we can to avoid calling destructors. @@ -169,7 +171,7 @@ MemoryBufferRef LinkerDriver::takeBuffer(std::unique_p } void LinkerDriver::addBuffer(std::unique_ptr mb, - bool wholeArchive) { + bool wholeArchive, bool lazy) { StringRef filename = mb->getBufferIdentifier(); MemoryBufferRef mbref = takeBuffer(std::move(mb)); @@ -184,19 +186,28 @@ void LinkerDriver::addBuffer(std::unique_ptr file = CHECK(Archive::create(mbref), filename + ": failed to parse archive"); + Archive *archive = file.get(); + make>(std::move(file)); // take ownership - for (MemoryBufferRef m : getArchiveMembers(file.get())) - addArchiveBuffer(m, "", filename, 0); + int memberIndex = 0; + for (MemoryBufferRef m : getArchiveMembers(archive)) + addArchiveBuffer(m, "", filename, memberIndex++); return; } symtab->addFile(make(mbref)); break; case file_magic::bitcode: - symtab->addFile(make(mbref, "", 0)); + if (lazy) + symtab->addFile(make(mbref)); + else + symtab->addFile(make(mbref, "", 0)); break; case file_magic::coff_object: case file_magic::coff_import_library: - symtab->addFile(make(mbref)); + if (lazy) + symtab->addFile(make(mbref)); + else + symtab->addFile(make(mbref)); break; case file_magic::pdb: loadTypeServerSource(mbref); @@ -217,7 +228,7 @@ void LinkerDriver::addBuffer(std::unique_ptr>(createFutureForFile(path)); std::string pathStr = path; @@ -237,7 +248,7 @@ void LinkerDriver::enqueuePath(StringRef path, bool wh else error(msg + "; did you mean '" + nearest + "'"); } else - driver->addBuffer(std::move(mbOrErr.first), wholeArchive); + driver->addBuffer(std::move(mbOrErr.first), wholeArchive, lazy); }); } @@ -268,13 +279,12 @@ void LinkerDriver::addArchiveBuffer(MemoryBufferRef mb } void LinkerDriver::enqueueArchiveMember(const Archive::Child &c, - StringRef symName, + const Archive::Symbol &sym, StringRef parentName) { - auto reportBufferError = [=](Error &&e, - StringRef childName) { + auto reportBufferError = [=](Error &&e, StringRef childName) { fatal("could not get the buffer for the member defining symbol " + - symName + ": " + parentName + "(" + childName + "): " + + toCOFFString(sym) + ": " + parentName + "(" + childName + "): " + toString(std::move(e))); }; @@ -285,7 +295,8 @@ void LinkerDriver::enqueueArchiveMember(const Archive: reportBufferError(mbOrErr.takeError(), check(c.getFullName())); MemoryBufferRef mb = mbOrErr.get(); enqueueTask([=]() { - driver->addArchiveBuffer(mb, symName, parentName, offsetInArchive); + driver->addArchiveBuffer(mb, toCOFFString(sym), parentName, + offsetInArchive); }); return; } @@ -293,15 +304,17 @@ void LinkerDriver::enqueueArchiveMember(const Archive: std::string childName = CHECK( c.getFullName(), "could not get the filename for the member defining symbol " + - symName); + toCOFFString(sym)); auto future = std::make_shared>( createFutureForFile(childName)); enqueueTask([=]() { auto mbOrErr = future->get(); if (mbOrErr.second) reportBufferError(errorCodeToError(mbOrErr.second), childName); - driver->addArchiveBuffer(takeBuffer(std::move(mbOrErr.first)), symName, - parentName, /* OffsetInArchive */ 0); + // Pass empty string as archive name so that the original filename is + // used as the buffer identifier. + driver->addArchiveBuffer(takeBuffer(std::move(mbOrErr.first)), + toCOFFString(sym), "", /*OffsetInArchive=*/0); }); } @@ -355,7 +368,7 @@ void LinkerDriver::parseDirectives(InputFile *file) { break; case OPT_defaultlib: if (Optional path = findLib(arg->getValue())) - enqueuePath(*path, false); + enqueuePath(*path, false, false); break; case OPT_entry: config->entry = addUndefined(mangle(arg->getValue())); @@ -590,6 +603,7 @@ static std::string createResponseFile(const opt::Input for (auto *arg : args) { switch (arg->getOption().getID()) { case OPT_linkrepro: + case OPT_reproduce: case OPT_INPUT: case OPT_defaultlib: case OPT_libpath: @@ -704,9 +718,8 @@ static std::string getImplibPath() { return out.str(); } +// The import name is calculated as follows: // -// The import name is caculated as the following: -// // | LIBRARY w/ ext | LIBRARY w/o ext | no LIBRARY // -----+----------------+---------------------+------------------ // LINK | {value} | {value}.{.dll/.exe} | {output name} @@ -987,30 +1000,37 @@ static void parsePDBAltPath(StringRef altPath) { config->pdbAltPath = buf; } -/// Check that at most one resource obj file was used. +/// Convert resource files and potentially merge input resource object +/// trees into one resource tree. /// Call after ObjFile::Instances is complete. -static void diagnoseMultipleResourceObjFiles() { - // The .rsrc$01 section in a resource obj file contains a tree description - // of resources. Merging multiple resource obj files would require merging - // the trees instead of using usual linker section merging semantics. - // Since link.exe disallows linking more than one resource obj file with - // LNK4078, mirror that. The normal use of resource files is to give the - // linker many .res files, which are then converted to a single resource obj - // file internally, so this is not a big restriction in practice. - ObjFile *resourceObjFile = nullptr; +void LinkerDriver::convertResources() { + std::vector resourceObjFiles; + for (ObjFile *f : ObjFile::instances) { - if (!f->isResourceObjFile) - continue; + if (f->isResourceObjFile()) + resourceObjFiles.push_back(f); + } - if (!resourceObjFile) { - resourceObjFile = f; - continue; - } - - error(toString(f) + + if (!config->mingw && + (resourceObjFiles.size() > 1 || + (resourceObjFiles.size() == 1 && !resources.empty()))) { + error((!resources.empty() ? "internal .obj file created from .res files" + : toString(resourceObjFiles[1])) + ": more than one resource obj file not allowed, already got " + - toString(resourceObjFile)); + toString(resourceObjFiles.front())); + return; } + + if (resources.empty() && resourceObjFiles.size() <= 1) { + // No resources to convert, and max one resource object file in + // the input. Keep that preconverted resource section as is. + for (ObjFile *f : resourceObjFiles) + f->includeResourceChunks(); + return; + } + ObjFile *f = make(convertResToCOFF(resources, resourceObjFiles)); + symtab->addFile(f); + f->includeResourceChunks(); } // In MinGW, if no symbols are chosen to be exported, then all symbols are @@ -1051,6 +1071,26 @@ void LinkerDriver::maybeExportMinGWSymbols(const opt:: }); } +// lld has a feature to create a tar file containing all input files as well as +// all command line options, so that other people can run lld again with exactly +// the same inputs. This feature is accessible via /linkrepro and /reproduce. +// +// /linkrepro and /reproduce are very similar, but /linkrepro takes a directory +// name while /reproduce takes a full path. We have /linkrepro for compatibility +// with Microsoft link.exe. +Optional getReproduceFile(const opt::InputArgList &args) { + if (auto *arg = args.getLastArg(OPT_reproduce)) + return std::string(arg->getValue()); + + if (auto *arg = args.getLastArg(OPT_linkrepro)) { + SmallString<64> path = StringRef(arg->getValue()); + sys::path::append(path, "repro.tar"); + return path.str().str(); + } + + return None; +} + void LinkerDriver::link(ArrayRef argsArr) { // Needed for LTO. InitializeAllTargetInfos(); @@ -1069,7 +1109,7 @@ void LinkerDriver::link(ArrayRef argsArr // Parse command line options. ArgParser parser; - opt::InputArgList args = parser.parseLINK(argsArr); + opt::InputArgList args = parser.parse(argsArr); // Parse and evaluate -mllvm options. std::vector v; @@ -1113,17 +1153,15 @@ void LinkerDriver::link(ArrayRef argsArr // options are handled. config->mingw = args.hasArg(OPT_lldmingw); - if (auto *arg = args.getLastArg(OPT_linkrepro)) { - SmallString<64> path = StringRef(arg->getValue()); - sys::path::append(path, "repro.tar"); - + // Handle /linkrepro and /reproduce. + if (Optional path = getReproduceFile(args)) { Expected> errOrWriter = - TarWriter::create(path, "repro"); + TarWriter::create(*path, sys::path::stem(*path)); if (errOrWriter) { tar = std::move(*errOrWriter); } else { - error("/linkrepro: failed to open " + path + ": " + + error("/linkrepro: failed to open " + *path + ": " + toString(errOrWriter.takeError())); } } @@ -1139,7 +1177,8 @@ void LinkerDriver::link(ArrayRef argsArr searchPaths.push_back(""); for (auto *arg : args.filtered(OPT_libpath)) searchPaths.push_back(arg->getValue()); - addLibSearchPaths(); + if (!args.hasArg(OPT_lldignoreenv)) + addLibSearchPaths(); // Handle /ignore for (auto *arg : args.filtered(OPT_ignore)) { @@ -1419,6 +1458,13 @@ void LinkerDriver::link(ArrayRef argsArr for (auto *arg : args.filtered(OPT_section)) parseSection(arg->getValue()); + // Handle /align + if (auto *arg = args.getLastArg(OPT_align)) { + parseNumbers(arg->getValue(), &config->align); + if (!isPowerOf2_64(config->align)) + error("/align: not a power of two: " + StringRef(arg->getValue())); + } + // Handle /aligncomm for (auto *arg : args.filtered(OPT_aligncomm)) parseAligncomm(arg->getValue()); @@ -1464,6 +1510,7 @@ void LinkerDriver::link(ArrayRef argsArr getOldNewOptions(args, OPT_thinlto_prefix_replace); config->thinLTOObjectSuffixReplace = getOldNewOptions(args, OPT_thinlto_object_suffix_replace); + config->ltoObjPath = args.getLastArgValue(OPT_lto_obj_path); // Handle miscellaneous boolean flags. config->allowBind = args.hasFlag(OPT_allowbind, OPT_allowbind_no, true); config->allowIsolation = @@ -1528,19 +1575,45 @@ void LinkerDriver::link(ArrayRef argsArr return false; }; - // Create a list of input files. Files can be given as arguments - // for /defaultlib option. - for (auto *arg : args.filtered(OPT_INPUT, OPT_wholearchive_file)) - if (Optional path = findFile(arg->getValue())) - enqueuePath(*path, isWholeArchive(*path)); + // Create a list of input files. These can be given as OPT_INPUT options + // and OPT_wholearchive_file options, and we also need to track OPT_start_lib + // and OPT_end_lib. + bool inLib = false; + for (auto *arg : args) { + switch (arg->getOption().getID()) { + case OPT_end_lib: + if (!inLib) + error("stray " + arg->getSpelling()); + inLib = false; + break; + case OPT_start_lib: + if (inLib) + error("nested " + arg->getSpelling()); + inLib = true; + break; + case OPT_wholearchive_file: + if (Optional path = findFile(arg->getValue())) + enqueuePath(*path, true, inLib); + break; + case OPT_INPUT: + if (Optional path = findFile(arg->getValue())) + enqueuePath(*path, isWholeArchive(*path), inLib); + break; + default: + // Ignore other options. + break; + } + } + // Process files specified as /defaultlib. These should be enequeued after + // other files, which is why they are in a separate loop. for (auto *arg : args.filtered(OPT_defaultlib)) if (Optional path = findLib(arg->getValue())) - enqueuePath(*path, false); + enqueuePath(*path, false, false); // Windows specific -- Create a resource file containing a manifest file. if (config->manifest == Configuration::Embed) - addBuffer(createManifestRes(), false); + addBuffer(createManifestRes(), false, false); // Read all input files given via the command line. run(); @@ -1565,12 +1638,6 @@ void LinkerDriver::link(ArrayRef argsArr for (auto *arg : args.filtered(OPT_functionpadmin, OPT_functionpadmin_opt)) parseFunctionPadMin(arg, config->machine); - // Input files can be Windows resource files (.res files). We use - // WindowsResource to convert resource files to a regular COFF file, - // then link the resulting file normally. - if (!resources.empty()) - symtab->addFile(make(convertResToCOFF(resources))); - if (tar) tar->append("response.txt", createResponseFile(args, filePaths, @@ -1746,33 +1813,24 @@ void LinkerDriver::link(ArrayRef argsArr u->weakAlias = symtab->addUndefined(to); } + // If any inputs are bitcode files, the LTO code generator may create + // references to library functions that are not explicit in the bitcode + // file's symbol table. If any of those library functions are defined in a + // bitcode file in an archive member, we need to arrange to use LTO to + // compile those archive members by adding them to the link beforehand. + if (!BitcodeFile::instances.empty()) + for (auto *s : lto::LTO::getRuntimeLibcallSymbols()) + symtab->addLibcall(s); + // Windows specific -- if __load_config_used can be resolved, resolve it. if (symtab->findUnderscore("_load_config_used")) addUndefined(mangle("_load_config_used")); } while (run()); - if (errorCount()) - return; - - // Do LTO by compiling bitcode input files to a set of native COFF files then - // link those files (unless -thinlto-index-only was given, in which case we - // resolve symbols and write indices, but don't generate native code or link). - symtab->addCombinedLTOObjects(); - - // If -thinlto-index-only is given, we should create only "index - // files" and not object files. Index file creation is already done - // in addCombinedLTOObject, so we are done if that's the case. - if (config->thinLTOIndexOnly) - return; - - // If we generated native object files from bitcode files, this resolves - // references to the symbols we use from them. - run(); - if (args.hasArg(OPT_include_optional)) { // Handle /includeoptional for (auto *arg : args.filtered(OPT_include_optional)) - if (dyn_cast_or_null(symtab->find(arg->getValue()))) + if (dyn_cast_or_null(symtab->find(arg->getValue()))) addUndefined(arg->getValue()); while (run()); } @@ -1795,11 +1853,36 @@ void LinkerDriver::link(ArrayRef argsArr run(); } - // Make sure we have resolved all symbols. - symtab->reportRemainingUndefines(); + // At this point, we should not have any symbols that cannot be resolved. + // If we are going to do codegen for link-time optimization, check for + // unresolvable symbols first, so we don't spend time generating code that + // will fail to link anyway. + if (!BitcodeFile::instances.empty() && !config->forceUnresolved) + symtab->reportUnresolvable(); if (errorCount()) return; + // Do LTO by compiling bitcode input files to a set of native COFF files then + // link those files (unless -thinlto-index-only was given, in which case we + // resolve symbols and write indices, but don't generate native code or link). + symtab->addCombinedLTOObjects(); + + // If -thinlto-index-only is given, we should create only "index + // files" and not object files. Index file creation is already done + // in addCombinedLTOObject, so we are done if that's the case. + if (config->thinLTOIndexOnly) + return; + + // If we generated native object files from bitcode files, this resolves + // references to the symbols we use from them. + run(); + + // Resolve remaining undefined symbols and warn about imported locals. + symtab->resolveRemainingUndefines(); + if (errorCount()) + return; + + config->hadExplicitExports = !config->exports.empty(); if (config->mingw) { // In MinGW, all symbols are automatically exported if no symbols // are chosen to be exported. @@ -1823,10 +1906,12 @@ void LinkerDriver::link(ArrayRef argsArr } // Windows specific -- when we are creating a .dll file, we also - // need to create a .lib file. + // need to create a .lib file. In MinGW mode, we only do that when the + // -implib option is given explicitly, for compatibility with GNU ld. if (!config->exports.empty() || config->dll) { fixupExports(); - createImportLibrary(/*asLib=*/false); + if (!config->mingw || !config->implib.empty()) + createImportLibrary(/*asLib=*/false); assignExportOrdinals(); } @@ -1870,7 +1955,7 @@ void LinkerDriver::link(ArrayRef argsArr markLive(symtab->getChunks()); // Needs to happen after the last call to addFile(). - diagnoseMultipleResourceObjFiles(); + convertResources(); // Identify identical COMDAT sections to merge them. if (config->doICF) { Modified: vendor/lld/dist/COFF/Driver.h ============================================================================== --- vendor/lld/dist/COFF/Driver.h Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/Driver.h Wed Oct 23 17:52:45 2019 (r353950) @@ -43,8 +43,8 @@ class COFFOptTable : public llvm::opt::OptTable { (pub class ArgParser { public: - // Concatenate LINK environment variable and given arguments and parse them. - llvm::opt::InputArgList parseLINK(std::vector args); + // Parses command line options. + llvm::opt::InputArgList parse(llvm::ArrayRef args); // Tokenizes a given string and then parses as command line options. llvm::opt::InputArgList parse(StringRef s) { return parse(tokenize(s)); } @@ -56,8 +56,8 @@ class ArgParser { (public) parseDirectives(StringRef s); private: - // Parses command line options. - llvm::opt::InputArgList parse(llvm::ArrayRef args); + // Concatenate LINK environment variable. + void addLINK(SmallVector &argv); std::vector tokenize(StringRef s); @@ -72,12 +72,12 @@ class LinkerDriver { (public) void parseDirectives(InputFile *file); // Used by ArchiveFile to enqueue members. - void enqueueArchiveMember(const Archive::Child &c, StringRef symName, + void enqueueArchiveMember(const Archive::Child &c, const Archive::Symbol &sym, StringRef parentName); MemoryBufferRef takeBuffer(std::unique_ptr mb); - void enqueuePath(StringRef path, bool wholeArchive); + void enqueuePath(StringRef path, bool wholeArchive, bool lazy); private: std::unique_ptr tar; // for /linkrepro @@ -98,6 +98,10 @@ class LinkerDriver { (public) // Library search path. The first element is always "" (current directory). std::vector searchPaths; + // Convert resource files and potentially merge input resource object + // trees into one resource tree. + void convertResources(); + void maybeExportMinGWSymbols(const llvm::opt::InputArgList &args); // We don't want to add the same file more than once. @@ -120,7 +124,8 @@ class LinkerDriver { (public) StringRef findDefaultEntry(); WindowsSubsystem inferSubsystem(); - void addBuffer(std::unique_ptr mb, bool wholeArchive); + void addBuffer(std::unique_ptr mb, bool wholeArchive, + bool lazy); void addArchiveBuffer(MemoryBufferRef mbref, StringRef symName, StringRef parentName, uint64_t offsetInArchive); @@ -184,7 +189,8 @@ void assignExportOrdinals(); void checkFailIfMismatch(StringRef arg, InputFile *source); // Convert Windows resource files (.res files) to a .obj file. -MemoryBufferRef convertResToCOFF(ArrayRef mbs); +MemoryBufferRef convertResToCOFF(ArrayRef mbs, + ArrayRef objs); void runMSVCLinker(std::string rsp, ArrayRef objects); Modified: vendor/lld/dist/COFF/DriverUtils.cpp ============================================================================== --- vendor/lld/dist/COFF/DriverUtils.cpp Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/DriverUtils.cpp Wed Oct 23 17:52:45 2019 (r353950) @@ -322,7 +322,7 @@ class TemporaryFile { (public) if (!contents.empty()) { std::error_code ec; - raw_fd_ostream os(path, ec, sys::fs::F_None); + raw_fd_ostream os(path, ec, sys::fs::OF_None); if (ec) fatal("failed to open " + path + ": " + ec.message()); os << contents; @@ -410,7 +410,7 @@ static std::string createManifestXmlWithExternalMt(Str // Create the default manifest file as a temporary file. TemporaryFile Default("defaultxml", "manifest"); std::error_code ec; - raw_fd_ostream os(Default.path, ec, sys::fs::F_Text); + raw_fd_ostream os(Default.path, ec, sys::fs::OF_Text); if (ec) fatal("failed to open " + Default.path + ": " + ec.message()); os << defaultXml; @@ -511,7 +511,7 @@ void createSideBySideManifest() { if (path == "") path = config->outputFile + ".manifest"; std::error_code ec; - raw_fd_ostream out(path, ec, sys::fs::F_Text); + raw_fd_ostream out(path, ec, sys::fs::OF_Text); if (ec) fatal("failed to create manifest: " + ec.message()); out << createManifestXml(); @@ -700,26 +700,42 @@ void checkFailIfMismatch(StringRef arg, InputFile *sou // Convert Windows resource files (.res files) to a .obj file. // Does what cvtres.exe does, but in-process and cross-platform. -MemoryBufferRef convertResToCOFF(ArrayRef mbs) { - object::WindowsResourceParser parser; +MemoryBufferRef convertResToCOFF(ArrayRef mbs, + ArrayRef objs) { + object::WindowsResourceParser parser(/* MinGW */ config->mingw); + std::vector duplicates; for (MemoryBufferRef mb : mbs) { std::unique_ptr bin = check(object::createBinary(mb)); object::WindowsResource *rf = dyn_cast(bin.get()); if (!rf) fatal("cannot compile non-resource file as resource"); - std::vector duplicates; if (auto ec = parser.parse(rf, duplicates)) fatal(toString(std::move(ec))); + } - for (const auto &dupeDiag : duplicates) - if (config->forceMultipleRes) - warn(dupeDiag); - else - error(dupeDiag); + // Note: This processes all .res files before all objs. Ideally they'd be + // handled in the same order they were linked (to keep the right one, if + // there are duplicates that are tolerated due to forceMultipleRes). + for (ObjFile *f : objs) { + object::ResourceSectionRef rsf; + if (auto ec = rsf.load(f->getCOFFObj())) + fatal(toString(f) + ": " + toString(std::move(ec))); + + if (auto ec = parser.parse(rsf, f->getName(), duplicates)) + fatal(toString(std::move(ec))); } + if (config->mingw) + parser.cleanUpManifests(duplicates); + + for (const auto &dupeDiag : duplicates) + if (config->forceMultipleRes) + warn(dupeDiag); + else + error(dupeDiag); + Expected> e = llvm::object::writeWindowsResourceCOFF(config->machine, parser, config->timestamp); @@ -757,15 +773,15 @@ static void handleColorDiagnostics(opt::InputArgList & if (!arg) return; if (arg->getOption().getID() == OPT_color_diagnostics) { - errorHandler().colorDiagnostics = true; + enableColors(true); } else if (arg->getOption().getID() == OPT_no_color_diagnostics) { - errorHandler().colorDiagnostics = false; + enableColors(false); } else { StringRef s = arg->getValue(); if (s == "always") - errorHandler().colorDiagnostics = true; + enableColors(true); else if (s == "never") - errorHandler().colorDiagnostics = false; + enableColors(false); else if (s != "auto") error("unknown option: --color-diagnostics=" + s); } @@ -792,13 +808,17 @@ opt::InputArgList ArgParser::parse(ArrayRef) - // and then parse the argument again. + + // Expand response files (arguments in the form of @) and insert + // flags from %LINK% and %_LINK_%, and then parse the argument again. SmallVector expandedArgv(argv.data(), argv.data() + argv.size()); + if (!args.hasArg(OPT_lldignoreenv)) + addLINK(expandedArgv); cl::ExpandResponseFiles(saver, getQuotingStyle(args), expandedArgv); args = table.ParseArgs(makeArrayRef(expandedArgv).drop_front(), missingIndex, missingCount); @@ -868,7 +888,7 @@ ArgParser::parseDirectives(StringRef s) { // link.exe has an interesting feature. If LINK or _LINK_ environment // variables exist, their contents are handled as command line strings. // So you can pass extra arguments using them. -opt::InputArgList ArgParser::parseLINK(std::vector argv) { +void ArgParser::addLINK(SmallVector &argv) { // Concatenate LINK env and command line arguments, and then parse them. if (Optional s = Process::GetEnv("LINK")) { std::vector v = tokenize(*s); @@ -878,7 +898,6 @@ opt::InputArgList ArgParser::parseLINK(std::vector v = tokenize(*s); argv.insert(std::next(argv.begin()), v.begin(), v.end()); } - return parse(argv); } std::vector ArgParser::tokenize(StringRef s) { Modified: vendor/lld/dist/COFF/ICF.cpp ============================================================================== --- vendor/lld/dist/COFF/ICF.cpp Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/ICF.cpp Wed Oct 23 17:52:45 2019 (r353950) @@ -13,7 +13,7 @@ // // On Windows, ICF is enabled by default. // -// See ELF/ICF.cpp for the details about the algortihm. +// See ELF/ICF.cpp for the details about the algorithm. // //===----------------------------------------------------------------------===// @@ -77,7 +77,7 @@ class ICF { (private) // section is insignificant to the user program and the behaviour matches that // of the Visual C++ linker. bool ICF::isEligible(SectionChunk *c) { - // Non-comdat chunks, dead chunks, and writable chunks are not elegible. + // Non-comdat chunks, dead chunks, and writable chunks are not eligible. bool writable = c->getOutputCharacteristics() & llvm::COFF::IMAGE_SCN_MEM_WRITE; if (!c->isCOMDAT() || !c->live || writable) return false; @@ -274,7 +274,7 @@ void ICF::run(ArrayRef vec) { for (Symbol *b : sc->symbols()) if (auto *sym = dyn_cast_or_null(b)) hash += sym->getChunk()->eqClass[cnt % 2]; - // Set MSB to 1 to avoid collisions with non-hash classs. + // Set MSB to 1 to avoid collisions with non-hash classes. sc->eqClass[(cnt + 1) % 2] = hash | (1U << 31); }); } @@ -297,7 +297,7 @@ void ICF::run(ArrayRef vec) { log("ICF needed " + Twine(cnt) + " iterations"); - // Merge sections in the same classs. + // Merge sections in the same classes. forEachClass([&](size_t begin, size_t end) { if (end - begin == 1) return; Modified: vendor/lld/dist/COFF/InputFiles.cpp ============================================================================== --- vendor/lld/dist/COFF/InputFiles.cpp Wed Oct 23 17:52:41 2019 (r353949) +++ vendor/lld/dist/COFF/InputFiles.cpp Wed Oct 23 17:52:45 2019 (r353950) @@ -47,6 +47,24 @@ using llvm::Triple; using llvm::support::ulittle32_t; namespace lld { + +// Returns the last element of a path, which is supposed to be a filename. +static StringRef getBasename(StringRef path) { + return sys::path::filename(path, sys::path::Style::windows); +} + +// Returns a string in the format of "foo.obj" or "foo.obj(bar.lib)". +std::string toString(const coff::InputFile *file) { + if (!file) + return ""; + if (file->parentName.empty() || file->kind() == coff::InputFile::ImportKind) + return file->getName(); + + return (getBasename(file->parentName) + "(" + getBasename(file->getName()) + + ")") + .str(); +} + namespace coff { std::vector ObjFile::instances; @@ -73,6 +91,10 @@ static void checkAndSetWeakAlias(SymbolTable *symtab, } } +static bool ignoredSymbolName(StringRef name) { + return name == "@feat.00" || name == "@comp.id"; +} + ArchiveFile::ArchiveFile(MemoryBufferRef m) : InputFile(ArchiveKind, m) {} void ArchiveFile::parse() { @@ -81,20 +103,20 @@ void ArchiveFile::parse() { // Read the symbol table to construct Lazy objects. for (const Archive::Symbol &sym : file->symbols()) - symtab->addLazy(this, sym); + symtab->addLazyArchive(this, sym); } // Returns a buffer pointing to a member file containing a given symbol. -void ArchiveFile::addMember(const Archive::Symbol *sym) { +void ArchiveFile::addMember(const Archive::Symbol &sym) { const Archive::Child &c = - CHECK(sym->getMember(), - "could not get the member for symbol " + sym->getName()); + CHECK(sym.getMember(), + "could not get the member for symbol " + toCOFFString(sym)); // Return an empty buffer if we have already returned the same buffer. if (!seen.insert(c.getChildOffset()).second) return; - driver->enqueueArchiveMember(c, sym->getName(), getName()); + driver->enqueueArchiveMember(c, sym, getName()); } std::vector getArchiveMembers(Archive *file) { @@ -116,6 +138,49 @@ std::vector getArchiveMembers(Archive return v; } +void LazyObjFile::fetch() { + if (mb.getBuffer().empty()) + return; + + InputFile *file; + if (isBitcode(mb)) + file = make(mb, "", 0, std::move(symbols)); + else + file = make(mb, std::move(symbols)); + mb = {}; + symtab->addFile(file); +} + +void LazyObjFile::parse() { + if (isBitcode(this->mb)) { + // Bitcode file. + std::unique_ptr obj = + CHECK(lto::InputFile::create(this->mb), this); + for (const lto::InputFile::Symbol &sym : obj->symbols()) { + if (!sym.isUndefined()) + symtab->addLazyObject(this, sym.getName()); + } + return; + } + + // Native object file. + std::unique_ptr coffObjPtr = CHECK(createBinary(mb), this); + COFFObjectFile *coffObj = cast(coffObjPtr.get()); + uint32_t numSymbols = coffObj->getNumberOfSymbols(); + for (uint32_t i = 0; i < numSymbols; ++i) { + COFFSymbolRef coffSym = check(coffObj->getSymbol(i)); + if (coffSym.isUndefined() || !coffSym.isExternal() || + coffSym.isWeakExternal()) + continue; + StringRef name; *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***