From owner-svn-src-head@freebsd.org Wed Jan 24 22:35:03 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60B02ECA3F6; Wed, 24 Jan 2018 22:35:03 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 146587BDF9; Wed, 24 Jan 2018 22:35:03 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0AD0423CBC; Wed, 24 Jan 2018 22:35:03 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OMZ2tZ008166; Wed, 24 Jan 2018 22:35:02 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OMZ0xv008145; Wed, 24 Jan 2018 22:35:00 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242235.w0OMZ0xv008145@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 22:35:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r328381 - in head: contrib/compiler-rt/lib/builtins contrib/llvm/include/llvm/Analysis contrib/llvm/include/llvm/CodeGen contrib/llvm/include/llvm/MC contrib/llvm/include/llvm/Support c... X-SVN-Group: head X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in head: contrib/compiler-rt/lib/builtins contrib/llvm/include/llvm/Analysis contrib/llvm/include/llvm/CodeGen contrib/llvm/include/llvm/MC contrib/llvm/include/llvm/Support contrib/llvm/lib/CodeGen c... X-SVN-Commit-Revision: 328381 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 22:35:03 -0000 Author: dim Date: Wed Jan 24 22:35:00 2018 New Revision: 328381 URL: https://svnweb.freebsd.org/changeset/base/328381 Log: Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to 6.0.0 (branches/release_60 r323338). MFC after: 3 months X-MFC-With: r327952 PR: 224669 Modified: head/contrib/compiler-rt/lib/builtins/clear_cache.c head/contrib/llvm/include/llvm/Analysis/RegionInfoImpl.h head/contrib/llvm/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h head/contrib/llvm/include/llvm/MC/MCCodeView.h head/contrib/llvm/include/llvm/Support/GenericDomTreeConstruction.h head/contrib/llvm/lib/CodeGen/GlobalMerge.cpp head/contrib/llvm/lib/CodeGen/PeepholeOptimizer.cpp head/contrib/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp head/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp head/contrib/llvm/lib/CodeGen/TargetLoweringBase.cpp head/contrib/llvm/lib/Linker/IRMover.cpp head/contrib/llvm/lib/MC/MCCodeView.cpp head/contrib/llvm/lib/Target/AArch64/AArch64InstructionSelector.cpp head/contrib/llvm/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.h head/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td head/contrib/llvm/lib/Target/X86/X86ISelLowering.cpp head/contrib/llvm/lib/Transforms/Scalar/GVNHoist.cpp head/contrib/llvm/lib/Transforms/Scalar/StructurizeCFG.cpp head/contrib/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp head/contrib/llvm/tools/clang/include/clang/Basic/Attr.td head/contrib/llvm/tools/clang/include/clang/Basic/BuiltinsX86.def head/contrib/llvm/tools/clang/include/clang/Basic/DiagnosticGroups.td head/contrib/llvm/tools/clang/include/clang/Basic/TokenKinds.def head/contrib/llvm/tools/clang/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h head/contrib/llvm/tools/clang/lib/AST/DeclBase.cpp head/contrib/llvm/tools/clang/lib/AST/ODRHash.cpp head/contrib/llvm/tools/clang/lib/CodeGen/CGBuiltin.cpp head/contrib/llvm/tools/clang/lib/CodeGen/ObjectFilePCHContainerOperations.cpp head/contrib/llvm/tools/clang/lib/Frontend/InitPreprocessor.cpp head/contrib/llvm/tools/clang/lib/Lex/Lexer.cpp head/contrib/llvm/tools/clang/lib/Lex/PPCaching.cpp head/contrib/llvm/tools/clang/lib/Lex/PPLexerChange.cpp head/contrib/llvm/tools/clang/lib/Sema/Scope.cpp head/contrib/llvm/tools/clang/lib/Sema/SemaTemplateDeduction.cpp head/contrib/llvm/tools/clang/lib/Sema/SemaTemplateInstantiateDecl.cpp head/contrib/llvm/tools/clang/lib/StaticAnalyzer/Checkers/MallocChecker.cpp head/contrib/llvm/tools/clang/lib/StaticAnalyzer/Checkers/ValistChecker.cpp head/contrib/llvm/tools/lld/COFF/Driver.cpp head/contrib/llvm/tools/lld/ELF/SyntheticSections.cpp head/contrib/llvm/tools/llvm-readobj/MachODumper.cpp head/lib/clang/include/clang/Basic/Version.inc head/lib/clang/include/lld/Common/Version.inc head/lib/clang/include/llvm/Support/VCSRevision.h Directory Properties: head/contrib/compiler-rt/ (props changed) head/contrib/libc++/ (props changed) head/contrib/llvm/ (props changed) head/contrib/llvm/tools/clang/ (props changed) head/contrib/llvm/tools/lld/ (props changed) head/contrib/llvm/tools/lldb/ (props changed) Modified: head/contrib/compiler-rt/lib/builtins/clear_cache.c ============================================================================== --- head/contrib/compiler-rt/lib/builtins/clear_cache.c Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/compiler-rt/lib/builtins/clear_cache.c Wed Jan 24 22:35:00 2018 (r328381) @@ -33,6 +33,11 @@ uintptr_t GetCurrentProcess(void); #include #endif +#if defined(__OpenBSD__) && defined(__mips__) + #include + #include +#endif + #if defined(__linux__) && defined(__mips__) #include #include @@ -142,6 +147,8 @@ void __clear_cache(void *start, void *end) { #else syscall(__NR_cacheflush, start, (end_int - start_int), BCACHE); #endif +#elif defined(__mips__) && defined(__OpenBSD__) + cacheflush(start, (uintptr_t)end - (uintptr_t)start, BCACHE); #elif defined(__aarch64__) && !defined(__APPLE__) uint64_t xstart = (uint64_t)(uintptr_t) start; uint64_t xend = (uint64_t)(uintptr_t) end; @@ -156,12 +163,14 @@ void __clear_cache(void *start, void *end) { * uintptr_t in case this runs in an IPL32 environment. */ const size_t dcache_line_size = 4 << ((ctr_el0 >> 16) & 15); - for (addr = xstart; addr < xend; addr += dcache_line_size) + for (addr = xstart & ~(dcache_line_size - 1); addr < xend; + addr += dcache_line_size) __asm __volatile("dc cvau, %0" :: "r"(addr)); __asm __volatile("dsb ish"); const size_t icache_line_size = 4 << ((ctr_el0 >> 0) & 15); - for (addr = xstart; addr < xend; addr += icache_line_size) + for (addr = xstart & ~(icache_line_size - 1); addr < xend; + addr += icache_line_size) __asm __volatile("ic ivau, %0" :: "r"(addr)); __asm __volatile("isb sy"); #elif defined (__powerpc64__) Modified: head/contrib/llvm/include/llvm/Analysis/RegionInfoImpl.h ============================================================================== --- head/contrib/llvm/include/llvm/Analysis/RegionInfoImpl.h Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/include/llvm/Analysis/RegionInfoImpl.h Wed Jan 24 22:35:00 2018 (r328381) @@ -254,23 +254,23 @@ std::string RegionBase::getNameStr() const { template void RegionBase::verifyBBInRegion(BlockT *BB) const { if (!contains(BB)) - llvm_unreachable("Broken region found: enumerated BB not in region!"); + report_fatal_error("Broken region found: enumerated BB not in region!"); BlockT *entry = getEntry(), *exit = getExit(); for (BlockT *Succ : make_range(BlockTraits::child_begin(BB), BlockTraits::child_end(BB))) { if (!contains(Succ) && exit != Succ) - llvm_unreachable("Broken region found: edges leaving the region must go " - "to the exit node!"); + report_fatal_error("Broken region found: edges leaving the region must go " + "to the exit node!"); } if (entry != BB) { for (BlockT *Pred : make_range(InvBlockTraits::child_begin(BB), InvBlockTraits::child_end(BB))) { if (!contains(Pred)) - llvm_unreachable("Broken region found: edges entering the region must " - "go to the entry node!"); + report_fatal_error("Broken region found: edges entering the region must " + "go to the entry node!"); } } } @@ -557,7 +557,7 @@ void RegionInfoBase::verifyBBMap(const RegionT *R) } else { BlockT *BB = Element->template getNodeAs(); if (getRegionFor(BB) != R) - llvm_unreachable("BB map does not match region nesting"); + report_fatal_error("BB map does not match region nesting"); } } } Modified: head/contrib/llvm/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h ============================================================================== --- head/contrib/llvm/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h Wed Jan 24 22:35:00 2018 (r328381) @@ -56,7 +56,7 @@ class BaseIndexOffset { (public) int64_t &Off); /// Parses tree in Ptr for base, index, offset addresses. - static BaseIndexOffset match(SDValue Ptr, const SelectionDAG &DAG); + static BaseIndexOffset match(LSBaseSDNode *N, const SelectionDAG &DAG); }; } // end namespace llvm Modified: head/contrib/llvm/include/llvm/MC/MCCodeView.h ============================================================================== --- head/contrib/llvm/include/llvm/MC/MCCodeView.h Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/include/llvm/MC/MCCodeView.h Wed Jan 24 22:35:00 2018 (r328381) @@ -177,13 +177,7 @@ class CodeViewContext { (public) unsigned IACol); /// Retreive the function info if this is a valid function id, or nullptr. - MCCVFunctionInfo *getCVFunctionInfo(unsigned FuncId) { - if (FuncId >= Functions.size()) - return nullptr; - if (Functions[FuncId].isUnallocatedFunctionInfo()) - return nullptr; - return &Functions[FuncId]; - } + MCCVFunctionInfo *getCVFunctionInfo(unsigned FuncId); /// Saves the information from the currently parsed .cv_loc directive /// and sets CVLocSeen. When the next instruction is assembled an entry @@ -199,50 +193,22 @@ class CodeViewContext { (public) CurrentCVLoc.setIsStmt(IsStmt); CVLocSeen = true; } - void clearCVLocSeen() { CVLocSeen = false; } bool getCVLocSeen() { return CVLocSeen; } + void clearCVLocSeen() { CVLocSeen = false; } + const MCCVLoc &getCurrentCVLoc() { return CurrentCVLoc; } bool isValidCVFileNumber(unsigned FileNumber); /// \brief Add a line entry. - void addLineEntry(const MCCVLineEntry &LineEntry) { - size_t Offset = MCCVLines.size(); - auto I = MCCVLineStartStop.insert( - {LineEntry.getFunctionId(), {Offset, Offset + 1}}); - if (!I.second) - I.first->second.second = Offset + 1; - MCCVLines.push_back(LineEntry); - } + void addLineEntry(const MCCVLineEntry &LineEntry); - std::vector getFunctionLineEntries(unsigned FuncId) { - std::vector FilteredLines; + std::vector getFunctionLineEntries(unsigned FuncId); - auto I = MCCVLineStartStop.find(FuncId); - if (I != MCCVLineStartStop.end()) - for (size_t Idx = I->second.first, End = I->second.second; Idx != End; - ++Idx) - if (MCCVLines[Idx].getFunctionId() == FuncId) - FilteredLines.push_back(MCCVLines[Idx]); - return FilteredLines; - } + std::pair getLineExtent(unsigned FuncId); - std::pair getLineExtent(unsigned FuncId) { - auto I = MCCVLineStartStop.find(FuncId); - // Return an empty extent if there are no cv_locs for this function id. - if (I == MCCVLineStartStop.end()) - return {~0ULL, 0}; - return I->second; - } - - ArrayRef getLinesForExtent(size_t L, size_t R) { - if (R <= L) - return None; - if (L >= MCCVLines.size()) - return None; - return makeArrayRef(&MCCVLines[L], R - L); - } + ArrayRef getLinesForExtent(size_t L, size_t R); /// Emits a line table substream. void emitLineTableForFunction(MCObjectStreamer &OS, unsigned FuncId, Modified: head/contrib/llvm/include/llvm/Support/GenericDomTreeConstruction.h ============================================================================== --- head/contrib/llvm/include/llvm/Support/GenericDomTreeConstruction.h Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/include/llvm/Support/GenericDomTreeConstruction.h Wed Jan 24 22:35:00 2018 (r328381) @@ -628,7 +628,7 @@ struct SemiNCAInfo { DecreasingLevel> Bucket; // Queue of tree nodes sorted by level in descending order. SmallDenseSet Affected; - SmallDenseSet Visited; + SmallDenseMap Visited; SmallVector AffectedQueue; SmallVector VisitedNotAffectedQueue; }; @@ -706,7 +706,7 @@ struct SemiNCAInfo { // algorithm does not really know or use the set of roots and can make a // different (implicit) decision about which nodes within an infinite loop // becomes a root. - if (DT.isVirtualRoot(TN->getIDom())) { + if (TN && !DT.isVirtualRoot(TN->getIDom())) { DEBUG(dbgs() << "Root " << BlockNamePrinter(R) << " is not virtual root's child\n" << "The entire tree needs to be rebuilt\n"); @@ -753,14 +753,16 @@ struct SemiNCAInfo { while (!II.Bucket.empty()) { const TreeNodePtr CurrentNode = II.Bucket.top().second; + const unsigned CurrentLevel = CurrentNode->getLevel(); II.Bucket.pop(); DEBUG(dbgs() << "\tAdding to Visited and AffectedQueue: " << BlockNamePrinter(CurrentNode) << "\n"); - II.Visited.insert(CurrentNode); + + II.Visited.insert({CurrentNode, CurrentLevel}); II.AffectedQueue.push_back(CurrentNode); // Discover and collect affected successors of the current node. - VisitInsertion(DT, BUI, CurrentNode, CurrentNode->getLevel(), NCD, II); + VisitInsertion(DT, BUI, CurrentNode, CurrentLevel, NCD, II); } // Finish by updating immediate dominators and levels. @@ -772,13 +774,17 @@ struct SemiNCAInfo { const TreeNodePtr TN, const unsigned RootLevel, const TreeNodePtr NCD, InsertionInfo &II) { const unsigned NCDLevel = NCD->getLevel(); - DEBUG(dbgs() << "Visiting " << BlockNamePrinter(TN) << "\n"); + DEBUG(dbgs() << "Visiting " << BlockNamePrinter(TN) << ", RootLevel " + << RootLevel << "\n"); SmallVector Stack = {TN}; assert(TN->getBlock() && II.Visited.count(TN) && "Preconditions!"); + SmallPtrSet Processed; + do { TreeNodePtr Next = Stack.pop_back_val(); + DEBUG(dbgs() << " Next: " << BlockNamePrinter(Next) << "\n"); for (const NodePtr Succ : ChildrenGetter::Get(Next->getBlock(), BUI)) { @@ -786,19 +792,31 @@ struct SemiNCAInfo { assert(SuccTN && "Unreachable successor found at reachable insertion"); const unsigned SuccLevel = SuccTN->getLevel(); - DEBUG(dbgs() << "\tSuccessor " << BlockNamePrinter(Succ) - << ", level = " << SuccLevel << "\n"); + DEBUG(dbgs() << "\tSuccessor " << BlockNamePrinter(Succ) << ", level = " + << SuccLevel << "\n"); + // Do not process the same node multiple times. + if (Processed.count(Next) > 0) + continue; + // Succ dominated by subtree From -- not affected. // (Based on the lemma 2.5 from the second paper.) if (SuccLevel > RootLevel) { DEBUG(dbgs() << "\t\tDominated by subtree From\n"); - if (II.Visited.count(SuccTN) != 0) - continue; + if (II.Visited.count(SuccTN) != 0) { + DEBUG(dbgs() << "\t\t\talready visited at level " + << II.Visited[SuccTN] << "\n\t\t\tcurrent level " + << RootLevel << ")\n"); + // A node can be necessary to visit again if we see it again at + // a lower level than before. + if (II.Visited[SuccTN] >= RootLevel) + continue; + } + DEBUG(dbgs() << "\t\tMarking visited not affected " << BlockNamePrinter(Succ) << "\n"); - II.Visited.insert(SuccTN); + II.Visited.insert({SuccTN, RootLevel}); II.VisitedNotAffectedQueue.push_back(SuccTN); Stack.push_back(SuccTN); } else if ((SuccLevel > NCDLevel + 1) && @@ -809,6 +827,8 @@ struct SemiNCAInfo { II.Bucket.push({SuccLevel, SuccTN}); } } + + Processed.insert(Next); } while (!Stack.empty()); } @@ -920,21 +940,21 @@ struct SemiNCAInfo { const NodePtr NCDBlock = DT.findNearestCommonDominator(From, To); const TreeNodePtr NCD = DT.getNode(NCDBlock); - // To dominates From -- nothing to do. - if (ToTN == NCD) return; + // If To dominates From -- nothing to do. + if (ToTN != NCD) { + DT.DFSInfoValid = false; - DT.DFSInfoValid = false; + const TreeNodePtr ToIDom = ToTN->getIDom(); + DEBUG(dbgs() << "\tNCD " << BlockNamePrinter(NCD) << ", ToIDom " + << BlockNamePrinter(ToIDom) << "\n"); - const TreeNodePtr ToIDom = ToTN->getIDom(); - DEBUG(dbgs() << "\tNCD " << BlockNamePrinter(NCD) << ", ToIDom " - << BlockNamePrinter(ToIDom) << "\n"); - - // To remains reachable after deletion. - // (Based on the caption under Figure 4. from the second paper.) - if (FromTN != ToIDom || HasProperSupport(DT, BUI, ToTN)) - DeleteReachable(DT, BUI, FromTN, ToTN); - else - DeleteUnreachable(DT, BUI, ToTN); + // To remains reachable after deletion. + // (Based on the caption under Figure 4. from the second paper.) + if (FromTN != ToIDom || HasProperSupport(DT, BUI, ToTN)) + DeleteReachable(DT, BUI, FromTN, ToTN); + else + DeleteUnreachable(DT, BUI, ToTN); + } if (IsPostDom) UpdateRootsAfterUpdate(DT, BUI); } Modified: head/contrib/llvm/lib/CodeGen/GlobalMerge.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/GlobalMerge.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/GlobalMerge.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -577,7 +577,8 @@ bool GlobalMerge::doInitialization(Module &M) { for (auto &GV : M.globals()) { // Merge is safe for "normal" internal or external globals only if (GV.isDeclaration() || GV.isThreadLocal() || - GV.hasSection() || GV.hasImplicitSection()) + GV.hasSection() || GV.hasImplicitSection() || + GV.hasDLLExportStorageClass()) continue; // It's not safe to merge globals that may be preempted Modified: head/contrib/llvm/lib/CodeGen/PeepholeOptimizer.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/PeepholeOptimizer.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/PeepholeOptimizer.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -719,15 +719,14 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, u CurSrcPair = Pair; ValueTracker ValTracker(CurSrcPair.Reg, CurSrcPair.SubReg, *MRI, !DisableAdvCopyOpt, TII); - ValueTrackerResult Res; - bool ShouldRewrite = false; - do { - // Follow the chain of copies until we reach the top of the use-def chain - // or find a more suitable source. - Res = ValTracker.getNextSource(); + // Follow the chain of copies until we find a more suitable source, a phi + // or have to abort. + while (true) { + ValueTrackerResult Res = ValTracker.getNextSource(); + // Abort at the end of a chain (without finding a suitable source). if (!Res.isValid()) - break; + return false; // Insert the Def -> Use entry for the recently found source. ValueTrackerResult CurSrcRes = RewriteMap.lookup(CurSrcPair); @@ -763,26 +762,21 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, u if (TargetRegisterInfo::isPhysicalRegister(CurSrcPair.Reg)) return false; + // Keep following the chain if the value isn't any better yet. const TargetRegisterClass *SrcRC = MRI->getRegClass(CurSrcPair.Reg); - ShouldRewrite = TRI->shouldRewriteCopySrc(DefRC, SubReg, SrcRC, - CurSrcPair.SubReg); - } while (!ShouldRewrite); + if (!TRI->shouldRewriteCopySrc(DefRC, SubReg, SrcRC, CurSrcPair.SubReg)) + continue; - // Continue looking for new sources... - if (Res.isValid()) - continue; + // We currently cannot deal with subreg operands on PHI instructions + // (see insertPHI()). + if (PHICount > 0 && CurSrcPair.SubReg != 0) + continue; - // Do not continue searching for a new source if the there's at least - // one use-def which cannot be rewritten. - if (!ShouldRewrite) - return false; + // We found a suitable source, and are done with this chain. + break; + } } - if (PHICount >= RewritePHILimit) { - DEBUG(dbgs() << "findNextSource: PHI limit reached\n"); - return false; - } - // If we did not find a more suitable source, there is nothing to optimize. return CurSrcPair.Reg != Reg; } @@ -799,6 +793,9 @@ insertPHI(MachineRegisterInfo *MRI, const TargetInstrI assert(!SrcRegs.empty() && "No sources to create a PHI instruction?"); const TargetRegisterClass *NewRC = MRI->getRegClass(SrcRegs[0].Reg); + // NewRC is only correct if no subregisters are involved. findNextSource() + // should have rejected those cases already. + assert(SrcRegs[0].SubReg == 0 && "should not have subreg operand"); unsigned NewVR = MRI->createVirtualRegister(NewRC); MachineBasicBlock *MBB = OrigPHI->getParent(); MachineInstrBuilder MIB = BuildMI(*MBB, OrigPHI, OrigPHI->getDebugLoc(), Modified: head/contrib/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -3842,9 +3842,16 @@ bool DAGCombiner::SearchForAndLoads(SDNode *N, EVT ExtVT; if (isAndLoadExtLoad(Mask, Load, Load->getValueType(0), ExtVT) && isLegalNarrowLoad(Load, ISD::ZEXTLOAD, ExtVT)) { - // Only add this load if we can make it more narrow. - if (ExtVT.bitsLT(Load->getMemoryVT())) + + // ZEXTLOAD is already small enough. + if (Load->getExtensionType() == ISD::ZEXTLOAD && + ExtVT.bitsGE(Load->getMemoryVT())) + continue; + + // Use LE to convert equal sized loads to zext. + if (ExtVT.bitsLE(Load->getMemoryVT())) Loads.insert(Load); + continue; } return false; @@ -3899,11 +3906,13 @@ bool DAGCombiner::BackwardsPropagateMask(SDNode *N, Se if (Loads.size() == 0) return false; + DEBUG(dbgs() << "Backwards propagate AND: "; N->dump()); SDValue MaskOp = N->getOperand(1); // If it exists, fixup the single node we allow in the tree that needs // masking. if (FixupNode) { + DEBUG(dbgs() << "First, need to fix up: "; FixupNode->dump()); SDValue And = DAG.getNode(ISD::AND, SDLoc(FixupNode), FixupNode->getValueType(0), SDValue(FixupNode, 0), MaskOp); @@ -3914,14 +3923,21 @@ bool DAGCombiner::BackwardsPropagateMask(SDNode *N, Se // Narrow any constants that need it. for (auto *LogicN : NodesWithConsts) { - auto *C = cast(LogicN->getOperand(1)); - SDValue And = DAG.getNode(ISD::AND, SDLoc(C), C->getValueType(0), - SDValue(C, 0), MaskOp); - DAG.UpdateNodeOperands(LogicN, LogicN->getOperand(0), And); + SDValue Op0 = LogicN->getOperand(0); + SDValue Op1 = LogicN->getOperand(1); + + if (isa(Op0)) + std::swap(Op0, Op1); + + SDValue And = DAG.getNode(ISD::AND, SDLoc(Op1), Op1.getValueType(), + Op1, MaskOp); + + DAG.UpdateNodeOperands(LogicN, Op0, And); } // Create narrow loads. for (auto *Load : Loads) { + DEBUG(dbgs() << "Propagate AND back to: "; Load->dump()); SDValue And = DAG.getNode(ISD::AND, SDLoc(Load), Load->getValueType(0), SDValue(Load, 0), MaskOp); DAG.ReplaceAllUsesOfValueWith(SDValue(Load, 0), And); @@ -5209,7 +5225,7 @@ SDValue DAGCombiner::MatchLoadCombine(SDNode *N) { return SDValue(); // Loads must share the same base address - BaseIndexOffset Ptr = BaseIndexOffset::match(L->getBasePtr(), DAG); + BaseIndexOffset Ptr = BaseIndexOffset::match(L, DAG); int64_t ByteOffsetFromBase = 0; if (!Base) Base = Ptr; @@ -12928,7 +12944,7 @@ void DAGCombiner::getStoreMergeCandidates( StoreSDNode *St, SmallVectorImpl &StoreNodes) { // This holds the base pointer, index, and the offset in bytes from the base // pointer. - BaseIndexOffset BasePtr = BaseIndexOffset::match(St->getBasePtr(), DAG); + BaseIndexOffset BasePtr = BaseIndexOffset::match(St, DAG); EVT MemVT = St->getMemoryVT(); SDValue Val = peekThroughBitcast(St->getValue()); @@ -12949,7 +12965,7 @@ void DAGCombiner::getStoreMergeCandidates( EVT LoadVT; if (IsLoadSrc) { auto *Ld = cast(Val); - LBasePtr = BaseIndexOffset::match(Ld->getBasePtr(), DAG); + LBasePtr = BaseIndexOffset::match(Ld, DAG); LoadVT = Ld->getMemoryVT(); // Load and store should be the same type. if (MemVT != LoadVT) @@ -12968,7 +12984,7 @@ void DAGCombiner::getStoreMergeCandidates( return false; // The Load's Base Ptr must also match if (LoadSDNode *OtherLd = dyn_cast(Val)) { - auto LPtr = BaseIndexOffset::match(OtherLd->getBasePtr(), DAG); + auto LPtr = BaseIndexOffset::match(OtherLd, DAG); if (LoadVT != OtherLd->getMemoryVT()) return false; if (!(LBasePtr.equalBaseIndex(LPtr, DAG))) @@ -12992,7 +13008,7 @@ void DAGCombiner::getStoreMergeCandidates( Val.getOpcode() != ISD::EXTRACT_SUBVECTOR) return false; } - Ptr = BaseIndexOffset::match(Other->getBasePtr(), DAG); + Ptr = BaseIndexOffset::match(Other, DAG); return (BasePtr.equalBaseIndex(Ptr, DAG, Offset)); }; @@ -13365,7 +13381,7 @@ bool DAGCombiner::MergeConsecutiveStores(StoreSDNode * if (Ld->getMemoryVT() != MemVT) break; - BaseIndexOffset LdPtr = BaseIndexOffset::match(Ld->getBasePtr(), DAG); + BaseIndexOffset LdPtr = BaseIndexOffset::match(Ld, DAG); // If this is not the first ptr that we check. int64_t LdOffset = 0; if (LdBasePtr.getBase().getNode()) { @@ -17432,44 +17448,46 @@ bool DAGCombiner::isAlias(LSBaseSDNode *Op0, LSBaseSDN unsigned NumBytes1 = Op1->getMemoryVT().getStoreSize(); // Check for BaseIndexOffset matching. - BaseIndexOffset BasePtr0 = BaseIndexOffset::match(Op0->getBasePtr(), DAG); - BaseIndexOffset BasePtr1 = BaseIndexOffset::match(Op1->getBasePtr(), DAG); + BaseIndexOffset BasePtr0 = BaseIndexOffset::match(Op0, DAG); + BaseIndexOffset BasePtr1 = BaseIndexOffset::match(Op1, DAG); int64_t PtrDiff; - if (BasePtr0.equalBaseIndex(BasePtr1, DAG, PtrDiff)) - return !((NumBytes0 <= PtrDiff) || (PtrDiff + NumBytes1 <= 0)); + if (BasePtr0.getBase().getNode() && BasePtr1.getBase().getNode()) { + if (BasePtr0.equalBaseIndex(BasePtr1, DAG, PtrDiff)) + return !((NumBytes0 <= PtrDiff) || (PtrDiff + NumBytes1 <= 0)); - // If both BasePtr0 and BasePtr1 are FrameIndexes, we will not be - // able to calculate their relative offset if at least one arises - // from an alloca. However, these allocas cannot overlap and we - // can infer there is no alias. - if (auto *A = dyn_cast(BasePtr0.getBase())) - if (auto *B = dyn_cast(BasePtr1.getBase())) { - MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo(); - // If the base are the same frame index but the we couldn't find a - // constant offset, (indices are different) be conservative. - if (A != B && (!MFI.isFixedObjectIndex(A->getIndex()) || - !MFI.isFixedObjectIndex(B->getIndex()))) - return false; - } + // If both BasePtr0 and BasePtr1 are FrameIndexes, we will not be + // able to calculate their relative offset if at least one arises + // from an alloca. However, these allocas cannot overlap and we + // can infer there is no alias. + if (auto *A = dyn_cast(BasePtr0.getBase())) + if (auto *B = dyn_cast(BasePtr1.getBase())) { + MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo(); + // If the base are the same frame index but the we couldn't find a + // constant offset, (indices are different) be conservative. + if (A != B && (!MFI.isFixedObjectIndex(A->getIndex()) || + !MFI.isFixedObjectIndex(B->getIndex()))) + return false; + } - bool IsFI0 = isa(BasePtr0.getBase()); - bool IsFI1 = isa(BasePtr1.getBase()); - bool IsGV0 = isa(BasePtr0.getBase()); - bool IsGV1 = isa(BasePtr1.getBase()); - bool IsCV0 = isa(BasePtr0.getBase()); - bool IsCV1 = isa(BasePtr1.getBase()); + bool IsFI0 = isa(BasePtr0.getBase()); + bool IsFI1 = isa(BasePtr1.getBase()); + bool IsGV0 = isa(BasePtr0.getBase()); + bool IsGV1 = isa(BasePtr1.getBase()); + bool IsCV0 = isa(BasePtr0.getBase()); + bool IsCV1 = isa(BasePtr1.getBase()); - // If of mismatched base types or checkable indices we can check - // they do not alias. - if ((BasePtr0.getIndex() == BasePtr1.getIndex() || (IsFI0 != IsFI1) || - (IsGV0 != IsGV1) || (IsCV0 != IsCV1)) && - (IsFI0 || IsGV0 || IsCV0) && (IsFI1 || IsGV1 || IsCV1)) - return false; + // If of mismatched base types or checkable indices we can check + // they do not alias. + if ((BasePtr0.getIndex() == BasePtr1.getIndex() || (IsFI0 != IsFI1) || + (IsGV0 != IsGV1) || (IsCV0 != IsCV1)) && + (IsFI0 || IsGV0 || IsCV0) && (IsFI1 || IsGV1 || IsCV1)) + return false; + } - // If we know required SrcValue1 and SrcValue2 have relatively large alignment - // compared to the size and offset of the access, we may be able to prove they - // do not alias. This check is conservative for now to catch cases created by - // splitting vector types. + // If we know required SrcValue1 and SrcValue2 have relatively large + // alignment compared to the size and offset of the access, we may be able + // to prove they do not alias. This check is conservative for now to catch + // cases created by splitting vector types. int64_t SrcValOffset0 = Op0->getSrcValueOffset(); int64_t SrcValOffset1 = Op1->getSrcValueOffset(); unsigned OrigAlignment0 = Op0->getOriginalAlignment(); @@ -17479,8 +17497,8 @@ bool DAGCombiner::isAlias(LSBaseSDNode *Op0, LSBaseSDN int64_t OffAlign0 = SrcValOffset0 % OrigAlignment0; int64_t OffAlign1 = SrcValOffset1 % OrigAlignment1; - // There is no overlap between these relatively aligned accesses of similar - // size. Return no alias. + // There is no overlap between these relatively aligned accesses of + // similar size. Return no alias. if ((OffAlign0 + NumBytes0) <= OffAlign1 || (OffAlign1 + NumBytes1) <= OffAlign0) return false; @@ -17643,7 +17661,7 @@ bool DAGCombiner::findBetterNeighborChains(StoreSDNode // This holds the base pointer, index, and the offset in bytes from the base // pointer. - BaseIndexOffset BasePtr = BaseIndexOffset::match(St->getBasePtr(), DAG); + BaseIndexOffset BasePtr = BaseIndexOffset::match(St, DAG); // We must have a base and an offset. if (!BasePtr.getBase().getNode()) @@ -17669,7 +17687,7 @@ bool DAGCombiner::findBetterNeighborChains(StoreSDNode break; // Find the base pointer and offset for this memory node. - BaseIndexOffset Ptr = BaseIndexOffset::match(Index->getBasePtr(), DAG); + BaseIndexOffset Ptr = BaseIndexOffset::match(Index, DAG); // Check that the base pointer is the same as the original one. if (!BasePtr.equalBaseIndex(Ptr, DAG)) Modified: head/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -2965,12 +2965,12 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) { case ISD::ZERO_EXTEND: LHS = DAG.getNode(ISD::AssertZext, dl, OuterType, Res, DAG.getValueType(AtomicType)); - RHS = DAG.getNode(ISD::ZERO_EXTEND, dl, OuterType, Node->getOperand(2)); + RHS = DAG.getZeroExtendInReg(Node->getOperand(2), dl, AtomicType); ExtRes = LHS; break; case ISD::ANY_EXTEND: LHS = DAG.getZeroExtendInReg(Res, dl, AtomicType); - RHS = DAG.getNode(ISD::ZERO_EXTEND, dl, OuterType, Node->getOperand(2)); + RHS = DAG.getZeroExtendInReg(Node->getOperand(2), dl, AtomicType); break; default: llvm_unreachable("Invalid atomic op extension"); Modified: head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -7947,11 +7947,8 @@ bool SelectionDAG::areNonVolatileConsecutiveLoads(Load if (VT.getSizeInBits() / 8 != Bytes) return false; - SDValue Loc = LD->getOperand(1); - SDValue BaseLoc = Base->getOperand(1); - - auto BaseLocDecomp = BaseIndexOffset::match(BaseLoc, *this); - auto LocDecomp = BaseIndexOffset::match(Loc, *this); + auto BaseLocDecomp = BaseIndexOffset::match(Base, *this); + auto LocDecomp = BaseIndexOffset::match(LD, *this); int64_t Offset = 0; if (BaseLocDecomp.equalBaseIndex(LocDecomp, *this, Offset)) Modified: head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -21,6 +21,9 @@ using namespace llvm; bool BaseIndexOffset::equalBaseIndex(BaseIndexOffset &Other, const SelectionDAG &DAG, int64_t &Off) { + // Conservatively fail if we a match failed.. + if (!Base.getNode() || !Other.Base.getNode()) + return false; // Initial Offset difference. Off = Other.Offset - Offset; @@ -72,12 +75,28 @@ bool BaseIndexOffset::equalBaseIndex(BaseIndexOffset & } /// Parses tree in Ptr for base, index, offset addresses. -BaseIndexOffset BaseIndexOffset::match(SDValue Ptr, const SelectionDAG &DAG) { +BaseIndexOffset BaseIndexOffset::match(LSBaseSDNode *N, + const SelectionDAG &DAG) { + SDValue Ptr = N->getBasePtr(); + // (((B + I*M) + c)) + c ... SDValue Base = DAG.getTargetLoweringInfo().unwrapAddress(Ptr); SDValue Index = SDValue(); int64_t Offset = 0; bool IsIndexSignExt = false; + + // pre-inc/pre-dec ops are components of EA. + if (N->getAddressingMode() == ISD::PRE_INC) { + if (auto *C = dyn_cast(N->getOffset())) + Offset += C->getSExtValue(); + else // If unknown, give up now. + return BaseIndexOffset(SDValue(), SDValue(), 0, false); + } else if (N->getAddressingMode() == ISD::PRE_DEC) { + if (auto *C = dyn_cast(N->getOffset())) + Offset -= C->getSExtValue(); + else // If unknown, give up now. + return BaseIndexOffset(SDValue(), SDValue(), 0, false); + } // Consume constant adds & ors with appropriate masking. while (Base->getOpcode() == ISD::ADD || Base->getOpcode() == ISD::OR) { Modified: head/contrib/llvm/lib/CodeGen/TargetLoweringBase.cpp ============================================================================== --- head/contrib/llvm/lib/CodeGen/TargetLoweringBase.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/CodeGen/TargetLoweringBase.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -132,9 +132,18 @@ void TargetLoweringBase::InitLibcalls(const Triple &TT setLibcallName(RTLIB::FPEXT_F16_F32, "__extendhfsf2"); setLibcallName(RTLIB::FPROUND_F32_F16, "__truncsfhf2"); - // Darwin 10 and higher has an optimized __bzero. - if (!TT.isMacOSX() || !TT.isMacOSXVersionLT(10, 6) || TT.isArch64Bit()) { - setLibcallName(RTLIB::BZERO, TT.isAArch64() ? "bzero" : "__bzero"); + // Some darwins have an optimized __bzero/bzero function. + switch (TT.getArch()) { + case Triple::x86: + case Triple::x86_64: + if (TT.isMacOSX() && !TT.isMacOSXVersionLT(10, 6)) + setLibcallName(RTLIB::BZERO, "__bzero"); + break; + case Triple::aarch64: + setLibcallName(RTLIB::BZERO, "bzero"); + break; + default: + break; } if (darwinHasSinCos(TT)) { Modified: head/contrib/llvm/lib/Linker/IRMover.cpp ============================================================================== --- head/contrib/llvm/lib/Linker/IRMover.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/Linker/IRMover.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -954,7 +954,12 @@ Expected IRLinker::linkGlobalValueProto(Gl NewGV->setLinkage(GlobalValue::InternalLinkage); Constant *C = NewGV; - if (DGV) + // Only create a bitcast if necessary. In particular, with + // DebugTypeODRUniquing we may reach metadata in the destination module + // containing a GV from the source module, in which case SGV will be + // the same as DGV and NewGV, and TypeMap.get() will assert since it + // assumes it is being invoked on a type in the source module. + if (DGV && NewGV != SGV) C = ConstantExpr::getBitCast(NewGV, TypeMap.get(SGV->getType())); if (DGV && NewGV != DGV) { Modified: head/contrib/llvm/lib/MC/MCCodeView.cpp ============================================================================== --- head/contrib/llvm/lib/MC/MCCodeView.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/MC/MCCodeView.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -76,6 +76,14 @@ bool CodeViewContext::addFile(MCStreamer &OS, unsigned return true; } +MCCVFunctionInfo *CodeViewContext::getCVFunctionInfo(unsigned FuncId) { + if (FuncId >= Functions.size()) + return nullptr; + if (Functions[FuncId].isUnallocatedFunctionInfo()) + return nullptr; + return &Functions[FuncId]; +} + bool CodeViewContext::recordFunctionId(unsigned FuncId) { if (FuncId >= Functions.size()) Functions.resize(FuncId + 1); @@ -245,6 +253,67 @@ void CodeViewContext::emitFileChecksumOffset(MCObjectS MCSymbolRefExpr::create(Files[Idx].ChecksumTableOffset, OS.getContext()); OS.EmitValueImpl(SRE, 4); +} + +void CodeViewContext::addLineEntry(const MCCVLineEntry &LineEntry) { + size_t Offset = MCCVLines.size(); + auto I = MCCVLineStartStop.insert( + {LineEntry.getFunctionId(), {Offset, Offset + 1}}); + if (!I.second) + I.first->second.second = Offset + 1; + MCCVLines.push_back(LineEntry); +} + +std::vector +CodeViewContext::getFunctionLineEntries(unsigned FuncId) { + std::vector FilteredLines; + auto I = MCCVLineStartStop.find(FuncId); + if (I != MCCVLineStartStop.end()) { + MCCVFunctionInfo *SiteInfo = getCVFunctionInfo(FuncId); + for (size_t Idx = I->second.first, End = I->second.second; Idx != End; + ++Idx) { + unsigned LocationFuncId = MCCVLines[Idx].getFunctionId(); + if (LocationFuncId == FuncId) { + // This was a .cv_loc directly for FuncId, so record it. + FilteredLines.push_back(MCCVLines[Idx]); + } else { + // Check if the current location is inlined in this function. If it is, + // synthesize a statement .cv_loc at the original inlined call site. + auto I = SiteInfo->InlinedAtMap.find(LocationFuncId); + if (I != SiteInfo->InlinedAtMap.end()) { + MCCVFunctionInfo::LineInfo &IA = I->second; + // Only add the location if it differs from the previous location. + // Large inlined calls will have many .cv_loc entries and we only need + // one line table entry in the parent function. + if (FilteredLines.empty() || + FilteredLines.back().getFileNum() != IA.File || + FilteredLines.back().getLine() != IA.Line || + FilteredLines.back().getColumn() != IA.Col) { + FilteredLines.push_back(MCCVLineEntry( + MCCVLines[Idx].getLabel(), + MCCVLoc(FuncId, IA.File, IA.Line, IA.Col, false, false))); + } + } + } + } + } + return FilteredLines; +} + +std::pair CodeViewContext::getLineExtent(unsigned FuncId) { + auto I = MCCVLineStartStop.find(FuncId); + // Return an empty extent if there are no cv_locs for this function id. + if (I == MCCVLineStartStop.end()) + return {~0ULL, 0}; + return I->second; +} + +ArrayRef CodeViewContext::getLinesForExtent(size_t L, size_t R) { + if (R <= L) + return None; + if (L >= MCCVLines.size()) + return None; + return makeArrayRef(&MCCVLines[L], R - L); } void CodeViewContext::emitLineTableForFunction(MCObjectStreamer &OS, Modified: head/contrib/llvm/lib/Target/AArch64/AArch64InstructionSelector.cpp ============================================================================== --- head/contrib/llvm/lib/Target/AArch64/AArch64InstructionSelector.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/Target/AArch64/AArch64InstructionSelector.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -868,6 +868,40 @@ bool AArch64InstructionSelector::select(MachineInstr & if (OpFlags & AArch64II::MO_GOT) { I.setDesc(TII.get(AArch64::LOADgot)); I.getOperand(1).setTargetFlags(OpFlags); + } else if (TM.getCodeModel() == CodeModel::Large) { + // Materialize the global using movz/movk instructions. + unsigned MovZDstReg = MRI.createVirtualRegister(&AArch64::GPR64RegClass); + auto InsertPt = std::next(I.getIterator()); + auto MovZ = + BuildMI(MBB, InsertPt, I.getDebugLoc(), TII.get(AArch64::MOVZXi)) + .addDef(MovZDstReg); + MovZ->addOperand(MF, I.getOperand(1)); + MovZ->getOperand(1).setTargetFlags(OpFlags | AArch64II::MO_G0 | + AArch64II::MO_NC); + MovZ->addOperand(MF, MachineOperand::CreateImm(0)); + constrainSelectedInstRegOperands(*MovZ, TII, TRI, RBI); + + auto BuildMovK = [&](unsigned SrcReg, unsigned char Flags, + unsigned Offset, unsigned ForceDstReg) { + unsigned DstReg = + ForceDstReg ? ForceDstReg + : MRI.createVirtualRegister(&AArch64::GPR64RegClass); + auto MovI = BuildMI(MBB, InsertPt, MovZ->getDebugLoc(), + TII.get(AArch64::MOVKXi)) + .addDef(DstReg) + .addReg(SrcReg); + MovI->addOperand(MF, MachineOperand::CreateGA( + GV, MovZ->getOperand(1).getOffset(), Flags)); + MovI->addOperand(MF, MachineOperand::CreateImm(Offset)); + constrainSelectedInstRegOperands(*MovI, TII, TRI, RBI); + return DstReg; + }; + unsigned DstReg = BuildMovK(MovZ->getOperand(0).getReg(), + AArch64II::MO_G1 | AArch64II::MO_NC, 16, 0); + DstReg = BuildMovK(DstReg, AArch64II::MO_G2 | AArch64II::MO_NC, 32, 0); + BuildMovK(DstReg, AArch64II::MO_G3, 48, I.getOperand(0).getReg()); + I.eraseFromParent(); + return true; } else { I.setDesc(TII.get(AArch64::MOVaddr)); I.getOperand(1).setTargetFlags(OpFlags | AArch64II::MO_PAGE); Modified: head/contrib/llvm/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp ============================================================================== --- head/contrib/llvm/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -821,7 +821,6 @@ namespace llvm { MutableArrayRef NewMask, unsigned Options = None); OpRef packp(ShuffleMask SM, OpRef Va, OpRef Vb, ResultStack &Results, MutableArrayRef NewMask); - OpRef zerous(ShuffleMask SM, OpRef Va, ResultStack &Results); OpRef vmuxs(ArrayRef Bytes, OpRef Va, OpRef Vb, ResultStack &Results); OpRef vmuxp(ArrayRef Bytes, OpRef Va, OpRef Vb, @@ -1137,25 +1136,6 @@ OpRef HvxSelector::packp(ShuffleMask SM, OpRef Va, OpR } return concat(Out[0], Out[1], Results); -} - -OpRef HvxSelector::zerous(ShuffleMask SM, OpRef Va, ResultStack &Results) { - DEBUG_WITH_TYPE("isel", {dbgs() << __func__ << '\n';}); - - int VecLen = SM.Mask.size(); - SmallVector UsedBytes(VecLen); - bool HasUnused = false; - for (int I = 0; I != VecLen; ++I) { - if (SM.Mask[I] != -1) - UsedBytes[I] = 0xFF; - else - HasUnused = true; - } - if (!HasUnused) - return Va; - SDValue B = getVectorConstant(UsedBytes, SDLoc(Results.InpNode)); - Results.push(Hexagon::V6_vand, getSingleVT(MVT::i8), {Va, OpRef(B)}); - return OpRef::res(Results.top()); } OpRef HvxSelector::vmuxs(ArrayRef Bytes, OpRef Va, OpRef Vb, Modified: head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp ============================================================================== --- head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.cpp Wed Jan 24 22:35:00 2018 (r328381) @@ -142,6 +142,9 @@ PPCTargetLowering::PPCTargetLowering(const PPCTargetMa setOperationAction(ISD::BITREVERSE, MVT::i32, Legal); setOperationAction(ISD::BITREVERSE, MVT::i64, Legal); + // Sub-word ATOMIC_CMP_SWAP need to ensure that the input is zero-extended. + setOperationAction(ISD::ATOMIC_CMP_SWAP, MVT::i32, Custom); + // PowerPC has an i16 but no i8 (or i1) SEXTLOAD. for (MVT VT : MVT::integer_valuetypes()) { setLoadExtAction(ISD::SEXTLOAD, VT, MVT::i1, Promote); @@ -1154,6 +1157,8 @@ const char *PPCTargetLowering::getTargetNodeName(unsig case PPCISD::Hi: return "PPCISD::Hi"; case PPCISD::Lo: return "PPCISD::Lo"; case PPCISD::TOC_ENTRY: return "PPCISD::TOC_ENTRY"; + case PPCISD::ATOMIC_CMP_SWAP_8: return "PPCISD::ATOMIC_CMP_SWAP_8"; + case PPCISD::ATOMIC_CMP_SWAP_16: return "PPCISD::ATOMIC_CMP_SWAP_16"; case PPCISD::DYNALLOC: return "PPCISD::DYNALLOC"; case PPCISD::DYNAREAOFFSET: return "PPCISD::DYNAREAOFFSET"; case PPCISD::GlobalBaseReg: return "PPCISD::GlobalBaseReg"; @@ -8834,6 +8839,42 @@ SDValue PPCTargetLowering::LowerBSWAP(SDValue Op, Sele return Op; } +// ATOMIC_CMP_SWAP for i8/i16 needs to zero-extend its input since it will be +// compared to a value that is atomically loaded (atomic loads zero-extend). +SDValue PPCTargetLowering::LowerATOMIC_CMP_SWAP(SDValue Op, + SelectionDAG &DAG) const { + assert(Op.getOpcode() == ISD::ATOMIC_CMP_SWAP && + "Expecting an atomic compare-and-swap here."); + SDLoc dl(Op); + auto *AtomicNode = cast(Op.getNode()); + EVT MemVT = AtomicNode->getMemoryVT(); + if (MemVT.getSizeInBits() >= 32) + return Op; + + SDValue CmpOp = Op.getOperand(2); + // If this is already correctly zero-extended, leave it alone. + auto HighBits = APInt::getHighBitsSet(32, 32 - MemVT.getSizeInBits()); + if (DAG.MaskedValueIsZero(CmpOp, HighBits)) + return Op; + + // Clear the high bits of the compare operand. + unsigned MaskVal = (1 << MemVT.getSizeInBits()) - 1; + SDValue NewCmpOp = + DAG.getNode(ISD::AND, dl, MVT::i32, CmpOp, + DAG.getConstant(MaskVal, dl, MVT::i32)); + + // Replace the existing compare operand with the properly zero-extended one. + SmallVector Ops; + for (int i = 0, e = AtomicNode->getNumOperands(); i < e; i++) + Ops.push_back(AtomicNode->getOperand(i)); + Ops[2] = NewCmpOp; + MachineMemOperand *MMO = AtomicNode->getMemOperand(); + SDVTList Tys = DAG.getVTList(MVT::i32, MVT::Other); + auto NodeTy = + (MemVT == MVT::i8) ? PPCISD::ATOMIC_CMP_SWAP_8 : PPCISD::ATOMIC_CMP_SWAP_16; + return DAG.getMemIntrinsicNode(NodeTy, dl, Tys, Ops, MemVT, MMO); +} + SDValue PPCTargetLowering::LowerSIGN_EXTEND_INREG(SDValue Op, SelectionDAG &DAG) const { SDLoc dl(Op); @@ -9325,6 +9366,8 @@ SDValue PPCTargetLowering::LowerOperation(SDValue Op, return LowerREM(Op, DAG); case ISD::BSWAP: return LowerBSWAP(Op, DAG); + case ISD::ATOMIC_CMP_SWAP: + return LowerATOMIC_CMP_SWAP(Op, DAG); } } Modified: head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.h ============================================================================== --- head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.h Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/Target/PowerPC/PPCISelLowering.h Wed Jan 24 22:35:00 2018 (r328381) @@ -430,6 +430,11 @@ namespace llvm { /// The 4xf32 load used for v4i1 constants. QVLFSb, + /// ATOMIC_CMP_SWAP - the exact same as the target-independent nodes + /// except they ensure that the compare input is zero-extended for + /// sub-word versions because the atomic loads zero-extend. + ATOMIC_CMP_SWAP_8, ATOMIC_CMP_SWAP_16, + /// GPRC = TOC_ENTRY GA, TOC /// Loads the entry for GA from the TOC, where the TOC base is given by /// the last operand. @@ -955,6 +960,7 @@ namespace llvm { SDValue LowerINTRINSIC_VOID(SDValue Op, SelectionDAG &DAG) const; SDValue LowerREM(SDValue Op, SelectionDAG &DAG) const; SDValue LowerBSWAP(SDValue Op, SelectionDAG &DAG) const; + SDValue LowerATOMIC_CMP_SWAP(SDValue Op, SelectionDAG &DAG) const; SDValue LowerSCALAR_TO_VECTOR(SDValue Op, SelectionDAG &DAG) const; SDValue LowerSIGN_EXTEND_INREG(SDValue Op, SelectionDAG &DAG) const; SDValue LowerMUL(SDValue Op, SelectionDAG &DAG) const; Modified: head/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td ============================================================================== --- head/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td Wed Jan 24 22:04:16 2018 (r328380) +++ head/contrib/llvm/lib/Target/PowerPC/PPCInstrInfo.td Wed Jan 24 22:35:00 2018 (r328381) @@ -257,6 +257,13 @@ def PPCvcmp_o : SDNode<"PPCISD::VCMPo", SDT_PPCvcm def PPCcondbranch : SDNode<"PPCISD::COND_BRANCH", SDT_PPCcondbr, [SDNPHasChain, SDNPOptInGlue]>; +// PPC-specific atomic operations. +def PPCatomicCmpSwap_8 : + SDNode<"PPCISD::ATOMIC_CMP_SWAP_8", SDTAtomic3, + [SDNPHasChain, SDNPMayStore, SDNPMayLoad, SDNPMemOperand]>; +def PPCatomicCmpSwap_16 : + SDNode<"PPCISD::ATOMIC_CMP_SWAP_16", SDTAtomic3, + [SDNPHasChain, SDNPMayStore, SDNPMayLoad, SDNPMemOperand]>; def PPClbrx : SDNode<"PPCISD::LBRX", SDT_PPClbrx, [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>; *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***