From owner-svn-src-stable-10@FreeBSD.ORG Wed Nov 19 07:24:45 2014 Return-Path: Delivered-To: svn-src-stable-10@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE8F353F; Wed, 19 Nov 2014 07:24:45 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A038FA5E; Wed, 19 Nov 2014 07:24:45 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id sAJ7OjdN057175; Wed, 19 Nov 2014 07:24:45 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id sAJ7OiwV057171; Wed, 19 Nov 2014 07:24:44 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201411190724.sAJ7OiwV057171@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 19 Nov 2014 07:24:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r274696 - in stable: 10/contrib/llvm/include/llvm/CodeGen 10/contrib/llvm/lib/CodeGen/SelectionDAG 10/contrib/llvm/patches 9/contrib/llvm/include/llvm/CodeGen 9/contrib/llvm/lib/CodeGen... X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-10@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for only the 10-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Nov 2014 07:24:46 -0000 Author: dim Date: Wed Nov 19 07:24:43 2014 New Revision: 274696 URL: https://svnweb.freebsd.org/changeset/base/274696 Log: MFC r274442: Pull in r221709 from upstream llvm trunk (by Frédéric Riss): Totally forget deallocated SDNodes in SDDbgInfo. What would happen before that commit is that the SDDbgValues associated with a deallocated SDNode would be marked Invalidated, but SDDbgInfo would keep a map entry keyed by the SDNode pointer pointing to this list of invalidated SDDbgNodes. As the memory gets reused, the list might get wrongly associated with another new SDNode. As the SDDbgValues are cloned when they are transfered, this can lead to an exponential number of SDDbgValues being produced during DAGCombine like in http://llvm.org/bugs/show_bug.cgi?id=20893 Note that the previous behavior wasn't really buggy as the invalidation made sure that the SDDbgValues won't be used. This commit can be considered a memory optimization and as such is really hard to validate in a unit-test. This should fix abnormally large memory usage and resulting OOM crashes when compiling certain ports with debug information. Reported by: Dmitry Marakasov Upstream PRs: http://llvm.org/PR19031 http://llvm.org/PR20893 MFC r274483: The fix imported into llvm in r274442 contains some C++11 constructs, which gcc in base cannot handle. Replace these with C++98 equivalents. While here, add the patch for the adapted fix. Reported by: bz, kib Pointy hat to: dim Added: stable/10/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff - copied unchanged from r274483, head/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff Modified: stable/10/contrib/llvm/include/llvm/CodeGen/SelectionDAG.h stable/10/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Directory Properties: stable/10/ (props changed) Changes in other areas also in this revision: Added: stable/9/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff - copied unchanged from r274483, head/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff Modified: stable/9/contrib/llvm/include/llvm/CodeGen/SelectionDAG.h stable/9/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Directory Properties: stable/9/contrib/llvm/ (props changed) Modified: stable/10/contrib/llvm/include/llvm/CodeGen/SelectionDAG.h ============================================================================== --- stable/10/contrib/llvm/include/llvm/CodeGen/SelectionDAG.h Wed Nov 19 07:11:29 2014 (r274695) +++ stable/10/contrib/llvm/include/llvm/CodeGen/SelectionDAG.h Wed Nov 19 07:24:43 2014 (r274696) @@ -127,6 +127,10 @@ public: DbgValMap[Node].push_back(V); } + /// \brief Invalidate all DbgValues attached to the node and remove + /// it from the Node-to-DbgValues map. + void erase(const SDNode *Node); + void clear() { DbgValMap.clear(); DbgValues.clear(); Modified: stable/10/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp ============================================================================== --- stable/10/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Wed Nov 19 07:11:29 2014 (r274695) +++ stable/10/contrib/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Wed Nov 19 07:24:43 2014 (r274696) @@ -625,6 +625,15 @@ void SelectionDAG::DeleteNodeNotInCSEMap DeallocateNode(N); } +void SDDbgInfo::erase(const SDNode *Node) { + DbgValMapType::iterator I = DbgValMap.find(Node); + if (I == DbgValMap.end()) + return; + for (unsigned J = 0, N = I->second.size(); J != N; ++J) + I->second[J]->setIsInvalidated(); + DbgValMap.erase(I); +} + void SelectionDAG::DeallocateNode(SDNode *N) { if (N->OperandsNeedDelete) delete[] N->OperandList; @@ -635,10 +644,9 @@ void SelectionDAG::DeallocateNode(SDNode NodeAllocator.Deallocate(AllNodes.remove(N)); - // If any of the SDDbgValue nodes refer to this SDNode, invalidate them. - ArrayRef DbgVals = DbgInfo->getSDDbgValues(N); - for (unsigned i = 0, e = DbgVals.size(); i != e; ++i) - DbgVals[i]->setIsInvalidated(); + // If any of the SDDbgValue nodes refer to this SDNode, invalidate + // them and forget about that node. + DbgInfo->erase(N); } /// RemoveNodeFromCSEMaps - Take the specified node out of the CSE map that Copied: stable/10/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff (from r274483, head/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/10/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff Wed Nov 19 07:24:43 2014 (r274696, copy of r274483, head/contrib/llvm/patches/patch-r274442-llvm-r221709-debug-oom.diff) @@ -0,0 +1,73 @@ +Pull in r221709 from upstream llvm trunk (by Frédéric Riss): + + Totally forget deallocated SDNodes in SDDbgInfo. + + What would happen before that commit is that the SDDbgValues associated with + a deallocated SDNode would be marked Invalidated, but SDDbgInfo would keep + a map entry keyed by the SDNode pointer pointing to this list of invalidated + SDDbgNodes. As the memory gets reused, the list might get wrongly associated + with another new SDNode. As the SDDbgValues are cloned when they are transfered, + this can lead to an exponential number of SDDbgValues being produced during + DAGCombine like in http://llvm.org/bugs/show_bug.cgi?id=20893 + + Note that the previous behavior wasn't really buggy as the invalidation made + sure that the SDDbgValues won't be used. This commit can be considered a + memory optimization and as such is really hard to validate in a unit-test. + +This should fix abnormally large memory usage and resulting OOM crashes +when compiling certain ports with debug information. + +Reported by: Dmitry Marakasov +Upstream PRs: http://llvm.org/PR19031 http://llvm.org/PR20893 + +Introduced here: http://svnweb.freebsd.org/changeset/base/274442 + +Index: include/llvm/CodeGen/SelectionDAG.h +=================================================================== +--- include/llvm/CodeGen/SelectionDAG.h ++++ include/llvm/CodeGen/SelectionDAG.h +@@ -127,6 +127,10 @@ class SDDbgInfo { + DbgValMap[Node].push_back(V); + } + ++ /// \brief Invalidate all DbgValues attached to the node and remove ++ /// it from the Node-to-DbgValues map. ++ void erase(const SDNode *Node); ++ + void clear() { + DbgValMap.clear(); + DbgValues.clear(); +Index: lib/CodeGen/SelectionDAG/SelectionDAG.cpp +=================================================================== +--- lib/CodeGen/SelectionDAG/SelectionDAG.cpp ++++ lib/CodeGen/SelectionDAG/SelectionDAG.cpp +@@ -625,6 +625,15 @@ void SelectionDAG::DeleteNodeNotInCSEMaps(SDNode * + DeallocateNode(N); + } + ++void SDDbgInfo::erase(const SDNode *Node) { ++ DbgValMapType::iterator I = DbgValMap.find(Node); ++ if (I == DbgValMap.end()) ++ return; ++ for (unsigned J = 0, N = I->second.size(); J != N; ++J) ++ I->second[J]->setIsInvalidated(); ++ DbgValMap.erase(I); ++} ++ + void SelectionDAG::DeallocateNode(SDNode *N) { + if (N->OperandsNeedDelete) + delete[] N->OperandList; +@@ -635,10 +644,9 @@ void SelectionDAG::DeallocateNode(SDNode *N) { + + NodeAllocator.Deallocate(AllNodes.remove(N)); + +- // If any of the SDDbgValue nodes refer to this SDNode, invalidate them. +- ArrayRef DbgVals = DbgInfo->getSDDbgValues(N); +- for (unsigned i = 0, e = DbgVals.size(); i != e; ++i) +- DbgVals[i]->setIsInvalidated(); ++ // If any of the SDDbgValue nodes refer to this SDNode, invalidate ++ // them and forget about that node. ++ DbgInfo->erase(N); + } + + /// RemoveNodeFromCSEMaps - Take the specified node out of the CSE map that