From nobody Tue Oct  8 08:45:24 2024
X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XN8jc6094z5Z9Qm;
	Tue, 08 Oct 2024 08:45:24 +0000 (UTC)
	(envelope-from git@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4XN8jc4lMQz4GWk;
	Tue,  8 Oct 2024 08:45:24 +0000 (UTC)
	(envelope-from git@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1728377124;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=XITcKLtgo74SYZgyFcqtw7pR1RrFlTK8SXbjC1Y4Ir4=;
	b=kRVcOpX4F9z56Sh/ZK552Cb/mLei03gmGEfNM5roseaOy8Sp8pdqvy1hPSPCs14STx4MmN
	UWuiw2C+1GW0GAJxWd2qrFS2ec0ys0C2Eslk7nO9LIHwTO7i2iCMBxFgpR3cgJzZ5EfFs5
	hDEAoqn/mm5/M7q1652UZ85pZwlIpIGQhV4dRTsx/+f2aZKNJYBKPrkjDhiTJYDyUSKd7h
	jAVyA63Y7wxmIT+vjcb/yxMPsRDfaLgz2pImv3tObZEiodjDquh0My+Rm2G4FgL+WumnUW
	pEpIhqqD0/jOgwKNbaaJVYADfCUmNRoHxy/WUCRVMHl29STIRv0R1PNdyf+mPQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1728377124;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=XITcKLtgo74SYZgyFcqtw7pR1RrFlTK8SXbjC1Y4Ir4=;
	b=evCgUopDnlBMMkkKJ+tH20ZejSU9OhRLPHIvzBmAqDpLM1lt+aXEC2Hta2gNa1MfzBGmMq
	g788sV9SFr0heGfvmcp1lunpi6o2GVXB1VmA0xxs/2FRc/xlEwJTDFTTjwT+fuSxFAtb5Z
	Y/B0a354quv6hDMeB3xLd7WFiRFRYHOlEfqvchjFrl9/Ysdgx0cTHqyIQnZSJnsTzT8GyW
	WgZ4ZCHrB9bfVTSFAhyM3ReZ7YPoGB2Y1MrpyBkfsbhsfksxmhP3wIIDF/gC4RCB2hYtdQ
	MhUcdzqWijF16dzV4FI5SbV4qzT/5MKIboJGoj3gr2tjZfFg8Hj6kysKscobfA==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1728377124; a=rsa-sha256; cv=none;
	b=UVn0RXHUv0dC/PXYucSpv9K+MuIqb5KWtsXkFba9xwcslsiYkB43hBz1Fa8GjVel/SRPks
	//ePqUK2ur+Ujj482PoaKs+0fMN/uC2HDnXh1VJFlHmPc8D1KNfFLI4codU30VP6GSyCmj
	tLML+l+m57hbjTqgBrYruIp6lQWjM/rNSBEZ6geuJCQYKKmTaMAimVnw67LojI+T+R1ySp
	1rAyEp8rv/910yeEVOkgK+GpdYLJ9hliX9D9xGM71yeyA7bwZ3GFFYCllv5AjtKlq3ulXp
	G6Q5adWHiBhXbM2jxmCN0X9q58HrWrFf72QJ4whygdotY4EFCCh8QfiiP2X7Kw==
Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
	(Client did not present a certificate)
	by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XN8jc43vqzNyf;
	Tue,  8 Oct 2024 08:45:24 +0000 (UTC)
	(envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org ([127.0.1.44])
	by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4988jOsa021610;
	Tue, 8 Oct 2024 08:45:24 GMT
	(envelope-from git@gitrepo.freebsd.org)
Received: (from git@localhost)
	by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4988jOtN021607;
	Tue, 8 Oct 2024 08:45:24 GMT
	(envelope-from git)
Date: Tue, 8 Oct 2024 08:45:24 GMT
Message-Id: <202410080845.4988jOtN021607@gitrepo.freebsd.org>
To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org,
        dev-commits-src-branches@FreeBSD.org
From: Dimitry Andric <dim@FreeBSD.org>
Subject: git: a492ec843f84 - stable/13 - Merge commit b84d773fd004
  from llvm git (by Fangrui Song):
List-Id: Commit messages for all branches of the src repository <dev-commits-src-all.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all
List-Help: <mailto:dev-commits-src-all+help@freebsd.org>
List-Post: <mailto:dev-commits-src-all@freebsd.org>
List-Subscribe: <mailto:dev-commits-src-all+subscribe@freebsd.org>
List-Unsubscribe: <mailto:dev-commits-src-all+unsubscribe@freebsd.org>
X-BeenThere: dev-commits-src-all@freebsd.org
Sender: owner-dev-commits-src-all@FreeBSD.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Git-Committer: dim
X-Git-Repository: src
X-Git-Refname: refs/heads/stable/13
X-Git-Reftype: branch
X-Git-Commit: a492ec843f84caef3668b9467dbcf718eba5063b
Auto-Submitted: auto-generated

The branch stable/13 has been updated by dim:

URL: https://cgit.FreeBSD.org/src/commit/?id=a492ec843f84caef3668b9467dbcf718eba5063b

commit a492ec843f84caef3668b9467dbcf718eba5063b
Author:     Dimitry Andric <dim@FreeBSD.org>
AuthorDate: 2024-09-22 17:08:47 +0000
Commit:     Dimitry Andric <dim@FreeBSD.org>
CommitDate: 2024-10-08 08:37:09 +0000

    Merge commit b84d773fd004 from llvm git (by Fangrui Song):
    
      [Parallel] Revert sequential task changes
    
      https://reviews.llvm.org/D148728 introduced `bool Sequential` to unify
      `execute` and the old `spawn` without argument. However, sequential
      tasks might be executed by any worker thread (non-deterministic),
      leading to non-determinism output for ld.lld -z nocombreloc (see
      https://reviews.llvm.org/D133003).
    
      In addition, the extra member variables have overhead.
      This sequential task has only been used for lld parallel relocation
      scanning.
    
      This patch restores the behavior before https://reviews.llvm.org/D148728 .
    
      Fix #105958
    
      Pull Request: https://github.com/llvm/llvm-project/pull/109084
    
    This fixes the non-reproducibility we had noticed when linking our EFI
    loaders, and for which we committed a workaround in f5ce3f4ef562.
    
    MFC after:      3 days
    
    (cherry picked from commit 54521a2ff93ae06c95c31f79f89dc23c9b51c20b)
---
 contrib/llvm-project/lld/ELF/Relocations.cpp       | 56 ++++++++++++++--------
 .../llvm/include/llvm/Support/Parallel.h           |  4 +-
 contrib/llvm-project/llvm/lib/Support/Parallel.cpp | 49 +++++--------------
 3 files changed, 48 insertions(+), 61 deletions(-)

diff --git a/contrib/llvm-project/lld/ELF/Relocations.cpp b/contrib/llvm-project/lld/ELF/Relocations.cpp
index 92a1b9baaca3..8cb01b63d2cc 100644
--- a/contrib/llvm-project/lld/ELF/Relocations.cpp
+++ b/contrib/llvm-project/lld/ELF/Relocations.cpp
@@ -1584,30 +1584,44 @@ template <class ELFT> void elf::scanRelocations() {
   bool serial = !config->zCombreloc || config->emachine == EM_MIPS ||
                 config->emachine == EM_PPC64;
   parallel::TaskGroup tg;
-  for (ELFFileBase *f : ctx.objectFiles) {
-    auto fn = [f]() {
+  auto outerFn = [&]() {
+    for (ELFFileBase *f : ctx.objectFiles) {
+      auto fn = [f]() {
+        RelocationScanner scanner;
+        for (InputSectionBase *s : f->getSections()) {
+          if (s && s->kind() == SectionBase::Regular && s->isLive() &&
+              (s->flags & SHF_ALLOC) &&
+              !(s->type == SHT_ARM_EXIDX && config->emachine == EM_ARM))
+            scanner.template scanSection<ELFT>(*s);
+        }
+      };
+      if (serial)
+        fn();
+      else
+        tg.spawn(fn);
+    }
+    auto scanEH = [] {
       RelocationScanner scanner;
-      for (InputSectionBase *s : f->getSections()) {
-        if (s && s->kind() == SectionBase::Regular && s->isLive() &&
-            (s->flags & SHF_ALLOC) &&
-            !(s->type == SHT_ARM_EXIDX && config->emachine == EM_ARM))
-          scanner.template scanSection<ELFT>(*s);
+      for (Partition &part : partitions) {
+        for (EhInputSection *sec : part.ehFrame->sections)
+          scanner.template scanSection<ELFT>(*sec);
+        if (part.armExidx && part.armExidx->isLive())
+          for (InputSection *sec : part.armExidx->exidxSections)
+            if (sec->isLive())
+              scanner.template scanSection<ELFT>(*sec);
       }
     };
-    tg.spawn(fn, serial);
-  }
-
-  tg.spawn([] {
-    RelocationScanner scanner;
-    for (Partition &part : partitions) {
-      for (EhInputSection *sec : part.ehFrame->sections)
-        scanner.template scanSection<ELFT>(*sec);
-      if (part.armExidx && part.armExidx->isLive())
-        for (InputSection *sec : part.armExidx->exidxSections)
-          if (sec->isLive())
-            scanner.template scanSection<ELFT>(*sec);
-    }
-  });
+    if (serial)
+      scanEH();
+    else
+      tg.spawn(scanEH);
+  };
+  // If `serial` is true, call `spawn` to ensure that `scanner` runs in a thread
+  // with valid getThreadIndex().
+  if (serial)
+    tg.spawn(outerFn);
+  else
+    outerFn();
 }
 
 static bool handleNonPreemptibleIfunc(Symbol &sym, uint16_t flags) {
diff --git a/contrib/llvm-project/llvm/include/llvm/Support/Parallel.h b/contrib/llvm-project/llvm/include/llvm/Support/Parallel.h
index 8170da98f15a..c34619ab7d96 100644
--- a/contrib/llvm-project/llvm/include/llvm/Support/Parallel.h
+++ b/contrib/llvm-project/llvm/include/llvm/Support/Parallel.h
@@ -97,9 +97,7 @@ public:
   // Spawn a task, but does not wait for it to finish.
   // Tasks marked with \p Sequential will be executed
   // exactly in the order which they were spawned.
-  // Note: Sequential tasks may be executed on different
-  // threads, but strictly in sequential order.
-  void spawn(std::function<void()> f, bool Sequential = false);
+  void spawn(std::function<void()> f);
 
   void sync() const { L.sync(); }
 
diff --git a/contrib/llvm-project/llvm/lib/Support/Parallel.cpp b/contrib/llvm-project/llvm/lib/Support/Parallel.cpp
index 9b14b05b5211..26da702969b7 100644
--- a/contrib/llvm-project/llvm/lib/Support/Parallel.cpp
+++ b/contrib/llvm-project/llvm/lib/Support/Parallel.cpp
@@ -12,7 +12,6 @@
 #include "llvm/Support/Threading.h"
 
 #include <atomic>
-#include <deque>
 #include <future>
 #include <thread>
 #include <vector>
@@ -39,7 +38,7 @@ namespace {
 class Executor {
 public:
   virtual ~Executor() = default;
-  virtual void add(std::function<void()> func, bool Sequential = false) = 0;
+  virtual void add(std::function<void()> func) = 0;
   virtual size_t getThreadCount() const = 0;
 
   static Executor *getDefaultExecutor();
@@ -98,13 +97,10 @@ public:
     static void call(void *Ptr) { ((ThreadPoolExecutor *)Ptr)->stop(); }
   };
 
-  void add(std::function<void()> F, bool Sequential = false) override {
+  void add(std::function<void()> F) override {
     {
       std::lock_guard<std::mutex> Lock(Mutex);
-      if (Sequential)
-        WorkQueueSequential.emplace_front(std::move(F));
-      else
-        WorkQueue.emplace_back(std::move(F));
+      WorkStack.push_back(std::move(F));
     }
     Cond.notify_one();
   }
@@ -112,42 +108,23 @@ public:
   size_t getThreadCount() const override { return ThreadCount; }
 
 private:
-  bool hasSequentialTasks() const {
-    return !WorkQueueSequential.empty() && !SequentialQueueIsLocked;
-  }
-
-  bool hasGeneralTasks() const { return !WorkQueue.empty(); }
-
   void work(ThreadPoolStrategy S, unsigned ThreadID) {
     threadIndex = ThreadID;
     S.apply_thread_strategy(ThreadID);
     while (true) {
       std::unique_lock<std::mutex> Lock(Mutex);
-      Cond.wait(Lock, [&] {
-        return Stop || hasGeneralTasks() || hasSequentialTasks();
-      });
+      Cond.wait(Lock, [&] { return Stop || !WorkStack.empty(); });
       if (Stop)
         break;
-      bool Sequential = hasSequentialTasks();
-      if (Sequential)
-        SequentialQueueIsLocked = true;
-      else
-        assert(hasGeneralTasks());
-
-      auto &Queue = Sequential ? WorkQueueSequential : WorkQueue;
-      auto Task = std::move(Queue.back());
-      Queue.pop_back();
+      auto Task = std::move(WorkStack.back());
+      WorkStack.pop_back();
       Lock.unlock();
       Task();
-      if (Sequential)
-        SequentialQueueIsLocked = false;
     }
   }
 
   std::atomic<bool> Stop{false};
-  std::atomic<bool> SequentialQueueIsLocked{false};
-  std::deque<std::function<void()>> WorkQueue;
-  std::deque<std::function<void()>> WorkQueueSequential;
+  std::vector<std::function<void()>> WorkStack;
   std::mutex Mutex;
   std::condition_variable Cond;
   std::promise<void> ThreadsCreated;
@@ -205,16 +182,14 @@ TaskGroup::~TaskGroup() {
   L.sync();
 }
 
-void TaskGroup::spawn(std::function<void()> F, bool Sequential) {
+void TaskGroup::spawn(std::function<void()> F) {
 #if LLVM_ENABLE_THREADS
   if (Parallel) {
     L.inc();
-    detail::Executor::getDefaultExecutor()->add(
-        [&, F = std::move(F)] {
-          F();
-          L.dec();
-        },
-        Sequential);
+    detail::Executor::getDefaultExecutor()->add([&, F = std::move(F)] {
+      F();
+      L.dec();
+    });
     return;
   }
 #endif