From nobody Wed Jul 5 12:35:10 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QwzdV6x7Qz4lsq7; Wed, 5 Jul 2023 12:35:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QwzdV6TM5z4G3B; Wed, 5 Jul 2023 12:35:10 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1688560510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bktnkrZnD63MQRjQiwVfd31IjV6s9dgJfsuOj4mwm+Q=; b=hc0Hc0gLfwXLO3ESLf5ZoZ8QS8l/N93IgVEV78RXXSvityYa/v8mR6HquPDiCkfzWP8+hn 4WmxFsyrsvJYbW8Hx4LPkVQL5JNSkoFO8S39ut/ifMMsk3eLV1FUGEI3UAkFha2KZtG2+u EeVQ5IDLq/JRoEuO9dEknBCBdEL5nAuVKy3zqq51wgh/Vp1I806+l7GUMG4wca8NHJgOQh zS2tlwwpbDa67rH7X0RllaxpfYXdFrOmRzD4b1Ntc9Kk6Jrhakszen1Y80VJcpYeQZHivr fldpnCpgzxOyEX1z4qww1fy66sMRKMSyNEW3mGBufqbnPcodQKIWwTo0XfJF+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1688560510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bktnkrZnD63MQRjQiwVfd31IjV6s9dgJfsuOj4mwm+Q=; b=pFJ9ftxQmH/9nvJ84znt8ptbTnZV3Q+x5VqMabpCCD3T66SrpOFCHKRXSWsXjlW3hG6KNq FX6DEaUE0seh+tQM2rmDUKzZUyQVMZkdbyCvqgr59hwAsIM82BS0r35oIUvEopKDN6COGB ZZ62sArjLKnERvdte/puuXv/E7UtXYmgs+HAC/UTt7XzswKXBhfXkflUfWQODPRHyTL7SV Rtxh8DAC35mSNN90ku495RE9J+bpN9QGYMXq8DjUbVIW0RXf9OekyYgQM9Z84Z5QlhnCOc LExaiNbffzOBjsit/yFn3368YaXci38Mbe7IB1REt1UTQIb7hTKA8QfoPD1thA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1688560510; a=rsa-sha256; cv=none; b=UipvCn3coH2frd8xLkOlP4nsc0gl1m/MslZHBGC17fVr70Dg4G53zZQ3JsZhgS7g40ryRr bvkYYnHCadUTIpY5+GgPFjRVaHf23tkXkW3+MpHl0qBsnriQ4dk86JNYCbSyWl8pK5lERN DC7AX6+ZGBzi8FO13C9GKkMJdUeGf1qXUABnNimmXrKD/2HdKWGvUbLxWGKBmKdKMoNClc wGDPqYb0AYGNlnF8mt0MHdiRZfizRljL8KWoIihqni1DAKR9K4vzwLiuAKjKbeH6KenI38 9qJG+GyK0jtJdJPVqQQk0QBC3pCjpJ9cVC9Nd+tSMlYkmMgt2c5VPR2zKwYL/Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QwzdV5YfhzFTX; Wed, 5 Jul 2023 12:35:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 365CZAdG020429; Wed, 5 Jul 2023 12:35:10 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 365CZA8E020428; Wed, 5 Jul 2023 12:35:10 GMT (envelope-from git) Date: Wed, 5 Jul 2023 12:35:10 GMT Message-Id: <202307051235.365CZA8E020428@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: cbca929869a8 - stable/13 - libpthread: allocate rwlocks and spinlocks in dedicated cachelines List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: cbca929869a818c9b3353753df71210816a126d6 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=cbca929869a818c9b3353753df71210816a126d6 commit cbca929869a818c9b3353753df71210816a126d6 Author: Greg Becker AuthorDate: 2023-06-27 01:08:29 +0000 Commit: Mateusz Guzik CommitDate: 2023-07-05 12:34:09 +0000 libpthread: allocate rwlocks and spinlocks in dedicated cachelines Reduces severe performance degradation due to false-sharing. Note that this does not account for hardware which can perform adjacent cacheline prefetch. [mjg: massaged the commit message and the patch to use aligned_alloc instead of malloc] PR: 272238 MFC after: 1 week (cherry picked from commit a6c0d801ca5934bb9b9cca6870ea7406d5db0641) --- lib/libthr/thread/thr_pspinlock.c | 3 ++- lib/libthr/thread/thr_rwlock.c | 5 ++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/libthr/thread/thr_pspinlock.c b/lib/libthr/thread/thr_pspinlock.c index d1a53446f654..b1b8164c9777 100644 --- a/lib/libthr/thread/thr_pspinlock.c +++ b/lib/libthr/thread/thr_pspinlock.c @@ -60,7 +60,8 @@ _pthread_spin_init(pthread_spinlock_t *lock, int pshared) if (lock == NULL) return (EINVAL); if (pshared == PTHREAD_PROCESS_PRIVATE) { - lck = malloc(sizeof(struct pthread_spinlock)); + lck = aligned_alloc(CACHE_LINE_SIZE, + roundup(sizeof(struct pthread_spinlock), CACHE_LINE_SIZE)); if (lck == NULL) return (ENOMEM); *lock = lck; diff --git a/lib/libthr/thread/thr_rwlock.c b/lib/libthr/thread/thr_rwlock.c index 1d110a7bf285..8457250e60f7 100644 --- a/lib/libthr/thread/thr_rwlock.c +++ b/lib/libthr/thread/thr_rwlock.c @@ -32,6 +32,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include "namespace.h" #include @@ -102,9 +103,11 @@ rwlock_init(pthread_rwlock_t *rwlock, const pthread_rwlockattr_t *attr) if (attr == NULL || *attr == NULL || (*attr)->pshared == PTHREAD_PROCESS_PRIVATE) { - prwlock = calloc(1, sizeof(struct pthread_rwlock)); + prwlock = aligned_alloc(CACHE_LINE_SIZE, + roundup(sizeof(struct pthread_rwlock), CACHE_LINE_SIZE)); if (prwlock == NULL) return (ENOMEM); + memset(prwlock, 0, sizeof(struct pthread_rwlock)); *rwlock = prwlock; } else { prwlock = __thr_pshared_offpage(rwlock, 1);