From nobody Thu Jun 16 17:46:39 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 07C7B83295C; Thu, 16 Jun 2022 17:46:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LP8k76n3Rz4vGB; Thu, 16 Jun 2022 17:46:39 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1655401600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0/Os9dBFT7TzLVDlLs8ElW4P+x6uXDwRGlGgn+iVrcs=; b=tCL1T+pXN89aw+OtT/hQlLJSnCETVGzVuOlaVIfpmYjX1NIHxnw7mV65zQiC7b+TpRm+/+ qF10oE92rfkS6QUssufE/6jeLLDPieilahGQEX+MXTAf2IzCAsqeQJPE6y2S2XWsKzyqcI RReS+kHP672YLGstmHteayxXoc+1qaKtz3ZZj+OjSD3QHaWB+h+y+NqWwknD/jHZlHEH1z SEITcCMlb1+/jeOl6R5CV2soD1IBh+mvSlVRFJk+wJhz5S7jMCq5gxkvDR+VZgYAYF7Hh1 NKXz0GaOyZQLZ+qlDNjscBFO9jXcDVB2BomFCU3Mx9duU3SJy5b+5eabz3WMSg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C883E7A31; Thu, 16 Jun 2022 17:46:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 25GHkdT4014947; Thu, 16 Jun 2022 17:46:39 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 25GHkdIV014946; Thu, 16 Jun 2022 17:46:39 GMT (envelope-from git) Date: Thu, 16 Jun 2022 17:46:39 GMT Message-Id: <202206161746.25GHkdIV014946@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alexander Motin Subject: git: 6210ac95a194 - main - amd64: Stop using REP MOVSB for backward memmove()s. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 6210ac95a19416832601b571409a3e08b76d107f Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1655401600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0/Os9dBFT7TzLVDlLs8ElW4P+x6uXDwRGlGgn+iVrcs=; b=Rtc66wN4vAH7zcdhdTwDhPfT0EGZvv8v4HPFw5C124yKp6PlrumXWqqwueY1fQObDTlZGG j9ewKP5FlnJ6Z5Er3I6fbV3eziyPpUkYnvhKUaPbe/DE1wqmWHxmJ4mDBEW4x6aI/JK6uB AqcZr4WDpUeLyf3hgDyarHdhnmWa0TMu/b8rt5O1N5EywsuSvzDlzT1MOMdLSItrf66FEU x5SdAbFtLaKVPUk1YTFzkrujSQBpADTd5+M3uO8pAzU/OjOWrqznkWrgSK9K4NG15LSsyo uHBJmwuprpB2dkG41D8eh2nuAWeXMWnY+bTJj9EznshBCjQvtrDFElsAXZlCcA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1655401600; a=rsa-sha256; cv=none; b=DBiY2XFpC/H0vIvLn73Y2lpv/hhMLo1YcH1N0Kh10mkuPoznybfW4GHwvjo6W7fZ8rgz+j JxrcbQa1ftQC98LoBVDKnsYWQ+dOhr22cV45326byec+FOVMI32YZamo3z1bzT1kQFjl9f WLjndDjDC1bJ5RwnsShVE/tPjkrVNc4OXLs1Z1mBSv6/X6xOYYXHOGP5LX5oLTK0X4rZOB NDvfoBWU9z5xFlcW4/NVINiGr/fnJZQZ+KeKK5b8D9LRrfOrin1hVxX8cH64c4zBzOPMkn 84vKwkkOOC3hTj8xTRtSWaVetF+iZ2p4R4IogipdYQZdFPUoLrzJuDa1x5Qyjg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=6210ac95a19416832601b571409a3e08b76d107f commit 6210ac95a19416832601b571409a3e08b76d107f Author: Alexander Motin AuthorDate: 2022-06-16 17:01:12 +0000 Commit: Alexander Motin CommitDate: 2022-06-16 17:46:34 +0000 amd64: Stop using REP MOVSB for backward memmove()s. Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes REP MOVSB the fastest way to copy memory in most of cases. However Intel Optimization Reference Manual says: "setting the DF to force REP MOVSB to copy bytes from high towards low addresses will expe- rience significant performance degradation". Measurements on Intel Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs. This patch keeps ERMS use for forward ordered memory copies, but removes it for backward overlapped moves where it does not work. Reviewed by: mjg MFC after: 2 weeks --- sys/amd64/amd64/support.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/sys/amd64/amd64/support.S b/sys/amd64/amd64/support.S index e21374233c84..8e284c6f6a9e 100644 --- a/sys/amd64/amd64/support.S +++ b/sys/amd64/amd64/support.S @@ -507,13 +507,6 @@ END(memcmp) ALIGN_TEXT 2256: std -.if \erms == 1 - leaq -1(%rdi,%rcx),%rdi - leaq -1(%rsi,%rcx),%rsi - rep - movsb - cld -.else leaq -8(%rdi,%rcx),%rdi leaq -8(%rsi,%rcx),%rsi shrq $3,%rcx @@ -523,7 +516,6 @@ END(memcmp) movq %rdx,%rcx andb $7,%cl jne 2004b -.endif \end ret .endif