From nobody Thu Jun 30 01:15:56 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9444187BF66; Thu, 30 Jun 2022 01:15:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LYL4Y0XTQz4WK5; Thu, 30 Jun 2022 01:15:57 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PNUubVIv39D0D6Zt4FRC0limDdjRkF1jRXQ+ZCca2jk=; b=dFv65rs3vKyXPAGkgpbQ5oD9tPyrvBf8+GQ8DRNHP2iQ5GbneFYWr2szN1FuMUbuggBAL2 /ZHqaVEYCUIucrJFikwlOjuzF1LwguJuZPhguvHlD7CKjPNr8RU1+gnvoN/Zb/TunN3fJz 4H4Kpmi7HgKYdmj5vFZA0iPIkcHQthbw3l7XI8ZAkx+Pzt/t13BzwEWpxKIKRJvZt1x4qI e/jdIpPHzIs9c7UWSnxgDXhaauE7dnmIv3hJs4zI9wiF6Y672fh9qthrTkE87adaf7Mv1o hnVL3pLhqYNH8trX6u1dtDbMDg8f4fpedvNuVnX/jM/tpHgW9OgVXxnVoiDZxA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E6D961CBAC; Thu, 30 Jun 2022 01:15:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 25U1FurG097951; Thu, 30 Jun 2022 01:15:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 25U1FuUZ097950; Thu, 30 Jun 2022 01:15:56 GMT (envelope-from git) Date: Thu, 30 Jun 2022 01:15:56 GMT Message-Id: <202206300115.25U1FuUZ097950@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 15183f36e5a0 - stable/13 - amd64: Stop using REP MOVSB for backward memmove()s. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 15183f36e5a0b2ab1febdb7e88d20c0a3d3ab86e Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656551757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PNUubVIv39D0D6Zt4FRC0limDdjRkF1jRXQ+ZCca2jk=; b=iwPf9C7npy4hukVT0hdwxV8FCcyBPNnJQHMehcxuFEmWlZrCT4jtsH0EK5lGhjiHE1Cz6W AyKYoo5n7m5B0w2QGrR/d2Znr/FLCESK8Z15VDG91T3XOs17uWECtmO780j/ocwa6Mamok vXqf8q8QTeciAkbQC2EnWGXgA7p72KQW3ufM0yrda3p+gpVTZrGs0+Ff4WsAMIJCCAPkCi F0WsmxaSm/w3kgjNqycEBGu6UH//zseX07kcMJdRemQWfwCWwmAd1u2Xuk5dJQAQsj/Z5g 5m0BpRPNjOdch9hSCkxihCzEY86VxIsQtmoto5uZ4eAK1yShglSQ5Pvv0QN60w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1656551757; a=rsa-sha256; cv=none; b=vbcHBDBABOwd2fM7IBYbQxYm00+ZOewZULiyVfYGk5A+w9hIieiovgiReY4aWAgQ+d4YJJ um0bR4N2tESE6+Xbw3mirXHeybhX1dEKQ7RCWXyA5bN3TGfVRW7WBVnIBPfnbJlyMVw9/U kdgCIV23urw9Xnc2XBD5Rg7CRRPdm3YKRObq7HGMEEaO17DZxGr9P4cU2s+Lei/MKCbwA/ peZv57qG2xIKnIC0DuxfAfA5xnN/kmQL2/i3KaMWS23Fjc4AeivKrAgO55x+lWnJY2LsRJ 2YPnnDojzOByLXyBtaVOoQfoD59LQv3W1/6T7Oecg4co28+2/p9nhawOAEk5wA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=15183f36e5a0b2ab1febdb7e88d20c0a3d3ab86e commit 15183f36e5a0b2ab1febdb7e88d20c0a3d3ab86e Author: Alexander Motin AuthorDate: 2022-06-16 17:01:12 +0000 Commit: Alexander Motin CommitDate: 2022-06-30 01:15:49 +0000 amd64: Stop using REP MOVSB for backward memmove()s. Enhanced REP MOVSB feature of CPUs starting from Ivy Bridge makes REP MOVSB the fastest way to copy memory in most of cases. However Intel Optimization Reference Manual says: "setting the DF to force REP MOVSB to copy bytes from high towards low addresses will expe- rience significant performance degradation". Measurements on Intel Cascade Lake and Alder Lake, same as on AMD Zen3 show that it can drop throughput to as low as 2.5-3.5GB/s, comparing to ~10-30GB/s of REP MOVSQ or hand-rolled loop, used for non-ERMS CPUs. This patch keeps ERMS use for forward ordered memory copies, but removes it for backward overlapped moves where it does not work. Reviewed by: mjg MFC after: 2 weeks (cherry picked from commit 6210ac95a19416832601b571409a3e08b76d107f) --- sys/amd64/amd64/support.S | 8 -------- 1 file changed, 8 deletions(-) diff --git a/sys/amd64/amd64/support.S b/sys/amd64/amd64/support.S index 15f72a425cf1..09e73800bddd 100644 --- a/sys/amd64/amd64/support.S +++ b/sys/amd64/amd64/support.S @@ -507,13 +507,6 @@ END(memcmp) ALIGN_TEXT 2256: std -.if \erms == 1 - leaq -1(%rdi,%rcx),%rdi - leaq -1(%rsi,%rcx),%rsi - rep - movsb - cld -.else leaq -8(%rdi,%rcx),%rdi leaq -8(%rsi,%rcx),%rsi shrq $3,%rcx @@ -523,7 +516,6 @@ END(memcmp) movq %rdx,%rcx andb $7,%cl jne 2004b -.endif \end ret .endif