From nobody Mon Aug 28 17:46:13 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RZHzT2NxYz4rn57; Mon, 28 Aug 2023 17:46:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RZHzT1lMGz3Rfg; Mon, 28 Aug 2023 17:46:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1693244773; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1qWxLFDdtD8uEjr+P6vLUTTYPB+4nlkGkxTRgO8y950=; b=NzZl9XRd7un8bH1H32PSGWOL2NxJZWaWF799iJQfxJpolY44hPZvk1iEzSHszqzo47U8oF sIJjdixTPEBfsvlhJtIVFiifb3EMi3d7zXs5twEEbkoZPrC63e7UFKwx9ILdK2nP9PuovV PylzVjE+sxvcjRIjvSfcjXMD3PGMnp0FDyWS1450Pb379ZszGL2G+J3pE3MpAyTzuWvZRH 96ZjLCnUbW9DVEqNK6zH7ZXzTMkCp5fHEboZF0rdahlFXfxUqLUor8xgKZzUU0Ye944uQe x1KWC43W4VkQD4rOl0i7j7L4Yf3qiKjl+ZDU4U61b4L13hR1VMvmZA+ughsn6A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1693244773; a=rsa-sha256; cv=none; b=lF11x5k7VcfI/IhoDap8xHx0oVoNOFGlwj0KDB/a8lJSb4mHKekVDZjzWzYZMhvxlTuiW9 BEH/5J90MkQ0mQNU3A8QNXG5PaloLhXeXrltav+xVVRBK7HMYE80WkEQEssDWblpINJ7To U9okBg8z+RcwLx5rSAiKYWo8eKgcgO3EveoejqLZFR3IzCfR+Fr+Ay+tkUhAE/QtvwEJXH hX1Yk/85sLcGeW3RzMoColbc5ZP32YiS4RyKflqzCGUesmmvKnbB8+bQ+m7N4p1CzHyRIQ 4vVi/qwJm0/F7zX2f29lMoT7Kuq9YUDQk+QtfPM1sMxbkaxZbwhRH8uIXljiDQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1693244773; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1qWxLFDdtD8uEjr+P6vLUTTYPB+4nlkGkxTRgO8y950=; b=C73wIKdu9xKtHSL0rOutJclrf9/08hEMhc8+g1HGug5sEvXg25fQK0xqCr37QNVUXEbsII XyPVl+DKLMwlpBmGUaUmdwqK94nQLqIF8e5Rc+CBaCZZ0Wifo3N8jskptcLTUYH3O7qxpE Tvgck5dPLLSAQOCy0Z7aNLSEJIsOfZBcAoQoqqrrHIWRV9fW7lwzonI1P4TIWIoMrcIOsZ sOBPY8NwDJCBV/Rsid0QZcOLHMCyxgx9/QEdzc9ztba68e2IF9LQaHrgiijhvLFXF4yArC 593Q1SAAcGXUm6kwwPTyt/xkPEeVJQwsLykMMbsyxiz5yByCYoxiAsGd9zBjyQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RZHzT0V2zz3mf; Mon, 28 Aug 2023 17:46:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 37SHkD8F095260; Mon, 28 Aug 2023 17:46:13 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 37SHkDrx095257; Mon, 28 Aug 2023 17:46:13 GMT (envelope-from git) Date: Mon, 28 Aug 2023 17:46:13 GMT Message-Id: <202308281746.37SHkDrx095257@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Robert Clausecker Subject: git: 8b81116755b4 - stable/14 - lib/libc/amd64/string/strchrnul.S: fix edge case in scalar code List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: fuz X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 8b81116755b46c3f651bf873ee25f83f8758bd2b Auto-Submitted: auto-generated The branch stable/14 has been updated by fuz: URL: https://cgit.FreeBSD.org/src/commit/?id=8b81116755b46c3f651bf873ee25f83f8758bd2b commit 8b81116755b46c3f651bf873ee25f83f8758bd2b Author: Robert Clausecker AuthorDate: 2023-08-25 16:22:22 +0000 Commit: Robert Clausecker CommitDate: 2023-08-28 17:45:51 +0000 lib/libc/amd64/string/strchrnul.S: fix edge case in scalar code When the buffer is immediately preceeded by the character we are looking for and begins with one higher than that character, and the buffer is misaligned, a match was errorneously detected in the first character. Fix this by changing the way we prevent matches before the buffer from being detected: instead of removing the corresponding bit from the 0x80..80 mask, set the LSB of bytes before the buffer after xoring with the character we look for. The bug only affects amd64 with ARCHLEVEL=scalar (cf. simd(7)). The change comes at a 2% performance impact for short strings if ARCHLEVEL is set to scalar. The default configuration is not affected. os: FreeBSD arch: amd64 cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz │ strchrnul.scalar.0.out │ strchrnul.scalar.2.out │ │ sec/op │ sec/op vs base │ Short 57.89µ ± 2% 59.08µ ± 1% +2.07% (p=0.030 n=20) Mid 19.24µ ± 0% 19.73µ ± 0% +2.53% (p=0.000 n=20) Long 11.03µ ± 0% 11.03µ ± 0% ~ (p=0.547 n=20) geomean 23.07µ 23.43µ +1.53% │ strchrnul.scalar.0.out │ strchrnul.scalar.2.out │ │ B/s │ B/s vs base │ Short 2.011Gi ± 2% 1.970Gi ± 1% -2.02% (p=0.030 n=20) Mid 6.049Gi ± 0% 5.900Gi ± 0% -2.47% (p=0.000 n=20) Long 10.56Gi ± 0% 10.56Gi ± 0% ~ (p=0.547 n=20) geomean 5.045Gi 4.969Gi -1.50% MFC to: stable/14 MFC after: 3 days Approved by: mjg (blanket, via IRC), re (gjb) Sponsored by: The FreeBSD Foundation (cherry picked from commit 3d8ef251aa9dceabd57f7821a0e6749d35317db3) --- lib/libc/amd64/string/strchrnul.S | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/lib/libc/amd64/string/strchrnul.S b/lib/libc/amd64/string/strchrnul.S index 9aa9591b07ca..0e70b02311d7 100644 --- a/lib/libc/amd64/string/strchrnul.S +++ b/lib/libc/amd64/string/strchrnul.S @@ -52,22 +52,25 @@ ARCHENTRY(__strchrnul, scalar) movabs $0x0101010101010101, %r8 mov (%rdi), %rax # load first word imul %r8, %rsi # replicate char 8 times - movabs $0x8080808080808080, %r9 /* * Unaligned input: align to 8 bytes. Then proceed the same - * way as with aligned input, but ignore matches before the - * beginning of the string. This is achieved by shifting r9 - * into r10 to have 0x00 bytes before the string begins. + * way as with aligned input, but prevent matches before the + * beginning of the string. This is achieved by oring 0x01 + * into each byte of the buffer before the string */ shl $3, %ecx - mov %r9, %r10 + mov %r8, %r10 add $8, %rdi - shl %cl, %r10 # 0x80 where the string is + shl %cl, %r10 # 0x01 where the string is + xor %r8, %r10 # 0x01 where it is not neg %r8 # negate 01..01 so we can use lea + movabs $0x8080808080808080, %r9 mov %rsi, %rcx xor %rax, %rcx # str ^ c + or %r10, %rax # str without NUL bytes before it + or %r10, %rcx # (str ^ c) without matches before it lea (%rax, %r8, 1), %rdx # str - 0x01..01 lea (%rcx, %r8, 1), %r11 # (str ^ c) - 0x01..01 not %rax # ~str @@ -75,7 +78,7 @@ ARCHENTRY(__strchrnul, scalar) and %rdx, %rax # (str - 0x01..01) & ~str and %r11, %rcx # ((str ^ c - 0x01..01) & ~(str ^ c) or %rcx, %rax # matches for both - and %r10, %rax # not including junk bytes or bytes before the string + and %r9, %rax # not including junk bytes jnz 1f /* main loop unrolled twice */