From owner-dev-commits-src-all@freebsd.org Tue Feb 23 00:10:50 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E23FB5548F8; Tue, 23 Feb 2021 00:10:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DkzwV61Ggz3HZy; Tue, 23 Feb 2021 00:10:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C16ED73F0; Tue, 23 Feb 2021 00:10:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 11N0Ao98032783; Tue, 23 Feb 2021 00:10:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 11N0Aoac032782; Tue, 23 Feb 2021 00:10:50 GMT (envelope-from git) Date: Tue, 23 Feb 2021 00:10:50 GMT Message-Id: <202102230010.11N0Aoac032782@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mateusz Guzik Subject: git: 7f06b217c53c - main - amd64: import asm strlen into libc MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7f06b217c53c3f5e4ac81eb11125adfb71359ac6 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2021 00:10:50 -0000 The branch main has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=7f06b217c53c3f5e4ac81eb11125adfb71359ac6 commit 7f06b217c53c3f5e4ac81eb11125adfb71359ac6 Author: Mateusz Guzik AuthorDate: 2021-02-21 21:20:04 +0000 Commit: Mateusz Guzik CommitDate: 2021-02-23 00:09:55 +0000 amd64: import asm strlen into libc Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D28845 --- lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strlen.S | 81 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/lib/libc/amd64/string/Makefile.inc b/lib/libc/amd64/string/Makefile.inc index db88ac723539..cb370bc6be1c 100644 --- a/lib/libc/amd64/string/Makefile.inc +++ b/lib/libc/amd64/string/Makefile.inc @@ -8,4 +8,5 @@ MDSRCS+= \ memset.S \ strcat.S \ strcmp.S \ + strlen.S \ stpcpy.S diff --git a/lib/libc/amd64/string/strlen.S b/lib/libc/amd64/string/strlen.S new file mode 100644 index 000000000000..1d2428e3420e --- /dev/null +++ b/lib/libc/amd64/string/strlen.S @@ -0,0 +1,81 @@ +/* + * Written by Mateusz Guzik + * Public domain. + */ + +#include +__FBSDID("$FreeBSD$"); + +/* + * Note: this routine was written with kernel use in mind (read: no simd), + * it is only present in userspace as a temporary measure until something + * better gets imported. + */ + +#define ALIGN_TEXT .p2align 4,0x90 /* 16-byte alignment, nop filled */ + +/* + * strlen(string) + * %rdi + * + * Uses the ((x - 0x01....01) & ~x & 0x80....80) trick. + * + * 0x01....01 is replaced with 0x0 - 0x01....01 so that it can be added + * with leaq. + * + * For a description see either: + * - "Hacker's Delight" by Henry S. Warren, Jr. + * - "Optimizing subroutines in assembly language: An optimization guide for x86 platforms" + * by Agner Fog + * + * The latter contains a 32-bit variant of the same algorithm coded in assembly for i386. + */ +ENTRY(strlen) + movabsq $0xfefefefefefefeff,%r8 + movabsq $0x8080808080808080,%r9 + + movq %rdi,%r10 + movq %rdi,%rcx + testb $7,%dil + jz 2f + + /* + * Handle misaligned reads: align to 8 and fill + * the spurious bytes. + */ + andq $~7,%rdi + movq (%rdi),%r11 + shlq $3,%rcx + movq $-1,%rdx + shlq %cl,%rdx + notq %rdx + orq %rdx,%r11 + + leaq (%r11,%r8),%rcx + notq %r11 + andq %r11,%rcx + andq %r9,%rcx + jnz 3f + + /* + * Main loop. + */ + ALIGN_TEXT +1: + leaq 8(%rdi),%rdi +2: + movq (%rdi),%r11 + leaq (%r11,%r8),%rcx + notq %r11 + andq %r11,%rcx + andq %r9,%rcx + jz 1b +3: + bsfq %rcx,%rcx + shrq $3,%rcx + leaq (%rcx,%rdi),%rax + subq %r10,%rax + ret +END(strlen) + + .section .note.GNU-stack,"",%progbits