From nobody Thu Sep 8 13:32:27 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MNg636K0Nz4c7JJ; Thu, 8 Sep 2022 13:32:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MNg635K0Lz3FVY; Thu, 8 Sep 2022 13:32:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1662643947; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tuh0OOPLVMzfxPjG56wp2l2Fdk1vDDCgJCtflkoO6Eo=; b=n1j6syhYYfImt+gHqD74OS06kOCbRiL55FZnhmIihGoidkNAwpdRdMPwJhOreALBmQnu2F apheSiJxOPYW2OTjSb9sC98hnr9JiiIuMhRV3z2/xU/0vF7JAkjm+UYbiPf5iZgWzFv7c3 jajbjjoVpjuzHB/8P5AAhQPbW/KkuvuekS1xB9P0EDSKDgWp6lxBwtecCcgmijDYiqMUQF DkbiFSmko1zyik6yK4yAOnTMidex+tSFVkCYTxYOY3ydQ2ka968SvY9AT45ZQwuxdSj/0a hD51m5YETKcid4rYUQWAp+GD3uQU0+LHH2RHzerPLvA7PFPO6QZWB0xRn3YqeQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MNg633xySztG5; Thu, 8 Sep 2022 13:32:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 288DWR9Q073494; Thu, 8 Sep 2022 13:32:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 288DWRtt073487; Thu, 8 Sep 2022 13:32:27 GMT (envelope-from git) Date: Thu, 8 Sep 2022 13:32:27 GMT Message-Id: <202209081332.288DWRtt073487@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: 7d90ce7cae63 - main - Use the cached dc zva length in arm64 memset List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7d90ce7cae63a8121da0acc3ce36a713a98d6033 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1662643947; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tuh0OOPLVMzfxPjG56wp2l2Fdk1vDDCgJCtflkoO6Eo=; b=LObW0p3OnqgNKg+oJLf+A9nKMqcy+WJzMWrXIvDiAWhZmVZemzEd1dYoqWXWf/oHe/0Nhd +ueYit+mi72rARxWR9VkUD4MkwHI1+iWA1RoGTudlKrsKgd5r6QsfR+zhez0n8T/xz1I3H OEsWmAEJv+wqT8PAcWz9TgwObceW0qWTAHW1uH702bb6jxI/3c28WvgqSxW6kIH5AgRxRR b/rCrjpVngAgEM2UTRHaRctVjDpw549Mq/0Ek/wdYV2C23a8UoCDhIclmkWstQAnrM2fhY +hPX1/LLtAqnYSFKiZPL5J7EtcXTQj6udJXOHDXEsBDJ1Tu10EwZDWIvS5GTFQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1662643947; a=rsa-sha256; cv=none; b=W/rRHWWQdMT837/n1IT1WT7nrIQTkwinmRsTtM0yIsMna+RSJQf4s7w5+0H98pFecwd/gB G1UMN8Uy6Ww1WASBagUBaA4qeyvoeUqQ/5s3fyzUvHx2Q2Cq93qVpYc2zM0TKoGT2S7MjP a1IyAxFzy0A1uWn8yUb0YmjAuN59XoLYmUeU7fkTlWPfFO62liHsKqLkKGiE7IcAaH4T54 C1C7mRvT5xdjl/lWxb806rRBqaadeuyAeUhCMC/gmNSffLgqX3C3FTg+Qu1gencGBCd6vI fHikVgaZbsbt41qTMOnoEf+2eVj3UWR4+ZnnApNldjJ41trbdeoAN0J0/Cm+QQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=7d90ce7cae63a8121da0acc3ce36a713a98d6033 commit 7d90ce7cae63a8121da0acc3ce36a713a98d6033 Author: Andrew Turner AuthorDate: 2022-09-07 11:43:16 +0000 Commit: Andrew Turner CommitDate: 2022-09-08 13:30:11 +0000 Use the cached dc zva length in arm64 memset On boot we cache the length the 'dc zva' instruction will zero. Use this in the memset function to decide when to use it. As the cached value is in .bss it will be zero on boot so memset is safe to use before the value has been read. Sponsored by: The FreeBSD Foundation --- sys/arm64/arm64/memset.S | 48 +++++------------------------------------------- 1 file changed, 5 insertions(+), 43 deletions(-) diff --git a/sys/arm64/arm64/memset.S b/sys/arm64/arm64/memset.S index aaa196d35278..ec434493ce13 100644 --- a/sys/arm64/arm64/memset.S +++ b/sys/arm64/arm64/memset.S @@ -33,17 +33,6 @@ #include - -/* By default we assume that the DC instruction can be used to zero - data blocks more efficiently. In some circumstances this might be - unsafe, for example in an asymmetric multiprocessor environment with - different DC clear lengths (neither the upper nor lower lengths are - safe to use). The feature can be disabled by defining DONT_USE_DC. - - If code may be run in a virtualized environment, then define - MAYBE_VIRT. This will cause the code to cache the system register - values rather than re-reading them each call. */ - #define dstin x0 #define val w1 #define count x2 @@ -143,7 +132,6 @@ ENTRY(memset) b.ne .Ltail63 ret -#ifndef DONT_USE_DC /* For zeroing memory, check to see if we can use the ZVA feature to * zero entire 'cache' lines. */ .Lzero_mem: @@ -163,30 +151,11 @@ ENTRY(memset) * the line-clear code. */ cmp count, #128 b.lt .Lnot_short -#ifdef MAYBE_VIRT - /* For efficiency when virtualized, we cache the ZVA capability. */ - adrp tmp2, .Lcache_clear - ldr zva_len, [tmp2, #:lo12:.Lcache_clear] - tbnz zva_len, #31, .Lnot_short - cbnz zva_len, .Lzero_by_line - mrs tmp1, dczid_el0 - tbz tmp1, #4, 1f - /* ZVA not available. Remember this for next time. */ - mov zva_len, #~0 - str zva_len, [tmp2, #:lo12:.Lcache_clear] - b .Lnot_short -1: - mov tmp3w, #4 - and zva_len, tmp1w, #15 /* Safety: other bits reserved. */ - lsl zva_len, tmp3w, zva_len - str zva_len, [tmp2, #:lo12:.Lcache_clear] -#else - mrs tmp1, dczid_el0 - tbnz tmp1, #4, .Lnot_short - mov tmp3w, #4 - and zva_len, tmp1w, #15 /* Safety: other bits reserved. */ - lsl zva_len, tmp3w, zva_len -#endif + + adrp tmp2, dczva_line_size + add tmp2, tmp2, :lo12:dczva_line_size + ldr zva_len, [tmp2] + cbz zva_len, .Lnot_short .Lzero_by_line: /* Compute how far we need to go to become suitably aligned. We're @@ -225,11 +194,4 @@ ENTRY(memset) ands count, count, zva_bits_x b.ne .Ltail_maybe_long ret -#ifdef MAYBE_VIRT - .bss - .p2align 2 -.Lcache_clear: - .space 4 -#endif -#endif /* DONT_USE_DC */ END(memset)