From nobody Wed Sep 21 09:46:50 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MXYTl3Lglz4cv0T; Wed, 21 Sep 2022 09:46:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MXYTl0j3Rz3fLL; Wed, 21 Sep 2022 09:46:51 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1663753611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cb3y50suOrWQMLiwnYW7l3s9HvqRfyO7c4DHIRMbhOU=; b=xnQD5PDu+/cuJ4J72LRrPLza2IjtGRZadvnPlzR6xTe++uAnBoWVcSXR2vllh65+ca1Bs+ 2madMrlFsalFwk+QhRVk2P3QkV0JE73uN7hVxn2O6UDl3C9QE5ttnj4S94Vnj5RhuwfFRk x60/scEsq0RY9sMw9Lmr2vAULOSS/U78i7m4gpr03u5V3NzJUP52OUIUDBxaaCmrRnaAKo QTUpOW2VLYvzhCrnMr+NQfKr9+iS2LcBHpfSdeNjeZ9fIaOememMbP881cH+lYdRnJJKcM Zzn+GBSP9Etdnb9f0rUb0icjC5hsYFBdHT4iSj+xvJ+GR76/ESJ5Rmz84VMpSQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MXYTk657Mz121V; Wed, 21 Sep 2022 09:46:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 28L9koax076252; Wed, 21 Sep 2022 09:46:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 28L9kodT076251; Wed, 21 Sep 2022 09:46:50 GMT (envelope-from git) Date: Wed, 21 Sep 2022 09:46:50 GMT Message-Id: <202209210946.28L9kodT076251@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: a5a49aedc639 - stable/13 - Use the cached dc zva length in arm64 memset List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a5a49aedc6392110b64574a1661ca77098629469 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1663753611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cb3y50suOrWQMLiwnYW7l3s9HvqRfyO7c4DHIRMbhOU=; b=Ig4b+pu/1uRNWW+RKjJMUU3Hq3YKF5/t4J6+2sWVeAfCsjogSD7HbT7/u0BYoT0opBBpfh hER6zPKrUGm4a7nz7Y/gu4vIPksTDUVuyYHTiVcqBgz8pVBKc48vyT7Z2N972iFfLM20/k qLMr+i9ucIvUXevJLXY75TdqmSs8KNqvE47yl1aYqh3YzA0mTSDcGlG+3TOk+DS3khn26c d7Jpf3yNqW/10x9E1tiSaLCvxajNuZ+XGeICDwdMGMgYilAWogq+XGGIvPfiTtdTCyQzqJ kw/ia8XvwXjy02GnXmvpnWdBgCyk9hQEEmrVUKq6TKAg+iGlTKAvhuu9ZCvTJA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1663753611; a=rsa-sha256; cv=none; b=KRHh1rIUgcW0YjwJR2ZpoPCNYK26eJ081LY35jtqZte+kpCrIhybS8gHGTeZbt6WTLTryU sR8nbIcsFnB7WGV6lyI2ogvTu+Mcrx1rV8DDoZ+cbjZmpprBaeP+nWpgAdrxO8GU5Qyb1E 3iHJNsgUhG8GeowLrb6y26WeIZ48aXRea3DfJtxUh0iOGRVrRQwqhzsNr+w7hOuIoQv3VJ MyuUZhjdNre8F134URMbvhXKV451Fw4wDqwNkzAepcYydpaDMUOIRj9IwJB66EXnpnQh+P +lpRqxFgWYRuq6f8VRoEbEuKVM0zGOaNIWwnY3tStJiKplBvxVq4+YIophd9jw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=a5a49aedc6392110b64574a1661ca77098629469 commit a5a49aedc6392110b64574a1661ca77098629469 Author: Andrew Turner AuthorDate: 2022-09-07 11:43:16 +0000 Commit: Andrew Turner CommitDate: 2022-09-21 09:45:53 +0000 Use the cached dc zva length in arm64 memset On boot we cache the length the 'dc zva' instruction will zero. Use this in the memset function to decide when to use it. As the cached value is in .bss it will be zero on boot so memset is safe to use before the value has been read. Sponsored by: The FreeBSD Foundation (cherry picked from commit 7d90ce7cae63a8121da0acc3ce36a713a98d6033) --- sys/arm64/arm64/memset.S | 48 +++++------------------------------------------- 1 file changed, 5 insertions(+), 43 deletions(-) diff --git a/sys/arm64/arm64/memset.S b/sys/arm64/arm64/memset.S index aaa196d35278..ec434493ce13 100644 --- a/sys/arm64/arm64/memset.S +++ b/sys/arm64/arm64/memset.S @@ -33,17 +33,6 @@ #include - -/* By default we assume that the DC instruction can be used to zero - data blocks more efficiently. In some circumstances this might be - unsafe, for example in an asymmetric multiprocessor environment with - different DC clear lengths (neither the upper nor lower lengths are - safe to use). The feature can be disabled by defining DONT_USE_DC. - - If code may be run in a virtualized environment, then define - MAYBE_VIRT. This will cause the code to cache the system register - values rather than re-reading them each call. */ - #define dstin x0 #define val w1 #define count x2 @@ -143,7 +132,6 @@ ENTRY(memset) b.ne .Ltail63 ret -#ifndef DONT_USE_DC /* For zeroing memory, check to see if we can use the ZVA feature to * zero entire 'cache' lines. */ .Lzero_mem: @@ -163,30 +151,11 @@ ENTRY(memset) * the line-clear code. */ cmp count, #128 b.lt .Lnot_short -#ifdef MAYBE_VIRT - /* For efficiency when virtualized, we cache the ZVA capability. */ - adrp tmp2, .Lcache_clear - ldr zva_len, [tmp2, #:lo12:.Lcache_clear] - tbnz zva_len, #31, .Lnot_short - cbnz zva_len, .Lzero_by_line - mrs tmp1, dczid_el0 - tbz tmp1, #4, 1f - /* ZVA not available. Remember this for next time. */ - mov zva_len, #~0 - str zva_len, [tmp2, #:lo12:.Lcache_clear] - b .Lnot_short -1: - mov tmp3w, #4 - and zva_len, tmp1w, #15 /* Safety: other bits reserved. */ - lsl zva_len, tmp3w, zva_len - str zva_len, [tmp2, #:lo12:.Lcache_clear] -#else - mrs tmp1, dczid_el0 - tbnz tmp1, #4, .Lnot_short - mov tmp3w, #4 - and zva_len, tmp1w, #15 /* Safety: other bits reserved. */ - lsl zva_len, tmp3w, zva_len -#endif + + adrp tmp2, dczva_line_size + add tmp2, tmp2, :lo12:dczva_line_size + ldr zva_len, [tmp2] + cbz zva_len, .Lnot_short .Lzero_by_line: /* Compute how far we need to go to become suitably aligned. We're @@ -225,11 +194,4 @@ ENTRY(memset) ands count, count, zva_bits_x b.ne .Ltail_maybe_long ret -#ifdef MAYBE_VIRT - .bss - .p2align 2 -.Lcache_clear: - .space 4 -#endif -#endif /* DONT_USE_DC */ END(memset)