From owner-dev-commits-src-main@freebsd.org Sat May 1 08:25:21 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2CB6F62BB3C; Sat, 1 May 2021 08:25:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FXMk90ndyz3k62; Sat, 1 May 2021 08:25:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0E0A21493E; Sat, 1 May 2021 08:25:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1418PKw5069784; Sat, 1 May 2021 08:25:20 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1418PKuC069783; Sat, 1 May 2021 08:25:20 GMT (envelope-from git) Date: Sat, 1 May 2021 08:25:20 GMT Message-Id: <202105010825.1418PKuC069783@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: 2abd4f858146 - main - Add a way to map arm64 non-posted device memory MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 2abd4f858146a266d48e3c55c2e29376e8b00967 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2021 08:25:21 -0000 The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=2abd4f858146a266d48e3c55c2e29376e8b00967 commit 2abd4f858146a266d48e3c55c2e29376e8b00967 Author: Andrew Turner AuthorDate: 2021-04-08 11:54:20 +0000 Commit: Andrew Turner CommitDate: 2021-05-01 06:01:20 +0000 Add a way to map arm64 non-posted device memory On arm64 we currently use a non-posted write for device memory, however we should move to use posted writes. This is expected to work on most hardware, however we will need to support a non-posted option for some broken hardware. Reviewed by: imp, manu, bcr (manpage) Differential Revision: https://reviews.freebsd.org/D29722 --- share/man/man9/bus_space.9 | 7 ++++++- sys/arm64/arm64/bus_machdep.c | 6 +++++- sys/arm64/include/bus.h | 1 + sys/arm64/include/vm.h | 5 +++++ usr.bin/vmstat/vmstat.c | 3 +++ 5 files changed, 20 insertions(+), 2 deletions(-) diff --git a/share/man/man9/bus_space.9 b/share/man/man9/bus_space.9 index 80e041dea4eb..9d5ca602acfe 100644 --- a/share/man/man9/bus_space.9 +++ b/share/man/man9/bus_space.9 @@ -52,7 +52,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 7, 2020 +.Dd May 1, 2021 .Dt BUS_SPACE 9 .Os .Sh NAME @@ -876,6 +876,11 @@ call should fail. If this flag is not specified, the system may map the space in whatever way is most convenient. +.It Dv BUS_SPACE_MAP_NONPOSTED +Try to map the space using non-posted device memory. +This is to support buses and devices where mapping with posted device +memory is unsupported or broken. +This flag is currently only available on arm64. .El .Pp Not all combinations of flags make sense or are supported with all diff --git a/sys/arm64/arm64/bus_machdep.c b/sys/arm64/arm64/bus_machdep.c index 69d7c5b591b2..b2136af38cad 100644 --- a/sys/arm64/arm64/bus_machdep.c +++ b/sys/arm64/arm64/bus_machdep.c @@ -99,9 +99,13 @@ static int generic_bs_map(void *t, bus_addr_t bpa, bus_size_t size, int flags, bus_space_handle_t *bshp) { + vm_memattr_t ma; void *va; - va = pmap_mapdev(bpa, size); + ma = VM_MEMATTR_DEVICE; + if (flags == BUS_SPACE_MAP_NONPOSTED) + ma = VM_MEMATTR_DEVICE_NP; + va = pmap_mapdev_attr(bpa, size, ma); if (va == NULL) return (ENOMEM); *bshp = (bus_space_handle_t)va; diff --git a/sys/arm64/include/bus.h b/sys/arm64/include/bus.h index a2bd432a5de5..61573b27728d 100644 --- a/sys/arm64/include/bus.h +++ b/sys/arm64/include/bus.h @@ -85,6 +85,7 @@ #define BUS_SPACE_MAP_CACHEABLE 0x01 #define BUS_SPACE_MAP_LINEAR 0x02 #define BUS_SPACE_MAP_PREFETCHABLE 0x04 +#define BUS_SPACE_MAP_NONPOSTED 0x08 #define BUS_SPACE_UNRESTRICTED (~0) diff --git a/sys/arm64/include/vm.h b/sys/arm64/include/vm.h index 3df3af24c010..e479aab52e26 100644 --- a/sys/arm64/include/vm.h +++ b/sys/arm64/include/vm.h @@ -36,7 +36,12 @@ #define VM_MEMATTR_WRITE_THROUGH 3 #define VM_MEMATTR_DEVICE_nGnRE 4 +/* + * VM_MEMATTR_DEVICE can be changed to VM_MEMATTR_DEVICE_nGnRE when + * the PCI drivers use VM_MEMATTR_DEVICE_NP for their config space. + */ #define VM_MEMATTR_DEVICE VM_MEMATTR_DEVICE_nGnRnE +#define VM_MEMATTR_DEVICE_NP VM_MEMATTR_DEVICE_nGnRnE #ifdef _KERNEL /* If defined vmstat will try to use both of these in a switch statement */ diff --git a/usr.bin/vmstat/vmstat.c b/usr.bin/vmstat/vmstat.c index 403dc6e2a054..ba1dc9eef883 100644 --- a/usr.bin/vmstat/vmstat.c +++ b/usr.bin/vmstat/vmstat.c @@ -1570,6 +1570,9 @@ display_object(struct kinfo_vmobject *kvo) #ifdef VM_MEMATTR_DEVICE MEMATTR_STR(VM_MEMATTR_DEVICE, "DEV") #endif +#ifdef VM_MEMATTR_DEVICE_NP + MEMATTR_STR(VM_MEMATTR_DEVICE, "NP") +#endif #ifdef VM_MEMATTR_CACHEABLE MEMATTR_STR(VM_MEMATTR_CACHEABLE, "C") #endif