From owner-freebsd-arch@freebsd.org Thu Dec 29 15:31:43 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5F2CC963E1 for ; Thu, 29 Dec 2016 15:31:43 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wj0-x244.google.com (mail-wj0-x244.google.com [IPv6:2a00:1450:400c:c01::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 34F9016B7 for ; Thu, 29 Dec 2016 15:31:43 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wj0-x244.google.com with SMTP id qs7so20334959wjc.1 for ; Thu, 29 Dec 2016 07:31:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=BCc3pYoQktWBW4mNkJhejVp3BfLObhJBmxU1Oeodw+U=; b=VnooW3XXuGJSZxsUMMo5HHLCgdod2dUSxW93Y/5KvN41Z5TZY7gExiLmz6DFECwzYf tXeLyL4sZyWrarXIKLWiHyKP1vyVI7fwgJ1/4vFtGrXNAcNmJQg7kDSxxRNslsCV1BNM h7V9LvMO59SddyxX8HMi1e3toVqcazgLv68jHXuiXs/FjRKYTei4bzCM5UkvWoZ6G1qN 2DqZjh26SPya3AwUzoUqoLLetRasc7fHgRRshsTduLy05IKH8oaNsyIKo/CAB3ohLTaa JTuNP89WmmxsC5SXhzDsmMjM3Via9J34z79MxxcM5TbvPqvUAZMpbbuniH7NbNVf5QT5 V+SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=BCc3pYoQktWBW4mNkJhejVp3BfLObhJBmxU1Oeodw+U=; b=RNksOxHVQeScmy54UIUQGEsKCwWcmhJ+zCcksMJUtM6R2NkfVAqc0aqOJ1bNHMa5Pd oF68KqCDO3aW6IYvXbt8YoP9dsKQIUpGmB+wLbiJtO8GIUZop53xCr7FDuQaM5lDy7rd N2aKHme8MxrhysteVI0ma1gTpKa7Vj+UAQiIkWaXCHQXXQ5I5MESsfDLGpheLYch7VEW NLOVldBadO+YGlbKxFWSQnaCzO3xzqGYaxPirAvqZIt+RxNIjmKGPq2QivgSKnOhMEoo nIsY5LrsqlNF5SSBrV5FD+VBGANDTwVgs09YnIbmbs9Gus6o77W3hX16rERmwPBV5VYV IVjg== X-Gm-Message-State: AIkVDXL4/62jSLicot4QUeyDK1XjCd+uGiL+lsZZWjxI3JPmCX7k6sM0pjbqef+qTn3qyw== X-Received: by 10.194.93.104 with SMTP id ct8mr43685498wjb.87.1483025501314; Thu, 29 Dec 2016 07:31:41 -0800 (PST) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by smtp.gmail.com with ESMTPSA id f76sm65954587wmd.15.2016.12.29.07.31.40 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Thu, 29 Dec 2016 07:31:40 -0800 (PST) Date: Thu, 29 Dec 2016 16:31:38 +0100 From: Mateusz Guzik To: Bruce Evans Cc: freebsd-arch@freebsd.org Subject: Re: __read_only in the kernel Message-ID: <20161229153138.GC29676@dft-labs.eu> References: <20161127212503.GA23218@dft-labs.eu> <20161130011033.GA20999@dft-labs.eu> <20161201070246.S1023@besplex.bde.org> <20161229114554.GA29676@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20161229114554.GA29676@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2016 15:31:43 -0000 On Thu, Dec 29, 2016 at 12:45:55PM +0100, Mateusz Guzik wrote: > On Thu, Dec 01, 2016 at 08:05:54AM +1100, Bruce Evans wrote: > > On Wed, 30 Nov 2016, Mateusz Guzik wrote: > > >>diff --git a/sys/amd64/include/param.h b/sys/amd64/include/param.h > > >>index a619e395..ab66e79 100644 > > >>--- a/sys/amd64/include/param.h > > >>+++ b/sys/amd64/include/param.h > > >>@@ -152,4 +152,6 @@ > > >> #define INKERNEL(va) (((va) >= DMAP_MIN_ADDRESS && (va) < DMAP_MAX_ADDRESS) \ > > >> || ((va) >= VM_MIN_KERNEL_ADDRESS && (va) < VM_MAX_KERNEL_ADDRESS)) > > >> > > >>+#define __read_mostly __attribute__((__section__(".data.read_mostly"))) > > >>+ > > >> #endif /* !_AMD64_INCLUDE_PARAM_H_ */ > > > > 1. Hard-coded gccism: __attribute__(). > > 1a. Non-use of the FreeBSD macro __section() whose purpose is to make it > > easy to avoid using __attribute__() for precisely what it is used for > > here. > > 2. Misplaced definition. Such definitions go in . This one > > has no dependencies on amd64 except possibly for bugs elsewhere, but > > is only in an amd64 header. > > > [..] > > According to userland tests, section statements like the above and the > > ones in don't need any linker support to work, since they > > create sections as necessary. > > > > So the above definition in should almost perfectly for > > all arches, even without linker support. Style bug (1) is smaller if > > it is there. > > > > I wanted to avoid providing the definition for archs which don't have > the linker script bit and this was the only header I found which is md > and effectively always included. > > Indeed it seems the section is harmless even without the explicit > support. > > > >>diff --git a/sys/conf/ldscript.amd64 b/sys/conf/ldscript.amd64 > > >>index 5d86b03..ae98447 100644 > > >>--- a/sys/conf/ldscript.amd64 > > >>+++ b/sys/conf/ldscript.amd64 > > >>@@ -151,6 +151,11 @@ SECTIONS > > >> KEEP (*(.gnu.linkonce.d.*personality*)) > > >> } > > >> .data1 : { *(.data1) } > > >>+ .data_read_mostly : > > >>+ { > > >>+ *(.data.read_mostly) > > >>+ . = ALIGN(64); > > >>+ } > > >> _edata = .; PROVIDE (edata = .); > > >> __bss_start = .; > > >> .bss : > > > > For arches without this linker support, the variables would be grouped > > but not aligned so much. > > > > Aligning the subsection seems to be useless anyway. This only aligns > > the first variable in the subsection. Most variables are still only > > aligned according to their natural or specified alignment. This is > > rarely as large as 64. But I think variables in the subsection can > > be more aligned than the subsection. If they had to be (as in a.out), > > then it is the responsibility of the linker to align the subsection > > to more than the default if a single variable in the subsection needs > > more than the default. > > > > With the indended use grouping side-by-side is beneficial - as the vars > in question are supposed to be rarely modified, there is no problem with > them sharing a cache line. Making them all use dedicated lines would > only waste memory. > > That said, what about the patch below. I also grepped the tree and found > 2 surprises, handled in the patch. > Scratch the previous patch. I extended it with __exclusive_cache_line (happy with ideas for a better name) - to be used for something which has to be alone in the cacheline, e.g. an atomic counter. diff --git a/sys/compat/linuxkpi/common/include/linux/compiler.h b/sys/compat/linuxkpi/common/include/linux/compiler.h index c780abc..c32f1fa 100644 --- a/sys/compat/linuxkpi/common/include/linux/compiler.h +++ b/sys/compat/linuxkpi/common/include/linux/compiler.h @@ -67,7 +67,6 @@ #define typeof(x) __typeof(x) #define uninitialized_var(x) x = x -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) #define __always_unused __unused #define __must_check __result_use_check diff --git a/sys/conf/ldscript.amd64 b/sys/conf/ldscript.amd64 index 5d86b03..45685a4 100644 --- a/sys/conf/ldscript.amd64 +++ b/sys/conf/ldscript.amd64 @@ -150,6 +150,17 @@ SECTIONS *(.data .data.* .gnu.linkonce.d.*) KEEP (*(.gnu.linkonce.d.*personality*)) } + . = ALIGN(64); + .data.read_mostly : + { + *(.data.read_mostly) + } + . = ALIGN(64); + .data.exclusive_cache_line : + { + *(.data.exclusive_cache_line) + } + . = ALIGN(64); .data1 : { *(.data1) } _edata = .; PROVIDE (edata = .); __bss_start = .; diff --git a/sys/dev/drm2/drm_os_freebsd.h b/sys/dev/drm2/drm_os_freebsd.h index dc01c6a..11c9feb 100644 --- a/sys/dev/drm2/drm_os_freebsd.h +++ b/sys/dev/drm2/drm_os_freebsd.h @@ -80,7 +80,6 @@ typedef void irqreturn_t; #define __init #define __exit -#define __read_mostly #define BUILD_BUG_ON(x) CTASSERT(!(x)) #define BUILD_BUG_ON_NOT_POWER_OF_2(x) diff --git a/sys/sys/systm.h b/sys/sys/systm.h index a1ce9b4..719e063 100644 --- a/sys/sys/systm.h +++ b/sys/sys/systm.h @@ -445,4 +445,8 @@ extern void (*softdep_ast_cleanup)(void); void counted_warning(unsigned *counter, const char *msg); +#define __read_mostly __section(".data.read_mostly") +#define __exclusive_cache_line __aligned(CACHE_LINE_SIZE) \ + __section(".data.exclusive_cache_line") + #endif /* !_SYS_SYSTM_H_ */ -- Mateusz Guzik