From owner-svn-src-all@freebsd.org Sun Mar 26 20:15:10 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B95BD1EF80; Sun, 26 Mar 2017 20:15:10 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3355812F7; Sun, 26 Mar 2017 20:15:10 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v2QKF98g062427; Sun, 26 Mar 2017 20:15:09 GMT (envelope-from rwatson@FreeBSD.org) Received: (from rwatson@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v2QKF9k9062425; Sun, 26 Mar 2017 20:15:09 GMT (envelope-from rwatson@FreeBSD.org) Message-Id: <201703262015.v2QKF9k9062425@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rwatson set sender to rwatson@FreeBSD.org using -f From: Robert Watson Date: Sun, 26 Mar 2017 20:15:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r316003 - head/share/man/man9 X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Mar 2017 20:15:10 -0000 Author: rwatson Date: Sun Mar 26 20:15:08 2017 New Revision: 316003 URL: https://svnweb.freebsd.org/changeset/base/316003 Log: Add a man page for the kernel's dynamic per-CPU memory allocator. MFC after: 3 days Added: head/share/man/man9/dpcpu.9 (contents, props changed) Modified: head/share/man/man9/Makefile Modified: head/share/man/man9/Makefile ============================================================================== --- head/share/man/man9/Makefile Sun Mar 26 19:47:43 2017 (r316002) +++ head/share/man/man9/Makefile Sun Mar 26 20:15:08 2017 (r316003) @@ -116,6 +116,7 @@ MAN= accept_filter.9 \ disk.9 \ dnv.9 \ domain.9 \ + dpcpu.9 \ drbr.9 \ driver.9 \ DRIVER_MODULE.9 \ Added: head/share/man/man9/dpcpu.9 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/man/man9/dpcpu.9 Sun Mar 26 20:15:08 2017 (r316003) @@ -0,0 +1,163 @@ +.\"- +.\" Copyright (c) 2017 Robert N. M. Watson +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd March 26, 2017 +.Dt DPCPU 9 +.Os +.Sh NAME +.Nm dpcpu +.Nd Kernel Dynamic Per-CPU Memory Allocator +.Sh SYNOPSIS +.In sys/pcpu.h +.Ss Per-CPU Variable Definition and Declaration +.Fn DPCPU_DEFINE "type" "name" +.Fn DPCPU_DECLARE "type" "name" +.Ss Current CPU Accessor Functions +.Fn DPCPU_PTR "name" +.Fn DPCPU_GET "name" +.Fn DPCPU_SET "name" "value" +.Ss Named CPU Accessor Functions +.Fn DPCPU_ID_PTR "cpu" "name" +.Fn DPCPU_ID_GET "cpu" "name" +.Fn DPCPU_ID_SET "cpu" "name" "value" +.Sh DESCRIPTION +.Nm +instantiates one instance of a global variable with each CPU in the system. +Dynamically allocated per-CPU variables are defined using +.Fn DPCPU_DEFINE , +which defines a variable of name +.Ar name +and type +.Ar type . +Arbitrary C types may be used, including structures and arrays. +If no initialization is provided, then each per-CPU instance of the variable +will be zero-filled (i.e., as though allocated in BSS): +.Bd -literal -offset 1234 +DPCPU_DEFINE(int, fooint); +.Ed +.Pp +Values may also be initialized statically with the definition, causing each +per-CPU instance to be initialized with the value: +.Bd -literal -offset 1234 +DPCPU_DEFINE(int, fooint) = 1; +.Ed +.Pp +Syntactically, the definition may be treated as a variable. +For example, a dynamic per-CPU variable may be declared as +.Dv static : +.Bd -literal -offset 1234 +static DPCPU_DEFINE(int, fooint); +.Ed +.Pp +.Fn DPCPU_DECLARE +produces a declaration of the per-CPU variable suitable for use in header +files. +.Pp +The current CPU's variable instance can be accessed via +.Nm DPCPU_PTR +(which returns a pointer to the per-CPU instance), +.Nm DPCPU_GET +(which retrieves the value of the per-CPU instance), +and +.Nm DPCPU_SET +(which sets the value of the per-CPU instance). +.Pp +Instances of variables associated with specific CPUs can be accessed via the +.Nm DPCPU_ID_PTR , +.Nm DPCPU_ID_GET , +and +.Nm DPGPU_ID_SET +accessor functions, which accept an additional CPU ID argument, +.Ar cpu . +.Ss Synchronization +In addition to the ordinary synchronization concerns associated with global +variables, which may imply the use of +.Xr atomic 9 , +.Xr mutex 9 , +or other kernel synchronization primitives, it is further the case that +thread migration could dynamically change the instance of a variable being +accessed by a thread between operations. +This requires additional care when reasoning about and protecting per-CPU +variables. +.Pp +For example, it may be desirable to protect access using +.Xr critical_section 9 +to prevent both preemption and migration during use. +Alternatively, it may be desirable to cache the CPU ID at the start of a +sequence of accesses, using suitable synchronization to make non-atomic +sequences safe in the presence of migration. +.Bd -literal -offset 1234 +static DPCPU_DEFINE(int, foo_int); +static DPCPU_DEFINE(struct mutex, foo_lock); + +void +foo_int_increment(void) +{ + int cpu, value; + + /* Safe as atomic access. */ + atomic_add_int(DPCPU_PTR(foo_int), 1); + + /* + * Protect with a critical section, which prevents preemption + * and migration. However, access to instances from remote CPUs + * is not safe, as critical sections prevent concurrent access + * only from the current CPU. + */ + critical_enter(); + value = DPCPU_GET(foo_int); + value++; + DPCPU_SET(foo_int, value); + critical_exit(); + + /* + * Protect with a per-CPU mutex, tolerating migration, but + * potentially accessing the variable from multiple CPUs if + * migration occurs after reading curcpu. Remote access to a + * per-CPU variable is safe as long as the correct mutex is + * acquired. + */ + cpu = curcpu; + mtx_lock(DPCPU_ID_PTR(cpu, foo_lock)); + value = DPCPU_ID_GET(cpu, foo_int); + value++; + DPCPU_ID_SET(cpu, foo_int); + mtx_unlock(DPCPU_ID_PTR(cpu, foo_lock)); +} +.Ed +.Sh SEE ALSO +.Xr atomic 9 , +.Xr critical_enter 9 , +.Xr mutex 9 +.Sh HISTORY +.Nm +was first introduced by +.An Jeff Roberson +in +.Fx 8.0 . +This manual page was written by +.An Robert N. M. Watson.