Date: Mon, 5 Aug 2002 23:05:56 -0700 From: Luigi Rizzo <rizzo@icir.org> To: Peter Wemm <peter@wemm.org> Cc: Terry Lambert <tlambert2@mindspring.com>, smp@freebsd.org Subject: Re: how to create per-cpu variables in SMP kernels ? Message-ID: <20020805230556.C26751@iguana.icir.org> In-Reply-To: <20020805221415.B0C732A7D6@canning.wemm.org>; from peter@wemm.org on Mon, Aug 05, 2002 at 03:14:15PM -0700 References: <20020805015340.A17716@iguana.icir.org> <20020805221415.B0C732A7D6@canning.wemm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Peter,
thanks for the explaination.
I still have a few doubts on this (let's restrict to the -current
case where the code seems more readable):
--- MINOR DETAIL ---
* I wonder why the macro __PCPU_GET() in sys/i386/include/pcpu.h
cannot store directly into __result for operand sizes of 1,2,4
instead of going through a temporary variable. I.e. what would
be wrong in having
#define __PCPU_GET(name) ({ \
__pcpu_type(name) __result; \
\
if (sizeof(__result) == 1) { \
__asm __volatile("movb %%fs:%1,%0" \
: "=r" (__result) \
: "m" (*(u_char *)(__pcpu_offset(name)))); \
} else if (sizeof(__result) == 2) { \
Probably the same holds for __PCPU_SET().
--- OVERALL IMPLEMENTATION OF THE PER-CPU DATA ---
Partly following Terry's description, i thought an arrangement
like the following could be relatively simple to implement and not
require any recourse to assembly code, does not impact the compiler's
ability to do optimizations, and does not require an extra
segment descriptor to access the struct pcpu.
It relies on the following variables, my_pcpu to access the
pcpu data of the local processor, all_pcpu to view all pcpu
data (including our own, at a different mapping in vm space):
struct pcpu *my_pcpu;
struct pcpu *all_pcpu[MAXCPU]; /* XXX volatile */
Early in the boot process we allocate MAXCPU physical pages,
and MAXCPU+1 entries in the VM space. Individual pcpu structs
go at the beginning of each of the physical pages, and the
VM -> physical mapping of the first MAXCPU VM entries is the
same for all processors. Then all_pcpu[i] can be initialized
with a pointer to the beginning of the i-th VM page.
The MAXCPU+1-th VM entry maps differently on each CPU,
so that it effectively permits access to the per-cpu data.
my_pcpu can be initialized with a pointer to the MAXCPU+1-th VM page.
At this point, curproc and all other per-cpu variables for the
local CPU can be accessed through
my_pcpu->curproc
and similar, whereas we can get to other cpu's data with
all_pcpu[i]->curproc
without the need for using %fs or special assembly language to
access these fields.
Then we can discuss how/where to put "volatile" keywords.
In principle, all references through all_pcpu[] should be
readonly and treated as volatile, with perhaps the exception of
some section of code at machine startup. On the contrary we could
safely assume that references through my_pcpu are non-volatile
as the local processor should be the only one to mess with them
Anything wrong with this description ?
cheers
luigi
On Mon, Aug 05, 2002 at 03:14:15PM -0700, Peter Wemm wrote:
...
> Sort-of. There is both a compile time issue and a runtime issue.
>
> Using the %fs:variable segment overrides doesn't make a lot of difference,
> but the compiler is effectively wired so that they are treated as volatile.
...
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020805230556.C26751>