Date: Tue, 19 Apr 2011 07:12:21 -1000 From: Clifton Royston <cliftonr@lava.net> To: freebsd-hackers@freebsd.org Subject: Re: SMP question w.r.t. reading kernel variables Message-ID: <20110419171221.GA49924@lava.net> In-Reply-To: <201104181712.14457.jhb@freebsd.org> References: <20110419120029.1394510656E3@hub.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Apr 19, 2011 at 12:00:29PM +0000, freebsd-hackers-request@freebsd.org wrote:
> Subject: Re: SMP question w.r.t. reading kernel variables
> To: Rick Macklem <rmacklem@uoguelph.ca>
> Cc: freebsd-hackers@freebsd.org
> Message-ID: <201104181712.14457.jhb@freebsd.org>
[John Baldwin]
> On Monday, April 18, 2011 4:22:37 pm Rick Macklem wrote:
> > > On Sunday, April 17, 2011 3:49:48 pm Rick Macklem wrote:
...
> > All of this makes sense. What I was concerned about was memory cache
> > consistency and whet (if anything) has to be done to make sure a thread
> > doesn't see a stale cached value for the memory location.
> >
> > Here's a generic example of what I was thinking of:
> > (assume x is a global int and y is a local int on the thread's stack)
> > - time proceeds down the screen
> > thread X on CPU 0 thread Y on CPU 1
> > x = 0;
> > x = 0; /* 0 for x's location in CPU 1's memory cache */
> > x = 1;
> > y = x;
> > --> now, is "y" guaranteed to be 1 or can it get the stale cached 0 value?
> > if not, what needs to be done to guarantee it?
>
> Well, the bigger problem is getting the CPU and compiler to order the
> instructions such that they don't execute out of order, etc. Because of that,
> even if your code has 'x = 0; x = 1;' as adjacent threads in thread X,
> the 'x = 1' may actually execute a good bit after the 'y = x' on CPU 1.
Actually, as I recall the rules for C, it's worse than that. For
this (admittedly simplified scenario), "x=0;" in thread X may never
execute unless it's declared volatile, as the compiler may optimize it
out and emit no code for it.
> Locks force that to sychronize as the CPUs coordinate around the lock cookie
> (e.g. the 'mtx_lock' member of 'struct mutex').
>
> > Also, I see cases of:
> > mtx_lock(&np);
> > np->n_attrstamp = 0;
> > mtx_unlock(&np);
> > in the regular NFS client. Why is the assignment mutex locked? (I had assumed
> > it was related to the above memory caching issue, but now I'm not so sure.)
>
> In general I think writes to data that are protected by locks should always be
> protected by locks. In some cases you may be able to read data using "weaker"
> locking (where "no locking" can be a form of weaker locking, but also a
> read/shared lock is weak, and if a variable is protected by multiple locks,
> then any singe lock is weak, but sufficient for reading while all of the
> associated locks must be held for writing) than writing, but writing generally
> requires "full" locking (write locks, etc.).
What he said. In addition to all that, lock operations generate
"atomic" barriers which a compiler or optimizer is prevented from
moving code across.
-- Clifton
--
Clifton Royston -- cliftonr@iandicomputing.com / cliftonr@lava.net
President - I and I Computing * http://www.iandicomputing.com/
Custom programming, network design, systems and network consulting services
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110419171221.GA49924>