From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 19 17:23:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F7331065670 for ; Tue, 19 Apr 2011 17:23:29 +0000 (UTC) (envelope-from cliftonr@oz.volcano.org) Received: from hrndva-omtalb.mail.rr.com (hrndva-omtalb.mail.rr.com [71.74.56.124]) by mx1.freebsd.org (Postfix) with ESMTP id 48DD98FC15 for ; Tue, 19 Apr 2011 17:23:29 +0000 (UTC) Received: from hrndva-omtalb.mail.rr.com ([10.128.143.52]) by hrndva-qmta02.mail.rr.com with ESMTP id <20110419171344827.BALM5185@hrndva-qmta02.mail.rr.com> for ; Tue, 19 Apr 2011 17:13:44 +0000 X-Authority-Analysis: v=1.1 cv=qyUSAyc82z9xLljZQc9ErY9Tl2GSEfqK/XYZS35I9d8= c=1 sm=0 a=z1TLwsU0kBEA:10 a=q1A4uwALsQMA:10 a=TgPToyY2OQsA:10 a=kj9zAlcOel0A:10 a=G5OLwwqwWgs+1dCEPNHTSw==:17 a=6I5d2MoRAAAA:8 a=jb__rZ8GAAAA:8 a=H9iEQFZ8AAAA:8 a=ac3IfjoEGWV7Sfb5AkkA:9 a=6HQiZRNdpsNgcHOFxLYA:7 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 a=sHp_62vNEjwA:10 a=fZFZujrNNEQA:10 a=G5OLwwqwWgs+1dCEPNHTSw==:117 X-Cloudmark-Score: 0 X-Originating-IP: 75.80.196.236 Received: from [75.80.196.236] ([75.80.196.236:8576] helo=oz.volcano.org) by hrndva-oedge02.mail.rr.com (envelope-from ) (ecelerity 2.2.3.46 r()) with ESMTP id C6/17-08375-672CDAD4; Tue, 19 Apr 2011 17:12:22 +0000 Received: by oz.volcano.org (Postfix, from userid 1001) id D614E5082A; Tue, 19 Apr 2011 07:12:21 -1000 (HST) Date: Tue, 19 Apr 2011 07:12:21 -1000 From: Clifton Royston To: freebsd-hackers@freebsd.org Message-ID: <20110419171221.GA49924@lava.net> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20110419120029.1394510656E3@hub.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201104181712.14457.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Subject: Re: SMP question w.r.t. reading kernel variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Apr 2011 17:23:29 -0000 On Tue, Apr 19, 2011 at 12:00:29PM +0000, freebsd-hackers-request@freebsd.org wrote: > Subject: Re: SMP question w.r.t. reading kernel variables > To: Rick Macklem > Cc: freebsd-hackers@freebsd.org > Message-ID: <201104181712.14457.jhb@freebsd.org> [John Baldwin] > On Monday, April 18, 2011 4:22:37 pm Rick Macklem wrote: > > > On Sunday, April 17, 2011 3:49:48 pm Rick Macklem wrote: ... > > All of this makes sense. What I was concerned about was memory cache > > consistency and whet (if anything) has to be done to make sure a thread > > doesn't see a stale cached value for the memory location. > > > > Here's a generic example of what I was thinking of: > > (assume x is a global int and y is a local int on the thread's stack) > > - time proceeds down the screen > > thread X on CPU 0 thread Y on CPU 1 > > x = 0; > > x = 0; /* 0 for x's location in CPU 1's memory cache */ > > x = 1; > > y = x; > > --> now, is "y" guaranteed to be 1 or can it get the stale cached 0 value? > > if not, what needs to be done to guarantee it? > > Well, the bigger problem is getting the CPU and compiler to order the > instructions such that they don't execute out of order, etc. Because of that, > even if your code has 'x = 0; x = 1;' as adjacent threads in thread X, > the 'x = 1' may actually execute a good bit after the 'y = x' on CPU 1. Actually, as I recall the rules for C, it's worse than that. For this (admittedly simplified scenario), "x=0;" in thread X may never execute unless it's declared volatile, as the compiler may optimize it out and emit no code for it. > Locks force that to sychronize as the CPUs coordinate around the lock cookie > (e.g. the 'mtx_lock' member of 'struct mutex'). > > > Also, I see cases of: > > mtx_lock(&np); > > np->n_attrstamp = 0; > > mtx_unlock(&np); > > in the regular NFS client. Why is the assignment mutex locked? (I had assumed > > it was related to the above memory caching issue, but now I'm not so sure.) > > In general I think writes to data that are protected by locks should always be > protected by locks. In some cases you may be able to read data using "weaker" > locking (where "no locking" can be a form of weaker locking, but also a > read/shared lock is weak, and if a variable is protected by multiple locks, > then any singe lock is weak, but sufficient for reading while all of the > associated locks must be held for writing) than writing, but writing generally > requires "full" locking (write locks, etc.). What he said. In addition to all that, lock operations generate "atomic" barriers which a compiler or optimizer is prevented from moving code across. -- Clifton -- Clifton Royston -- cliftonr@iandicomputing.com / cliftonr@lava.net President - I and I Computing * http://www.iandicomputing.com/ Custom programming, network design, systems and network consulting services