From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 18 20:22:39 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35113106566B; Mon, 18 Apr 2011 20:22:39 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id D0C3F8FC14; Mon, 18 Apr 2011 20:22:38 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Au4AAMecrE2DaFvO/2dsb2JhbACET5JnjnyIb6sVkHWBKYFZgXV6BI4A X-IronPort-AV: E=Sophos;i="4.64,234,1301889600"; d="scan'208";a="118738597" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 18 Apr 2011 16:22:37 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B7545B3F4D; Mon, 18 Apr 2011 16:22:37 -0400 (EDT) Date: Mon, 18 Apr 2011 16:22:37 -0400 (EDT) From: Rick Macklem To: John Baldwin Message-ID: <857645844.237335.1303158157618.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201104180831.33796.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) Cc: freebsd-hackers@freebsd.org Subject: Re: SMP question w.r.t. reading kernel variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Apr 2011 20:22:39 -0000 > On Sunday, April 17, 2011 3:49:48 pm Rick Macklem wrote: > > Hi, > > > > I should know the answer to this, but... When reading a global > > kernel > > variable, where its modifications are protected by a mutex, is it > > necessary to get the mutex lock to just read its value? > > > > For example: > > A if ((mp->mnt_kern_flag & MNTK_UNMOUNTF) != 0) > > return (EPERM); > > versus > > B MNT_ILOCK(mp); > > if ((mp->mnt_kern_flag & MNTK_UNMOUNTF) != 0) { > > MNT_IUNLOCK(mp); > > return (EPERM); > > } > > MNT_IUNLOCK(mp); > > > > My hunch is that B is necessary if you need an up-to-date value > > for the variable (mp->mnt_kern_flag in this case). > > > > Is that correct? > > You already have good followups from Attilio and Kostik, but one thing > to keep > in mind is that if a simple read is part of a larger "atomic > operation" then > it may still need a lock. In this case Kostik points out that another > lock > prevents updates to mnt_kern_flag so that this is safe. However, if > not for > that you would need to consider the case that another thread sets the > flag on > the next instruction. Even the B case above might still have that > problem > since you drop the lock right after checking it and the rest of the > function > is implicitly assuming the flag is never set perhaps (or it needs to > handle > the case that the flag might become set in the future while > MNT_ILOCK() is > dropped). > > One way you can make that code handle that race is by holding > MNT_ILOCK() > around the entire function, but that approach is often only suitable > for a > simple routine. > All of this makes sense. What I was concerned about was memory cache consistency and whet (if anything) has to be done to make sure a thread doesn't see a stale cached value for the memory location. Here's a generic example of what I was thinking of: (assume x is a global int and y is a local int on the thread's stack) - time proceeds down the screen thread X on CPU 0 thread Y on CPU 1 x = 0; x = 0; /* 0 for x's location in CPU 1's memory cache */ x = 1; y = x; --> now, is "y" guaranteed to be 1 or can it get the stale cached 0 value? if not, what needs to be done to guarantee it? For the original example, I am fine so long as the bit is seen as set after dounmount() has set it. Also, I see cases of: mtx_lock(&np); np->n_attrstamp = 0; mtx_unlock(&np); in the regular NFS client. Why is the assignment mutex locked? (I had assumed it was related to the above memory caching issue, but now I'm not so sure.) Thanks a lot for all the good responses, rick ps: I guess it comes down to whether or not "atomic" includes ensuring memory cache consistency. I'll admit I assumed "atomic" meant that the memory access or modify couldn't be interleaved with one done to the same location by another CPU, but not memory cache consistency.