From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 20 21:27:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9BC7106564A for ; Wed, 20 Apr 2011 21:27:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 637C98FC08 for ; Wed, 20 Apr 2011 21:27:57 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkEHABVPr02DaFvO/2dsb2JhbACEUZMpjkKIb60pkRaBKYNOegSOIg X-IronPort-AV: E=Sophos;i="4.64,247,1301889600"; d="scan'208";a="119030013" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 20 Apr 2011 17:27:56 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 7266AB3F73; Wed, 20 Apr 2011 17:27:56 -0400 (EDT) Date: Wed, 20 Apr 2011 17:27:56 -0400 (EDT) From: Rick Macklem To: alc@freebsd.org Message-ID: <1793102603.378435.1303334876373.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) Cc: freebsd-hackers Subject: Re: SMP question w.r.t. reading kernel variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2011 21:27:57 -0000 [good stuff snipped for brevity] > > 1. Set MNTK_UNMOUNTF > 2. Acquire a standard FreeBSD mutex "m". > 3. Update some data structures. > 4. Release mutex "m". > > Then, other threads that acquire "m" after step 4 has occurred will > see > MNTK_UNMOUNTF as set. But, other threads that beat thread X to step 2 > may > or may not see MNTK_UNMOUNTF as set. > First off, Alan, thanks for the great explanation. I think it would be nice if this was captured somewhere in the docs, if it isn't already there somewhere (I couldn't spot it, but that doesn't mean anything:-). > The question that I have about your specific scenario is concerned > with > VOP_SYNC(). Do you care if another thread performing nfscl_getcl() > after > thread X has performed VOP_SYNC() doesn't see MNTK_UNMOUNTF as set? Well, no and yes. It doesn't matter if it doesn't see it after thread X performed nfs_sync(), but it does matter that the threads calling nfscl_getcl() see it before they compete with thread X for the sleep lock. > Another > relevant question is "Does VOP_SYNC() acquire and release the same > mutex as > nfscl_umount() and nfscl_getcl()?" > No. So, to get this to work correctly it sounds like I have to do one of the following: 1 - mtx_lock(m); mtx_unlock(m); in nfs_sync(), where "m" is the mutex used by nfscl_getcl() for the NFS open/lock state. or 2 - mtx_lock(m); mtx_unlock(m); mtx_lock(m); before the point where I care that the threads executing nfscl_getcl() see MNTK_UMOUNTF set in nfscl_umount(). or 3 - mtx_lock(m2); mtx_unlock(m2); in nfscl_getcl(), where m2 is the mutex used by thread X when setting MNTK_UMOUNTF, before mtx_lock(m); and then testing MNTK_UMOUNTF plus acquiring the sleep lock. (By doing it before, I can avoid any LOR issue and do an msleep() without worrying about having two mutex locks.) I think #3 reads the best, so I'll probably do that one. One more question, if you don't mind. Is step 3 in your explanation necessary for this to work? If it is, I can just create some global variable that I assign a value to between mtx_lock(m2); mtx_unlock(m2); but it won't be used for anything, so I thought I'd check if it is necessary? Thanks again for the clear explanation, rick