From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 20 21:40:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99275106566B; Wed, 20 Apr 2011 21:40:23 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3E5468FC0A; Wed, 20 Apr 2011 21:40:22 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkEHAFpSr02DaFvO/2dsb2JhbACEUJMqjkK2G5ERgSmDTnoEjiI X-IronPort-AV: E=Sophos;i="4.64,247,1301889600"; d="scan'208";a="119031229" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 20 Apr 2011 17:40:22 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 632DFB3F32; Wed, 20 Apr 2011 17:40:22 -0400 (EDT) Date: Wed, 20 Apr 2011 17:40:22 -0400 (EDT) From: Rick Macklem To: alc@freebsd.org Message-ID: <1904651983.378809.1303335622389.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1793102603.378435.1303334876373.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE7 (Win)/6.0.10_GA_2692) Cc: freebsd-hackers Subject: Re: SMP question w.r.t. reading kernel variables X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Apr 2011 21:40:23 -0000 > [good stuff snipped for brevity] > > > > 1. Set MNTK_UNMOUNTF > > 2. Acquire a standard FreeBSD mutex "m". > > 3. Update some data structures. > > 4. Release mutex "m". > > > > Then, other threads that acquire "m" after step 4 has occurred will > > see > > MNTK_UNMOUNTF as set. But, other threads that beat thread X to step > > 2 > > may > > or may not see MNTK_UNMOUNTF as set. > > > First off, Alan, thanks for the great explanation. I think it would be > nice if this was captured somewhere in the docs, if it isn't already > there somewhere (I couldn't spot it, but that doesn't mean > anything:-). > > > The question that I have about your specific scenario is concerned > > with > > VOP_SYNC(). Do you care if another thread performing nfscl_getcl() > > after > > thread X has performed VOP_SYNC() doesn't see MNTK_UNMOUNTF as set? > > Well, no and yes. It doesn't matter if it doesn't see it after thread > X > performed nfs_sync(), but it does matter that the threads calling > nfscl_getcl() > see it before they compete with thread X for the sleep lock. > > > Another > > relevant question is "Does VOP_SYNC() acquire and release the same > > mutex as > > nfscl_umount() and nfscl_getcl()?" > > > No. So, to get this to work correctly it sounds like I have to do one > of the following: > 1 - mtx_lock(m); mtx_unlock(m); in nfs_sync(), where "m" is the mutex > used > by nfscl_getcl() for the NFS open/lock state. > or > 2 - mtx_lock(m); mtx_unlock(m); mtx_lock(m); before the point where I > care > that the threads executing nfscl_getcl() see MNTK_UMOUNTF set in > nfscl_umount(). > or > 3 - mtx_lock(m2); mtx_unlock(m2); in nfscl_getcl(), where m2 is the > mutex used > by thread X when setting MNTK_UMOUNTF, before mtx_lock(m); and then > testing > MNTK_UMOUNTF plus acquiring the sleep lock. (By doing it before, I can > avoid > any LOR issue and do an msleep() without worrying about having two > mutex locks.) > > I think #3 reads the best, so I'll probably do that one. > > One more question, if you don't mind. > > Is step 3 in your explanation necessary for this to work? If it is, I > can just create > some global variable that I assign a value to between mtx_lock(m2); > mtx_unlock(m2); > but it won't be used for anything, so I thought I'd check if it is > necessary? > Oops, I screwed up this question. For my #3, all that needs to be done in nfscl_getcl() before I care if it sees MNTK_UMOUNTF set is mtx_lock(m2); since that has already gone through your steps 1-4. The question w.r.t. do you really need your step 3 would apply to the cases where I was using "m" (the mutex nfscl_umount() and nfscl_getcl() already use instead of the one used by thread X). rick