From owner-freebsd-hackers@freebsd.org Sun Apr 9 10:13:49 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE0B9D347F9 for ; Sun, 9 Apr 2017 10:13:49 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: from mail-pf0-x243.google.com (mail-pf0-x243.google.com [IPv6:2607:f8b0:400e:c00::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D9CC1A5 for ; Sun, 9 Apr 2017 10:13:49 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: by mail-pf0-x243.google.com with SMTP id i5so3126316pfc.3 for ; Sun, 09 Apr 2017 03:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=XzePEJ9rJfqoPW0mLbD6oYy8bbBOiUmYY+DWmXcf8YE=; b=Lg9IcjOeCme4j0qPFLYygXO8o9G8O7CS2oY6bYFPh+IyDWV1xGcx/gLXn/fzWZTPAv DlCq2ieNo4BSZpqXAoNKqS0f6J00eFVsLYEoB3WhzsDWfNgxb6/IMnJyZJKYc8WYEuW6 sRTM5bZtZL9Mpag3rFuL8PCoomnrACk8QqpH5nzv2qUG16uEWGVeX7Snz6cao3YBmDUY KIjARLxThR58u0ziX2R4uMH0Yh4fv39DhDdoiHuQmSshxT0w+u25P2zel/iY72UsNwKR y2KYvxP8dw6KzAp3I0NJs8zSHkg2W5dyW1Y9ibFASBQrSUixG6hp45yI2aBQenaEJapp 5FEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=XzePEJ9rJfqoPW0mLbD6oYy8bbBOiUmYY+DWmXcf8YE=; b=mQAt8EUyxXpBTviMhUgGsarB0DK6DUyfb4uMIyBiYcTKfrXnGtIHSrE3a6CzPI0+eW r1RadyvL5tv7U9hS2XuErFCYYNXbq4C4N5SxTgdm3ZQtld6ZPqw8RYttkR4OzPnTFXVy ic9okfap03TKeueOyakM0mwCkLbik8NWbSA8OoqDbPgURjR60o4aVi4SkGGXvFLbuuBL +nREjbSZoIHDz9e+im1dKUupnaaiL88JMRt0mu03/t3idTmgUSRe9HEvyRy9acOCkJ8z 0Wb8YZPehJKmPVG0T+YTQpRyx9AgJf3XhniRdx6Vn4WnAUkGvfep4uE/LFujxlDsFvNR 3vCg== X-Gm-Message-State: AN3rC/7XiLg11OyTo5B8+tivHdDhvFl7Qlg73Achb/iceHDLL1pEaD2GOS1ulr1fZNzqvw== X-Received: by 10.98.103.1 with SMTP id b1mr1919350pfc.184.1491732828452; Sun, 09 Apr 2017 03:13:48 -0700 (PDT) Received: from [192.168.0.100] ([110.64.91.54]) by smtp.gmail.com with ESMTPSA id x30sm18654332pgc.2.2017.04.09.03.13.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Apr 2017 03:13:47 -0700 (PDT) Subject: Re: Understanding the FreeBSD locking mechanism To: Ed Schouten References: Cc: freebsd-hackers@freebsd.org From: Yubin Ruan Message-ID: Date: Sun, 9 Apr 2017 18:13:40 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Apr 2017 10:13:49 -0000 On 2017/4/6 17:31, Ed Schouten wrote: > Hi Yubin, > > 2017-04-06 11:16 GMT+02:00 Yubin Ruan : >> Does this function provides the ordinary "spinlock" functionality? There >> is no special "test-and-set" instruction, and neither any extra locking >> to protect internal data structure manipulation. Isn't this subjected to >> race condition? > > Locking a spinlock is done through macro mtx_lock_spin(), which > expands to __mtx_lock_spin() in sys/sys/mutex.h. That macro first > calls into the function you looked at, spinlock_enter(), to disable > interrupts. It then calls into the _mtx_obtain_lock_fetch() to do the > test-and-set operation you were looking for. Thanks for replying. I have read some of those codes. Just a few more questions, if you don't mind: (1) why are spinlocks forced to disable interrupt in FreeBSD? From the book "The design and implementation of the FreeBSD Operating System", the authors say "spinning can result in deadlock if a thread interrupted the thread that held a mutex and then tried to acquire the mutex"...(section 4.3, Mutex Synchronization, paragraph 4) I don't get the point why a spinlock(or *spin mutex* in the FreeBSD world) has to disable interrupt. Being interrupted does not necessarily mean a deadlock. Assume that thread A holding a lock T gets interrupted by another thread B(context switch here) and thread B try to acquire the lock T. After finding out that lock T has already been acquired, thread B will just spin until it gets preempted, after which thread A gets waken up and run and release the lock T. So, you see there is not necessarily any deadlock even if thread A get interrupted. I can only remember two conditions where using spinlock without disabling interrupts will cause deadlock: #######1, spinlock used in an interrupt handler If a thread A holding a spinlock T get interrupted and the interrupt handler responsible for this interrupt try to acquire T, then we have deadlock, because A would never have a chance to run before the interrupt handler return, and the interrupt handler, unfortunately, will continue to spin ... so in this situation, one has to disable interrupt before spinning. As far as I know, in Linux, they provide two kinds of spinlocks: spin_lock(..); /* spinlock that does not disable interrupts */ spin_lock_irqsave(...); /* spinlock that disable local interrupt */ #######2, priority inversion problem If thread B with a higher priority get in and try to acquire the lock that thread A currently holds, then thread B would spin, while at the same time thread A has no chance to run because it has lower priority, thus not being able to release the lock. (I haven't investigate enough into the source code, so I don't know how FreeBSD and Linux handle this priority inversion problem. Maybe they use priority inheritance or random boosting?) thanks, Yubin Ruan