From owner-freebsd-hackers@freebsd.org Mon Apr 10 04:02:23 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D596D365CF for ; Mon, 10 Apr 2017 04:02:23 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: from mail-oi0-x242.google.com (mail-oi0-x242.google.com [IPv6:2607:f8b0:4003:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32F837A4 for ; Mon, 10 Apr 2017 04:02:23 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: by mail-oi0-x242.google.com with SMTP id w197so10298429oiw.1 for ; Sun, 09 Apr 2017 21:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=wqmoaea+s9YKC28jD3SwkdZ2MtrYqVk7eDe0InvWfv8=; b=sj4jc96EFNt5t/LDwRW71YGB1nuYyO+hhQoccrj+4/Nqfbz2N5i6Xwhj1GfUy7SDY+ YCP5tambaaCVc/DOlluipV/WNliezaWiKhEnBx/zj10g1XDQGgw3P8N0KLpK0+5MVX53 M2FZJe+K70vxn6uHeCfzlWauf7nHCACqRJUj/+ck1zEWFmLDbo0iIP2teqE9TWU/PpMz umy2lh91avEJw23QqSm6mOFaAOLdFByUMXL0hyscqJIR+UI8D/rMg6VFT3EQAVGmZbbJ NL9KkcGTjWwscfw3icMUnn099SvUSuZwreiKiol+v0huBlMU3auSud+uzGCKKQIZB8Mn Xlwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=wqmoaea+s9YKC28jD3SwkdZ2MtrYqVk7eDe0InvWfv8=; b=pPxREbvl3dL+6zrgeQXw6iXsJdPIGuFsc3uY3lYcGzC1u1M5CDOHmM19/V/lXgaTCU +Z/WFJBAWIF0iODGYgtJQelzTNrILYDLf2B9kJyA2w71kjOz8xn9bHVLpzb7wrMhDNSH k6H+4SefCGLfO342xYyGNjEog8E8kmGJjQmXiYHeFb+D0RxWROOO38jdeFuL3H247ETY f4pCy1RECpCqihuu3pxKBaD4fmxvmGC6ugJVUBfkA+pA35QdSGHceCtiWr3/Z7XN/B25 W4Y6Ny9xtGYjrIi/YuTspLUj8TKllS2zradeCyKOADLwRV80j3gx/hU+IuXSdSvVUhBR RQIg== X-Gm-Message-State: AN3rC/6RbfN812Kpr/CF2Ym5BRM9KVPxEM0xx7B6nUFkbtAYfWTwqoGufLCELsgF0ukiVA== X-Received: by 10.157.82.9 with SMTP id e9mr7888758oth.50.1491796942394; Sun, 09 Apr 2017 21:02:22 -0700 (PDT) Received: from [192.168.0.100] ([110.64.91.54]) by smtp.gmail.com with ESMTPSA id p3sm4067200ota.54.2017.04.09.21.02.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Apr 2017 21:02:21 -0700 (PDT) Subject: Re: Understanding the FreeBSD locking mechanism To: Warner Losh References: <3f93930c-7f10-4d0b-35f2-2b07d64081f0@gmail.com> Cc: Ryan Stone , "freebsd-hackers@freebsd.org" , Ed Schouten From: Yubin Ruan Message-ID: Date: Mon, 10 Apr 2017 12:01:51 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 04:02:23 -0000 On 2017/4/10 9:51, Warner Losh wrote: > On Sun, Apr 9, 2017 at 7:28 PM, Yubin Ruan wrote: >> On 2017/4/10 0:24, Ryan Stone wrote: >>> >>> >>> >>> On Sun, Apr 9, 2017 at 6:13 AM, Yubin Ruan >> > wrote: >>> >>> >>> #######1, spinlock used in an interrupt handler >>> If a thread A holding a spinlock T get interrupted and the interrupt >>> handler responsible for this interrupt try to acquire T, then we have >>> deadlock, because A would never have a chance to run before the >>> interrupt handler return, and the interrupt handler, unfortunately, >>> will continue to spin ... so in this situation, one has to disable >>> interrupt before spinning. >>> >>> As far as I know, in Linux, they provide two kinds of spinlocks: >>> >>> spin_lock(..); /* spinlock that does not disable interrupts */ >>> spin_lock_irqsave(...); /* spinlock that disable local interrupt * >>> >>> >>> In the FreeBSD locking style, a spinlock is only used in the case where >>> one needs to synchronize with an interrupt handler. This is why spinlocks >>> always disable local interrupts in FreeBSD. >>> >>> FreeBSD's lock for the first case is the MTX_DEF mutex, which is >>> adaptively-spinning blocking mutex implementation. In short, the MTX_DEF >>> mutex will spin waiting for the lock if the owner is running, but will >>> block if the owner is deschedules. This prevents expensive trips through >>> the scheduler for the common case where the mutex is only held for short >>> periods, without wasting CPU cycles spinning in cases where the owner >>> thread >>> is descheduled and therefore will not be completing soon. >> >> >> Great explanation! I read the man page at: >> >>> >>> https://www.freebsd.org/cgi/man.cgi?query=mutex&sektion=9&apropos=0&manpath=FreeBSD+11.0-RELEASE+and+Ports >> >> and now clear about MTX_DEF and MTX_SPIN mutexs. But, still a few more >> question, if you don't mind: >> >> Is it true that a thread holding a MTX_DEF mutex can be descheduled? >> (shouldn't it disable interrupt like a MTX_SPIN mutex?) It is said on >> the main page that MTX_DEF mutex is used by default in FreeBSD, so its >> usecase must be very common. If a thread holding a MTX_DEF mutex can be >> descheduled, which means that it did not disable interrupt, then we may >> have lots of deadlock here, right? > > Yes, they can be descheduled. But that's not a problem. No other > thread can acquire the MTX_DEF lock. If another thread tries, it will > sleep and wait for the thread that holds the MTX_DEF lock to release > it. Eventually, the thread will get time to run again, and then > release the lock. Threads that just hold a MTX_DEF lock may also > migrate from CPU to CPU too. > > Warner > Does that imply that MTX_DEF should not be used in something like interrupt handler? Putting an interrupt handler into sleep doesn't make so much sense. Yubin >>> #######2, priority inversion problem >>> If thread B with a higher priority get in and try to acquire the lock >>> that thread A currently holds, then thread B would spin, while at the >>> same time thread A has no chance to run because it has lower priority, >>> thus not being able to release the lock. >>> (I haven't investigate enough into the source code, so I don't know >>> how FreeBSD and Linux handle this priority inversion problem. Maybe >>> they use priority inheritance or random boosting?) >>> >>> >>> FreeBSD's spin locks prevent priority inversion by preventing the holder >>> thread from being descheduled. >>> >>> MTX_DEF locks implement priority inheritance. >> >> >> Nice hints. Thanks! >> >> regards, >> Yubin Ruan >> >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"