From owner-freebsd-hackers@freebsd.org Mon Apr 10 01:28:38 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 244F6D3571E for ; Mon, 10 Apr 2017 01:28:38 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: from mail-oi0-x242.google.com (mail-oi0-x242.google.com [IPv6:2607:f8b0:4003:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC868350 for ; Mon, 10 Apr 2017 01:28:37 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: by mail-oi0-x242.google.com with SMTP id w197so9934847oiw.1 for ; Sun, 09 Apr 2017 18:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=m7+gMY3lTT1hn2K6qNpp4NVm7bDJCwBzlsnyQ7RhTrE=; b=u3LNmAxNLhVRNJoBfwUIyPRTLFOqfLiFvBIYYzpGoJ7NM99PnPlprQ4nGbTD/TjV8q W6m81iYn3bAsVD/wvnMexgHx1tMruvi0M60ir4lNfMDpXbg7Go3jeIvX2yMc/WmRjTYx 8SScBBCcM6EjApMObiho0a+Q6MW5t+I2/QxoMuCvwlcY7qJkiiuidxChlR++yXj5cez7 7iP8tivsF4Ha9X3DMWIGmUEJojype/I2q7TiGqOmkdUaIiSrI76NgaQy4njcoI1PeAyt 8dQ4vvEJ++4KvHr+3EipmhlDXF9+K4OiwFgNNix/tma8GadA18BpaNrmTvcla1tIYe8D RbzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=m7+gMY3lTT1hn2K6qNpp4NVm7bDJCwBzlsnyQ7RhTrE=; b=tmsJlzFYpfhsF9Oix7p6IoGT4Nu+7pecdsyIbe2kbfVNrQpt9AvX8liYrJ6ubfdtH5 RxD6B8kaf/agZiDqba9unarU/qJDGoLd5Eoauq/VqU6Ot+0bLLuXK+PJCwvUyO0e0PN6 6fD5kEqEOY88Zgk0WQnP8j38gZhTWg7YC/SI/vYUWvR/5rsJnFKpMelqo5e0ffo/kF2M 18aZRa8Zz7PmauDZR4bfwIcVl+fJlm4nMLPojUInKtBMvjnDQ8eMs/hIO5LUkgyn5rIQ jyb2edBkbp4barJOTIPS07HMhSINRV48uIm4MOUo6YcU05q3SfGXP9P9ZFYlSW94WIYA mEWA== X-Gm-Message-State: AFeK/H0DOo7y1rm2KTLmrueWFt1FkdXtgwsuRZ9JSx2Iinymc38Iyb1U8lNfOXTEt7xm0A== X-Received: by 10.202.228.17 with SMTP id b17mr26697188oih.212.1491787716997; Sun, 09 Apr 2017 18:28:36 -0700 (PDT) Received: from [192.168.0.100] ([110.64.91.54]) by smtp.gmail.com with ESMTPSA id v49sm5638242otb.13.2017.04.09.18.28.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Apr 2017 18:28:35 -0700 (PDT) Subject: Re: Understanding the FreeBSD locking mechanism To: Ryan Stone References: Cc: Ed Schouten , "freebsd-hackers@freebsd.org" From: Yubin Ruan Message-ID: <3f93930c-7f10-4d0b-35f2-2b07d64081f0@gmail.com> Date: Mon, 10 Apr 2017 09:28:25 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 01:28:38 -0000 On 2017/4/10 0:24, Ryan Stone wrote: > > > On Sun, Apr 9, 2017 at 6:13 AM, Yubin Ruan > wrote: > > > #######1, spinlock used in an interrupt handler > If a thread A holding a spinlock T get interrupted and the interrupt > handler responsible for this interrupt try to acquire T, then we have > deadlock, because A would never have a chance to run before the > interrupt handler return, and the interrupt handler, unfortunately, > will continue to spin ... so in this situation, one has to disable > interrupt before spinning. > > As far as I know, in Linux, they provide two kinds of spinlocks: > > spin_lock(..); /* spinlock that does not disable interrupts */ > spin_lock_irqsave(...); /* spinlock that disable local interrupt * > > > In the FreeBSD locking style, a spinlock is only used in the case where > one needs to synchronize with an interrupt handler. This is why spinlocks > always disable local interrupts in FreeBSD. > > FreeBSD's lock for the first case is the MTX_DEF mutex, which is > adaptively-spinning blocking mutex implementation. In short, the MTX_DEF > mutex will spin waiting for the lock if the owner is running, but will > block if the owner is deschedules. This prevents expensive trips through > the scheduler for the common case where the mutex is only held for short > periods, without wasting CPU cycles spinning in cases where the owner thread > is descheduled and therefore will not be completing soon. Great explanation! I read the man page at: > https://www.freebsd.org/cgi/man.cgi?query=mutex&sektion=9&apropos=0&manpath=FreeBSD+11.0-RELEASE+and+Ports and now clear about MTX_DEF and MTX_SPIN mutexs. But, still a few more question, if you don't mind: Is it true that a thread holding a MTX_DEF mutex can be descheduled? (shouldn't it disable interrupt like a MTX_SPIN mutex?) It is said on the main page that MTX_DEF mutex is used by default in FreeBSD, so its usecase must be very common. If a thread holding a MTX_DEF mutex can be descheduled, which means that it did not disable interrupt, then we may have lots of deadlock here, right? > > #######2, priority inversion problem > If thread B with a higher priority get in and try to acquire the lock > that thread A currently holds, then thread B would spin, while at the > same time thread A has no chance to run because it has lower priority, > thus not being able to release the lock. > (I haven't investigate enough into the source code, so I don't know > how FreeBSD and Linux handle this priority inversion problem. Maybe > they use priority inheritance or random boosting?) > > > FreeBSD's spin locks prevent priority inversion by preventing the holder > thread from being descheduled. > > MTX_DEF locks implement priority inheritance. Nice hints. Thanks! regards, Yubin Ruan