From owner-freebsd-hackers@freebsd.org Mon Apr 10 01:51:47 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C7B3D35DA0 for ; Mon, 10 Apr 2017 01:51:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x242.google.com (mail-io0-x242.google.com [IPv6:2607:f8b0:4001:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 06683FED for ; Mon, 10 Apr 2017 01:51:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x242.google.com with SMTP id 68so13145607ioh.3 for ; Sun, 09 Apr 2017 18:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=MFIO27u4jXiPmjPOhFGd/MFX/n4NDaWQVAaq0tNGpmw=; b=eX/lkFirGcuoOEUaDvNFUCN3nP38/78LypHmxs66SbLaoIkhr1pZTlE7ATDFKsG7E6 xujQH4uFK3cVxwkPx7SswQDFxde86dkDNwEgLeoYcyiPSpcH8quJr2I6tzU/kluyfNPj lp8UOXS+8ULo6HkgQWfZggXh97eDuWTzvHkDlSoXqs8NAJQ3y+4aVf9yIflNfvcAG813 ig+zcU1t93J2P4YMnE+jGLuHPvXxk970k8SbPTJXbtdTHEWZOSGr3dYfklT8w7aYnm9W F48XXhk9+k2FfjtxCWOVw2jy5fVmDC5ANiZsXcF3qJAAALQmtlwP3mPQceZwGGGh6x5r xQtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=MFIO27u4jXiPmjPOhFGd/MFX/n4NDaWQVAaq0tNGpmw=; b=XpjKy2TdhZ4uhv4+bQ6n9M0nUgLRlBkerE75tQhyWZdTutzYM4WkYIb6XeUJobQKqW fXZIZfnlrXm010Ms3aAu7qOGNZKW8j8poB9CAAvhgtuLa/+w9nS7u2JuvktBHyaUNqmV o72x2mzPYmPaVYMzievUnsGxbib8v6X2bU6tQLDMMJ0/wcqFFw9DA0Sl64NPLFV1rjv4 RFhwK+taekhfq7Czxb8c1GvMpBv53ANOpmry/Y5ymLLSTyibh3ThJ3HxqbPRgrAmRo2T RT92i+79h/xPvGcIpccoJyZ/R8BVZbrUVCfG+80UvMJV0o9DYe/XzJhVDCny9f3minDz wt9w== X-Gm-Message-State: AN3rC/7sJAYQACOCsE+rer4KUETvOQOWDHTB7tHjSbHXGYBJFXeMyVAWCqhq+o8mitMWpWTksvOPdUChRrN6aA== X-Received: by 10.107.134.76 with SMTP id i73mr5787258iod.0.1491789105993; Sun, 09 Apr 2017 18:51:45 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Sun, 9 Apr 2017 18:51:45 -0700 (PDT) X-Originating-IP: [2607:fb10:7021:1::b517] In-Reply-To: <3f93930c-7f10-4d0b-35f2-2b07d64081f0@gmail.com> References: <3f93930c-7f10-4d0b-35f2-2b07d64081f0@gmail.com> From: Warner Losh Date: Sun, 9 Apr 2017 19:51:45 -0600 X-Google-Sender-Auth: UCHBvUdt3vKI0KRKbrTdZaRoxUE Message-ID: Subject: Re: Understanding the FreeBSD locking mechanism To: Yubin Ruan Cc: Ryan Stone , "freebsd-hackers@freebsd.org" , Ed Schouten Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2017 01:51:47 -0000 On Sun, Apr 9, 2017 at 7:28 PM, Yubin Ruan wrote: > On 2017/4/10 0:24, Ryan Stone wrote: >> >> >> >> On Sun, Apr 9, 2017 at 6:13 AM, Yubin Ruan > > wrote: >> >> >> #######1, spinlock used in an interrupt handler >> If a thread A holding a spinlock T get interrupted and the interrupt >> handler responsible for this interrupt try to acquire T, then we have >> deadlock, because A would never have a chance to run before the >> interrupt handler return, and the interrupt handler, unfortunately, >> will continue to spin ... so in this situation, one has to disable >> interrupt before spinning. >> >> As far as I know, in Linux, they provide two kinds of spinlocks: >> >> spin_lock(..); /* spinlock that does not disable interrupts */ >> spin_lock_irqsave(...); /* spinlock that disable local interrupt * >> >> >> In the FreeBSD locking style, a spinlock is only used in the case where >> one needs to synchronize with an interrupt handler. This is why spinlocks >> always disable local interrupts in FreeBSD. >> >> FreeBSD's lock for the first case is the MTX_DEF mutex, which is >> adaptively-spinning blocking mutex implementation. In short, the MTX_DEF >> mutex will spin waiting for the lock if the owner is running, but will >> block if the owner is deschedules. This prevents expensive trips through >> the scheduler for the common case where the mutex is only held for short >> periods, without wasting CPU cycles spinning in cases where the owner >> thread >> is descheduled and therefore will not be completing soon. > > > Great explanation! I read the man page at: > >> >> https://www.freebsd.org/cgi/man.cgi?query=mutex&sektion=9&apropos=0&manpath=FreeBSD+11.0-RELEASE+and+Ports > > and now clear about MTX_DEF and MTX_SPIN mutexs. But, still a few more > question, if you don't mind: > > Is it true that a thread holding a MTX_DEF mutex can be descheduled? > (shouldn't it disable interrupt like a MTX_SPIN mutex?) It is said on > the main page that MTX_DEF mutex is used by default in FreeBSD, so its > usecase must be very common. If a thread holding a MTX_DEF mutex can be > descheduled, which means that it did not disable interrupt, then we may > have lots of deadlock here, right? Yes, they can be descheduled. But that's not a problem. No other thread can acquire the MTX_DEF lock. If another thread tries, it will sleep and wait for the thread that holds the MTX_DEF lock to release it. Eventually, the thread will get time to run again, and then release the lock. Threads that just hold a MTX_DEF lock may also migrate from CPU to CPU too. Warner >> #######2, priority inversion problem >> If thread B with a higher priority get in and try to acquire the lock >> that thread A currently holds, then thread B would spin, while at the >> same time thread A has no chance to run because it has lower priority, >> thus not being able to release the lock. >> (I haven't investigate enough into the source code, so I don't know >> how FreeBSD and Linux handle this priority inversion problem. Maybe >> they use priority inheritance or random boosting?) >> >> >> FreeBSD's spin locks prevent priority inversion by preventing the holder >> thread from being descheduled. >> >> MTX_DEF locks implement priority inheritance. > > > Nice hints. Thanks! > > regards, > Yubin Ruan > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"