From owner-freebsd-arch@FreeBSD.ORG Thu Sep 12 20:21:39 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 93A28C44; Thu, 12 Sep 2013 20:21:39 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 7F0BD24B5; Thu, 12 Sep 2013 20:21:39 +0000 (UTC) Received: from [10.239.127.44] (5.sub-174-254-208.myvzw.com [174.254.208.5]) by elvis.mu.org (Postfix) with ESMTPSA id F2F5A1A3E3F; Thu, 12 Sep 2013 13:21:37 -0700 (PDT) References: <201309120824.52916.jhb@freebsd.org> Mime-Version: 1.0 (1.0) In-Reply-To: <201309120824.52916.jhb@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (10B329) From: Alfred Perlstein Subject: Re: Why do we need to acquire the current thread's lock before context switching? Date: Thu, 12 Sep 2013 13:21:35 -0700 To: John Baldwin Cc: Dheeraj Kandula , Svatopluk Kraus , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2013 20:21:39 -0000 Both these explanations are so great. Is there any way we can add this to pr= oc.h or maybe document somewhere and then link to it from proc.h? Sent from my iPhone On Sep 12, 2013, at 5:24 AM, John Baldwin wrote: > On Thursday, September 12, 2013 7:16:20 am Dheeraj Kandula wrote: >> Thanks a lot Svatopluk for the clarification. Right after I replied to >> Alfred's mail, I realized that it can't be thread specific lock as it >> should also protect the scheduler variables. So if I understand it right,= >> even though it is a mutex, it can be unlocked by another thread which is >> usually not the case with regular mutexes as the thread that locks it mus= t >> unlock it unlike a binary semaphore. Isn't it? >=20 > It's less complicated than that. :) It is a mutex, but to expand on what > Svatopluk said with an example: take a thread that is asleep on a sleep > queue. td_lock points to the relevant SC_LOCK() for the sleep queue chain= > in that case, so any other thread that wants to examine that thread's > state ends up locking the sleep queue while it examines that thread. In > particular, the thread that is doing a wakeup() can resume all of the > sleeping threads for a wait channel by holding the one SC_LOCK() for that > wait channel since that will be td_lock for all those threads. >=20 > In general mutexes are only unlocked by the thread that locks them, > and the td_lock of the old thread is unlocked during sched_switch(). > However, the old thread has to grab td_lock of the new thread during > sched_switch() and then hand it off to the new thread when it resumes. > This is why sched_throw() and sched_switch() in ULE directly assign > 'mtx_lock' of the run queue lock before calling cpu_throw() or > cpu_switch(). That gives the effect that the new thread resumes while > holding the lock pinted to by its td_lock. >=20 >> Dheeraj >>=20 >>=20 >> On Thu, Sep 12, 2013 at 7:04 AM, Svatopluk Kraus wrote= : >>=20 >>> Think about td_lock like something what is lent by current thread owner.= >>> If a thread is running, it's owned by scheduler and td_lock points >>> to scheduler lock. If a thread is sleeping, it's owned by sleeping queue= >>> and td_lock points to sleep queue lock. If a thread is contested, it's >>> owned by turnstile queue and td_lock points to turnstile queue lock. And= so >>> on. This way an owner can work with owned threads safely without giant >>> lock. The td_lock pointer is changed atomically, so it's safe. >>>=20 >>> Svatopluk Kraus >>>=20 >>> On Thu, Sep 12, 2013 at 12:48 PM, Dheeraj Kandula wr= ote: >>>=20 >>>> Thanks a lot Alfred for the clarification. So is the td_lock granular i= .e. >>>> one separate lock for each thread but also used for protecting the >>>> scheduler variables or is it just one lock used by all threads and the >>>> scheduler as well. I will anyway go through the code that you suggested= >>>> but >>>> just wanted to have a deeper understanding before I go about hunting in= >>>> the >>>> code. >>>>=20 >>>> Dheeraj >>>>=20 >>>>=20 >>>> On Thu, Sep 12, 2013 at 3:10 AM, Alfred Perlstein wrote= : >>>>=20 >>>>> On 9/11/13 2:39 PM, Dheeraj Kandula wrote: >>>>>=20 >>>>>> Hey All, >>>>>>=20 >>>>>> When the current thread is being context switched with a newly select= ed >>>>>> thread, why is the current thread's lock acquired before context >>>> switch =E2=80=93 >>>>>> mi_switch() is invoked after thread_lock(td) is called. A thread at a= ny >>>>>> time runs only on one of the cores of a CPU. Hence when it is being >>>>>> context >>>>>> switched it is added either to the real time runq or the timeshare >>>> runq or >>>>>> the idle runq with the lock still held or it is added to the sleep >>>> queue >>>>>> or >>>>>> the blocked queue. So this happens atomically even without the lock. >>>> Isn't >>>>>> it? Am I missing something here? I don't see any contention for the >>>> thread >>>>>> in order to demand a lock for the thread which will basically protect= >>>> the >>>>>> contents of the thread structure for the thread. >>>>>>=20 >>>>>> Dheeraj >>>>> The thread lock also happens to protect various scheduler variables: >>>>>=20 >>>>> struct mtx *volatile td_lock; /* replaces sched lock */ >>>>>=20 >>>>> see sys/kern/sched_ule.c on how the thread lock td_lock is changed >>>>> depending on what the thread is doing. >>>>>=20 >>>>> -- >>>>> Alfred Perlstein >>>> _______________________________________________ >>>> freebsd-arch@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >>>> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"= >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >=20 > --=20 > John Baldwin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >=20