From owner-freebsd-arch@FreeBSD.ORG Thu Sep 12 15:59:57 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4FB654AC for ; Thu, 12 Sep 2013 15:59:57 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 06D712CA9 for ; Thu, 12 Sep 2013 15:59:57 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CFF68B939; Thu, 12 Sep 2013 11:59:55 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Why do we need to acquire the current thread's lock before context switching? Date: Thu, 12 Sep 2013 08:24:52 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Message-Id: <201309120824.52916.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 12 Sep 2013 11:59:55 -0400 (EDT) Cc: Alfred Perlstein , Svatopluk Kraus , Dheeraj Kandula X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2013 15:59:57 -0000 On Thursday, September 12, 2013 7:16:20 am Dheeraj Kandula wrote: > Thanks a lot Svatopluk for the clarification. Right after I replied to > Alfred's mail, I realized that it can't be thread specific lock as it > should also protect the scheduler variables. So if I understand it right, > even though it is a mutex, it can be unlocked by another thread which is > usually not the case with regular mutexes as the thread that locks it must > unlock it unlike a binary semaphore. Isn't it? It's less complicated than that. :) It is a mutex, but to expand on what Svatopluk said with an example: take a thread that is asleep on a sleep queue. td_lock points to the relevant SC_LOCK() for the sleep queue chain in that case, so any other thread that wants to examine that thread's state ends up locking the sleep queue while it examines that thread. In particular, the thread that is doing a wakeup() can resume all of the sleeping threads for a wait channel by holding the one SC_LOCK() for that wait channel since that will be td_lock for all those threads. In general mutexes are only unlocked by the thread that locks them, and the td_lock of the old thread is unlocked during sched_switch(). However, the old thread has to grab td_lock of the new thread during sched_switch() and then hand it off to the new thread when it resumes. This is why sched_throw() and sched_switch() in ULE directly assign 'mtx_lock' of the run queue lock before calling cpu_throw() or cpu_switch(). That gives the effect that the new thread resumes while holding the lock pinted to by its td_lock. > Dheeraj >=20 >=20 > On Thu, Sep 12, 2013 at 7:04 AM, Svatopluk Kraus wrote: >=20 > > Think about td_lock like something what is lent by current thread owner. > > If a thread is running, it's owned by scheduler and td_lock points > > to scheduler lock. If a thread is sleeping, it's owned by sleeping queue > > and td_lock points to sleep queue lock. If a thread is contested, it's > > owned by turnstile queue and td_lock points to turnstile queue lock. An= d so > > on. This way an owner can work with owned threads safely without giant > > lock. The td_lock pointer is changed atomically, so it's safe. > > > > Svatopluk Kraus > > > > On Thu, Sep 12, 2013 at 12:48 PM, Dheeraj Kandula w= rote: > > > >> Thanks a lot Alfred for the clarification. So is the td_lock granular = i.e. > >> one separate lock for each thread but also used for protecting the > >> scheduler variables or is it just one lock used by all threads and the > >> scheduler as well. I will anyway go through the code that you suggested > >> but > >> just wanted to have a deeper understanding before I go about hunting in > >> the > >> code. > >> > >> Dheeraj > >> > >> > >> On Thu, Sep 12, 2013 at 3:10 AM, Alfred Perlstein wrot= e: > >> > >> > On 9/11/13 2:39 PM, Dheeraj Kandula wrote: > >> > > >> >> Hey All, > >> >> > >> >> When the current thread is being context switched with a newly sele= cted > >> >> thread, why is the current thread's lock acquired before context > >> switch =96 > >> >> mi_switch() is invoked after thread_lock(td) is called. A thread at= any > >> >> time runs only on one of the cores of a CPU. Hence when it is being > >> >> context > >> >> switched it is added either to the real time runq or the timeshare > >> runq or > >> >> the idle runq with the lock still held or it is added to the sleep > >> queue > >> >> or > >> >> the blocked queue. So this happens atomically even without the lock. > >> Isn't > >> >> it? Am I missing something here? I don't see any contention for the > >> thread > >> >> in order to demand a lock for the thread which will basically prote= ct > >> the > >> >> contents of the thread structure for the thread. > >> >> > >> >> Dheeraj > >> >> > >> >> > >> > The thread lock also happens to protect various scheduler variables: > >> > > >> > struct mtx *volatile td_lock; /* replaces sched lock */ > >> > > >> > see sys/kern/sched_ule.c on how the thread lock td_lock is changed > >> > depending on what the thread is doing. > >> > > >> > -- > >> > Alfred Perlstein > >> > > >> > > >> _______________________________________________ > >> freebsd-arch@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch > >> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > >> > > > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >=20 =2D-=20 John Baldwin