From owner-freebsd-arch@FreeBSD.ORG Thu Sep 12 20:00:58 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 728304CD; Thu, 12 Sep 2013 20:00:58 +0000 (UTC) (envelope-from dkandula@gmail.com) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CE0BC2359; Thu, 12 Sep 2013 20:00:57 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id t60so289376wes.14 for ; Thu, 12 Sep 2013 13:00:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=fze8uhHHRjiK8gKCJbK9e9OfJloWld0HbM9zidUaF/Q=; b=c+MCuQZu1Xjh4pAwrNfEVGLW+h0XcrKS6Gzql/CQ5uFTCYAmJkyMwObPjsuVpCV4rq GvU/D7VzU17ElGhCdC6DuHA5lCNYOo9UU6EP/E9YN50t9dZYTHNwsY8jNGuQpCdH0Ui+ 6+ay22MSazZSS762xrgHNU2xVD1NGV5/U78/ojew5Td4iJleKmfdf9mdR8HpTy9Of2Ab 9mRVZI9MqxEx0FdDdnSJ2ayWAW0hBVen2EgIQ/y+BzOEzZj1Rfir+F4wBPO4RCUEQW/U FgUZabrPzBqxcu7iJ3EPG++qm0y78fWsf+WiqzRyg5VFrCKK4hbJsUegFjtfRkuofo5s ppQQ== MIME-Version: 1.0 X-Received: by 10.194.109.35 with SMTP id hp3mr3249896wjb.55.1379016056246; Thu, 12 Sep 2013 13:00:56 -0700 (PDT) Received: by 10.194.38.167 with HTTP; Thu, 12 Sep 2013 13:00:56 -0700 (PDT) In-Reply-To: References: <201309120824.52916.jhb@freebsd.org> Date: Thu, 12 Sep 2013 16:00:56 -0400 Message-ID: Subject: Re: Why do we need to acquire the current thread's lock before context switching? From: Dheeraj Kandula To: John Baldwin Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Alfred Perlstein , Svatopluk Kraus , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2013 20:00:58 -0000 Hey John, I think I get it now clearly. The td_lock of each thread actually points to the Thread Queue's lock on which it is present. i.e. run queue which may either be the real time runq, timeshare runq or the idle runq. For sleep the td_lock points to the blocked_lock which is a global lock protecting the sleep queue I think. Before cpu_switch() is invoked, the old thread's td_lock is released as shown below: the code is from sched_switch of sched_ule.c lock_profile_release_lock (&TDQ_LOCKPTR (tdq)->lock_object ); TDQ_LOCKPTR (tdq)->mtx_lock =3D (uintptr_t )newtd ; Later after cpu_switch is done, lock_profile_obtain_lock_success (&TDQ_LOCKPTR (tdq)->lock_object , 0, 0, __FILE__ , __LINE__ ); is executed which locks the lock of the thread queue on the current CPU which can be on a different CPU. I assume the new thread's td_lock points to the current CPU's thread queue. Now it is clear that the mutex is unlocked by the same thread that locks it= . Hope my understanding is correct. Dheeraj On Thu, Sep 12, 2013 at 2:00 PM, Dheeraj Kandula wrote= : > Thanks John for the detailed clarification. Wow that is lot of > information. I will digest it and will email you any further questions th= at > I may have. > > Dheeraj > > > On Thu, Sep 12, 2013 at 8:24 AM, John Baldwin wrote: > >> On Thursday, September 12, 2013 7:16:20 am Dheeraj Kandula wrote: >> > Thanks a lot Svatopluk for the clarification. Right after I replied to >> > Alfred's mail, I realized that it can't be thread specific lock as it >> > should also protect the scheduler variables. So if I understand it >> right, >> > even though it is a mutex, it can be unlocked by another thread which = is >> > usually not the case with regular mutexes as the thread that locks it >> must >> > unlock it unlike a binary semaphore. Isn't it? >> >> It's less complicated than that. :) It is a mutex, but to expand on wha= t >> Svatopluk said with an example: take a thread that is asleep on a sleep >> queue. td_lock points to the relevant SC_LOCK() for the sleep queue cha= in >> in that case, so any other thread that wants to examine that thread's >> state ends up locking the sleep queue while it examines that thread. In >> particular, the thread that is doing a wakeup() can resume all of the >> sleeping threads for a wait channel by holding the one SC_LOCK() for tha= t >> wait channel since that will be td_lock for all those threads. >> >> In general mutexes are only unlocked by the thread that locks them, >> and the td_lock of the old thread is unlocked during sched_switch(). >> However, the old thread has to grab td_lock of the new thread during >> sched_switch() and then hand it off to the new thread when it resumes. >> This is why sched_throw() and sched_switch() in ULE directly assign >> 'mtx_lock' of the run queue lock before calling cpu_throw() or >> cpu_switch(). That gives the effect that the new thread resumes while >> holding the lock pinted to by its td_lock. >> >> > Dheeraj >> > >> > >> > On Thu, Sep 12, 2013 at 7:04 AM, Svatopluk Kraus >> wrote: >> > >> > > Think about td_lock like something what is lent by current thread >> owner. >> > > If a thread is running, it's owned by scheduler and td_lock points >> > > to scheduler lock. If a thread is sleeping, it's owned by sleeping >> queue >> > > and td_lock points to sleep queue lock. If a thread is contested, it= 's >> > > owned by turnstile queue and td_lock points to turnstile queue lock. >> And so >> > > on. This way an owner can work with owned threads safely without gia= nt >> > > lock. The td_lock pointer is changed atomically, so it's safe. >> > > >> > > Svatopluk Kraus >> > > >> > > On Thu, Sep 12, 2013 at 12:48 PM, Dheeraj Kandula > >wrote: >> > > >> > >> Thanks a lot Alfred for the clarification. So is the td_lock >> granular i.e. >> > >> one separate lock for each thread but also used for protecting the >> > >> scheduler variables or is it just one lock used by all threads and >> the >> > >> scheduler as well. I will anyway go through the code that you >> suggested >> > >> but >> > >> just wanted to have a deeper understanding before I go about huntin= g >> in >> > >> the >> > >> code. >> > >> >> > >> Dheeraj >> > >> >> > >> >> > >> On Thu, Sep 12, 2013 at 3:10 AM, Alfred Perlstein >> wrote: >> > >> >> > >> > On 9/11/13 2:39 PM, Dheeraj Kandula wrote: >> > >> > >> > >> >> Hey All, >> > >> >> >> > >> >> When the current thread is being context switched with a newly >> selected >> > >> >> thread, why is the current thread's lock acquired before context >> > >> switch =96 >> > >> >> mi_switch() is invoked after thread_lock(td) is called. A thread >> at any >> > >> >> time runs only on one of the cores of a CPU. Hence when it is >> being >> > >> >> context >> > >> >> switched it is added either to the real time runq or the timesha= re >> > >> runq or >> > >> >> the idle runq with the lock still held or it is added to the sle= ep >> > >> queue >> > >> >> or >> > >> >> the blocked queue. So this happens atomically even without the >> lock. >> > >> Isn't >> > >> >> it? Am I missing something here? I don't see any contention for >> the >> > >> thread >> > >> >> in order to demand a lock for the thread which will basically >> protect >> > >> the >> > >> >> contents of the thread structure for the thread. >> > >> >> >> > >> >> Dheeraj >> > >> >> >> > >> >> >> > >> > The thread lock also happens to protect various scheduler >> variables: >> > >> > >> > >> > struct mtx *volatile td_lock; /* replaces sched lock >> */ >> > >> > >> > >> > see sys/kern/sched_ule.c on how the thread lock td_lock is change= d >> > >> > depending on what the thread is doing. >> > >> > >> > >> > -- >> > >> > Alfred Perlstein >> > >> > >> > >> > >> > >> _______________________________________________ >> > >> freebsd-arch@freebsd.org mailing list >> > >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> > >> To unsubscribe, send any mail to " >> freebsd-arch-unsubscribe@freebsd.org" >> > >> >> > > >> > > >> > _______________________________________________ >> > freebsd-arch@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org= " >> > >> >> -- >> John Baldwin >> > >