From owner-freebsd-arch@FreeBSD.ORG Wed Sep 29 22:14:37 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5118C16A4CE for ; Wed, 29 Sep 2004 22:14:37 +0000 (GMT) Received: from duchess.speedfactory.net (duchess.speedfactory.net [66.23.201.84]) by mx1.FreeBSD.org (Postfix) with SMTP id A96AC43D2F for ; Wed, 29 Sep 2004 22:14:36 +0000 (GMT) (envelope-from ups@tree.com) Received: (qmail 22904 invoked by uid 89); 29 Sep 2004 22:14:34 -0000 Received: from duchess.speedfactory.net (66.23.201.84) by duchess.speedfactory.net with SMTP; 29 Sep 2004 22:14:34 -0000 Received: (qmail 22231 invoked by uid 89); 29 Sep 2004 22:14:19 -0000 Received: from unknown (HELO palm.tree.com) (66.23.216.49) by duchess.speedfactory.net with SMTP; 29 Sep 2004 22:14:19 -0000 Received: from [127.0.0.1] (localhost.tree.com [127.0.0.1]) by palm.tree.com (8.12.10/8.12.10) with ESMTP id i8TMEHmt017471; Wed, 29 Sep 2004 18:14:18 -0400 (EDT) (envelope-from ups@tree.com) From: Stephan Uphoff To: John Baldwin In-Reply-To: <200409291652.29990.jhb@FreeBSD.org> References: <1095468747.31297.241.camel@palm.tree.com> <1096477932.3733.1471.camel@palm.tree.com> <1096489576.3733.1868.camel@palm.tree.com> <200409291652.29990.jhb@FreeBSD.org> Content-Type: text/plain Message-Id: <1096496057.3733.2163.camel@palm.tree.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Wed, 29 Sep 2004 18:14:17 -0400 Content-Transfer-Encoding: 7bit cc: Peter Holm cc: Julian Elischer cc: "freebsd-arch@freebsd.org" Subject: Re: scheduler (sched_4bsd) questions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2004 22:14:37 -0000 On Wed, 2004-09-29 at 16:52, John Baldwin wrote: > > > OK - here is a crude patch to fix some problems with mutex priority > > > inheritance. My theory is that the clock thread gets stuck waiting on > > > GIANT. > > > > > > During release/acquisition of a contested sleep mutex there are a few > > > windows where a task can be preempted when actions (waking up blocked > > > threads, ownership of the mutex, ..) need to be atomic as far as > > > scheduling is concerned. Otherwise priority inheritance may fail. The > > > patch uses critical_enter/critical_exit to protect these regions against > > > preemption. > > > > > > It would be great if could run this in addition to the other patches. > > turnstile_claim() doesn't make any threads runnable and thus can't preempt. > The other place is supposed to preempt, and it should be ok to do so. Note > that since the turnstile chain lock is held, that includes a nested critical > section and any preemption will be deferred until the turnstile lock is > released via turnstile_release which happens in the middle of > turnstile_unpend() after it has finished building a list of all the threads > to be made runnable so that the turnstile object can be re-used safely. I > don't think this patch will make much of a difference (if any). Can you > provide a description of a case where you think the priority inheritance can > fail if turnstile_unpend() doesn't run in a nested critical section? This is a bit of a mind bender. I hope you have some aspirins close by ;-) Thread A holds a mutex x contested by Thread B and has priority pri(A). Thread B holds a mutex y. There is a thread C with priority pri(C) with pri(C) < pri(A). Thread A is in the process of releasing x. It removes thread B from the turnstile and holds a pointer to B in a private list. Thread A sets the owner of the turnstile to NULL and releases all spin locks. ( mtx_unlock_spin(&tc->tc_lock); line 148) This means interrupts are now enabled. An interrupt occurs (or is already pending) and the interrupt handler puts the associated interrupt thread I on the run queue. This causes a preemption from A to I. The interrupt thread I tries to acquire mutex y owned by B and blocks. I donates its priority to B - but inheritance stops at B. The next thread with the best priority is C and the cpu switches to C. However B needs A to run to make it to the run-queue. If y is GIANT and I is the clock thread C could run forever in userspace without being interrupted. There is another scenario that does not require an interrupt (preemption in setrunqueue(td, SRQ_BORING), two blocked threads ...). I was looking at the MUTEX_WAKE_ALL undefined case when I used the critical section for turnstile_claim(). However there are bigger problems with MUTEX_WAKE_ALL undefined so you are right - the critical section for turnstile_claim is pretty useless. Stephan