From owner-svn-src-projects@freebsd.org Wed Aug 10 18:53:12 2016 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25DA8BB5D5B for ; Wed, 10 Aug 2016 18:53:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CCCAA1CB6; Wed, 10 Aug 2016 18:53:11 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u7AIrBLE094245; Wed, 10 Aug 2016 18:53:11 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u7AIrB9X094244; Wed, 10 Aug 2016 18:53:11 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201608101853.u7AIrB9X094244@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 10 Aug 2016 18:53:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r303938 - projects/hps_head/sys/kern X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2016 18:53:12 -0000 Author: hselasky Date: Wed Aug 10 18:53:10 2016 New Revision: 303938 URL: https://svnweb.freebsd.org/changeset/base/303938 Log: Solve a LOR between the callout mutex and the sleepqueue mutex after recent changes. This is solved by making a custom msleep_spin() implementation where the sleepqueue mutex is locked first. Modified: projects/hps_head/sys/kern/kern_timeout.c Modified: projects/hps_head/sys/kern/kern_timeout.c ============================================================================== --- projects/hps_head/sys/kern/kern_timeout.c Wed Aug 10 18:45:26 2016 (r303937) +++ projects/hps_head/sys/kern/kern_timeout.c Wed Aug 10 18:53:10 2016 (r303938) @@ -1266,22 +1266,36 @@ callout_drain(struct callout *c) retval = callout_async_drain(c, &callout_drain_function); if (retval == CALLOUT_RET_DRAINING) { + void *ident = &callout_drain_function; struct callout_cpu *cc; int direct; + int busy; CTR3(KTR_CALLOUT, "need to drain %p func %p arg %p", c, c->c_func, c->c_arg); - cc = callout_lock(c); - direct = ((c->c_flags & CALLOUT_DIRECT) != 0); - - /* Wait for drain to complete */ - while (cc_exec_curr(cc, direct) == c) { - msleep_spin(&callout_drain_function, - (struct mtx *)&cc->cc_lock, "codrain", 0); - } + do { + /* + * The sleepq_lock() is lower rank than the + * callout_lock() and must be locked first: + */ + sleepq_lock(ident); + cc = callout_lock(c); + direct = ((c->c_flags & CALLOUT_DIRECT) != 0); + busy = (cc_exec_curr(cc, direct) == c); + CC_UNLOCK(cc); + DROP_GIANT(); + + if (busy && !SCHEDULER_STOPPED()) { + /* Wait for drain to complete */ + sleepq_add(ident, &cc->cc_lock.lock_object, "codrain", SLEEPQ_SLEEP, 0); + sleepq_wait(ident, 0); + } else { + sleepq_release(ident); + } - CC_UNLOCK(cc); + PICKUP_GIANT(); + } while (busy); } CTR4(KTR_CALLOUT, "%s: %p func %p arg %p",