From owner-svn-src-head@FreeBSD.ORG  Sat Mar  8 00:35:07 2014
Return-Path: <owner-svn-src-head@FreeBSD.ORG>
Delivered-To: svn-src-head@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 85E41F49;
 Sat,  8 Mar 2014 00:35:07 +0000 (UTC)
Received: from svn.freebsd.org (svn.freebsd.org
 [IPv6:2001:1900:2254:2068::e6a:0])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 735A5217;
 Sat,  8 Mar 2014 00:35:07 +0000 (UTC)
Received: from svn.freebsd.org ([127.0.1.70])
 by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s280Z7F9036150;
 Sat, 8 Mar 2014 00:35:07 GMT (envelope-from jeff@svn.freebsd.org)
Received: (from jeff@localhost)
 by svn.freebsd.org (8.14.8/8.14.8/Submit) id s280Z7xF036149;
 Sat, 8 Mar 2014 00:35:07 GMT (envelope-from jeff@svn.freebsd.org)
Message-Id: <201403080035.s280Z7xF036149@svn.freebsd.org>
From: Jeff Roberson <jeff@FreeBSD.org>
Date: Sat, 8 Mar 2014 00:35:07 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
 svn-src-head@freebsd.org
Subject: svn commit: r262917 - head/sys/kern
X-SVN-Group: head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-head@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: SVN commit messages for the src tree for head/-current
 <svn-src-head.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-src-head>,
 <mailto:svn-src-head-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-head/>
List-Post: <mailto:svn-src-head@freebsd.org>
List-Help: <mailto:svn-src-head-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-head>,
 <mailto:svn-src-head-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 08 Mar 2014 00:35:07 -0000

Author: jeff
Date: Sat Mar  8 00:35:06 2014
New Revision: 262917
URL: http://svnweb.freebsd.org/changeset/base/262917

Log:
   - Make runq_steal_from more aggressive.  Previously it would examine only
     a single priority queue.  If that queue had a thread or threads which
     could not be migrated we would fail to steal load.  This could cause
     starvation in situations where cores are idle.
  
  Submitted by:	Doug Kilpatrick <dkilpatrick@isilon.com>
  Tested by:	pho
  Reviewed by:	mav
  Sponsored by:	EMC / Isilon Storage Division

Modified:
  head/sys/kern/sched_ule.c

Modified: head/sys/kern/sched_ule.c
==============================================================================
--- head/sys/kern/sched_ule.c	Sat Mar  8 00:14:40 2014	(r262916)
+++ head/sys/kern/sched_ule.c	Sat Mar  8 00:35:06 2014	(r262917)
@@ -1057,32 +1057,27 @@ runq_steal_from(struct runq *rq, int cpu
 	struct rqhead *rqh;
 	struct thread *td, *first;
 	int bit;
-	int pri;
 	int i;
 
 	rqb = &rq->rq_status;
 	bit = start & (RQB_BPW -1);
-	pri = 0;
 	first = NULL;
 again:
 	for (i = RQB_WORD(start); i < RQB_LEN; bit = 0, i++) {
 		if (rqb->rqb_bits[i] == 0)
 			continue;
-		if (bit != 0) {
-			for (pri = bit; pri < RQB_BPW; pri++)
-				if (rqb->rqb_bits[i] & (1ul << pri))
-					break;
-			if (pri >= RQB_BPW)
+		if (bit == 0)
+			bit = RQB_FFS(rqb->rqb_bits[i]);
+		for (; bit < RQB_BPW; bit++) {
+			if ((rqb->rqb_bits[i] & (1ul << bit)) == 0)
 				continue;
-		} else
-			pri = RQB_FFS(rqb->rqb_bits[i]);
-		pri += (i << RQB_L2BPW);
-		rqh = &rq->rq_queues[pri];
-		TAILQ_FOREACH(td, rqh, td_runq) {
-			if (first && THREAD_CAN_MIGRATE(td) &&
-			    THREAD_CAN_SCHED(td, cpu))
-				return (td);
-			first = td;
+			rqh = &rq->rq_queues[bit + (i << RQB_L2BPW)];
+			TAILQ_FOREACH(td, rqh, td_runq) {
+				if (first && THREAD_CAN_MIGRATE(td) &&
+				    THREAD_CAN_SCHED(td, cpu))
+					return (td);
+				first = td;
+			}
 		}
 	}
 	if (start != 0) {