From owner-freebsd-current@FreeBSD.ORG Thu Feb 26 15:26:00 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2EA05106566C for ; Thu, 26 Feb 2009 15:26:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id F2B108FC08 for ; Thu, 26 Feb 2009 15:25:59 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id 82F6346B23; Thu, 26 Feb 2009 10:25:59 -0500 (EST) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n1QFPOR2042853; Thu, 26 Feb 2009 10:25:53 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-current@freebsd.org Date: Thu, 26 Feb 2009 10:24:37 -0500 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200902261024.37346.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Thu, 26 Feb 2009 10:25:53 -0500 (EST) X-Virus-Scanned: ClamAV 0.94.2/9051/Thu Feb 26 08:08:01 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Justin Teller Subject: Re: Possible case of sched_ule never honoring cpu_set affinity? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2009 15:26:00 -0000 On Tuesday 24 February 2009 5:46:11 pm Justin Teller wrote: > I believe I've found a case where sched_ule can never honor cpu_set > affinity, and I've attached a suggestion to fix it -- if I'm way off > on how this should be implemented, please let me know! > > Imagine the case where sched_affinity is called and the thread is on a > RUNQ on the wrong CPU -- sched_affinity simply exits, relying on > something else in the scheduler to migrate the thread if need be. The > next time the thread is chosen to run, it runs on the wrong CPU. > Worse, if the thread never goes to sleep or is chosen to be moved by > the load balancer, the thread will continue running on the wrong CPU > indefinitely. > > Attached is a suggestion of how to change the scheduler to honor > affinity in this case. With the attached diff, sched_ule will allow a > thread to run on the wrong CPU for one slice. Then when the thread > moves through sched_switch, if it was running on the wrong CPU, it > will be migrated to the right CPU. I've written a test where one > thread will bind another thread to a particular CPU (using cpusets) > then waits until it's running on the right CPU before binding it to a > different CPU (and continuing ad nauseam). Without the change, the > test will sometimes hang waiting for the second thread to get on the > correct CPU -- with the change, it works every time. I think your patch is correct. -- John Baldwin