From owner-freebsd-current@FreeBSD.ORG Thu Sep 15 14:33:29 2011 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E0171065670 for ; Thu, 15 Sep 2011 14:33:29 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AFB708FC15 for ; Thu, 15 Sep 2011 14:33:28 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA14478; Thu, 15 Sep 2011 17:33:26 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E720CB6.3070103@FreeBSD.org> Date: Thu, 15 Sep 2011 17:33:26 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-current@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: couple of sched_ule issues X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Sep 2011 14:33:29 -0000 This is more of a "just for the record" email. I think I've already stated the following observations, but I suspect that they drowned in the noise of a thread in which I mentioned them. 1. Incorrect topology is built for single-package SMP systems. That topology has two levels ("shared nothing" and "shared package") with exactly the same CPU sets. That doesn't work well with the rebalancing algorithm which assumes that each level is a proper/strict subset of its parent. 2. CPU load comparison algorithms are biased towards lower logical CPU IDs. With all other things being equal the algorithms will always pick a CPU with a lower ID. This creates certain load asymmetry and predictable patterns in load distribution. Another observation. It seems that ULE makes a decision about thread-to-CPU affinity at the time when a thread gets switched out. This looks logical from the implementation point of view. But it doesn't seem logical from a general point of view - when the thread will be becoming running again its affinity profile may become completely different. I think that it would depend on how much a thread actually spends not running. -- Andriy Gapon