From owner-freebsd-current@FreeBSD.ORG Tue Nov 8 20:47:20 2011 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B70C1065771 for ; Tue, 8 Nov 2011 20:47:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 93CB98FC1A for ; Tue, 8 Nov 2011 20:47:19 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA28181; Tue, 08 Nov 2011 22:47:17 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RNsZg-0008Lm-Os; Tue, 08 Nov 2011 22:47:16 +0200 Message-ID: <4EB99553.5010005@FreeBSD.org> Date: Tue, 08 Nov 2011 22:47:15 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111002 Thunderbird/7.0.1 MIME-Version: 1.0 To: Jeff Roberson References: <4E720CB6.3070103@FreeBSD.org> In-Reply-To: X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org Subject: Re: couple of sched_ule issues X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2011 20:47:20 -0000 on 03/11/2011 22:17 Jeff Roberson said the following: > On Thu, 15 Sep 2011, Andriy Gapon wrote: > >> >> This is more of a "just for the record" email. >> I think I've already stated the following observations, but I suspect that they >> drowned in the noise of a thread in which I mentioned them. >> >> 1. Incorrect topology is built for single-package SMP systems. >> That topology has two levels ("shared nothing" and "shared package") with exactly >> the same CPU sets. That doesn't work well with the rebalancing algorithm which >> assumes that each level is a proper/strict subset of its parent. >> >> 2. CPU load comparison algorithms are biased towards lower logical CPU IDs. >> With all other things being equal the algorithms will always pick a CPU with a >> lower ID. This creates certain load asymmetry and predictable patterns in load >> distribution. > > If all other things truly are equal why does selecting a lower cpu number matter? Honestly, I am not sure. This bias could definitely produce an "inequality" from the point of view of how many threads each CPU gets. But I can not say if there could be any "inequality" in how much CPU time each thread gets. But I don't rule out that that might be possible... Please also see this post of mine: http://thread.gmane.org/gmane.os.freebsd.current/133763/focus=134192 >> Another observation. >> It seems that ULE makes a decision about thread-to-CPU affinity at the time >> when a >> thread gets switched out. This looks logical from the implementation point of >> view. But it doesn't seem logical from a general point of view - when the thread >> will be becoming running again its affinity profile may become completely >> different. I think that it would depend on how much a thread actually spends not >> running. > > The decision is made at sched_add() time. sched_pickcpu() does the work and > selects the run-queue we will be added to. We consider the CPU that the thread > was last running on but the decision is made at the time that a run queue must > be selected. Ah, yes. But, OTOH, it doesn't look like sched_add would be called for a CPU-bound thread (no voluntary sleeps) ? -- Andriy Gapon