From owner-freebsd-stable@freebsd.org Sun Sep 13 16:41:02 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DBBBA03CA1 for ; Sun, 13 Sep 2015 16:41:02 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 86E541FBF for ; Sun, 13 Sep 2015 16:41:02 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 85AAFA03CA0; Sun, 13 Sep 2015 16:41:02 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85407A03C9F for ; Sun, 13 Sep 2015 16:41:02 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40F3A1FBC for ; Sun, 13 Sep 2015 16:41:02 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZbAKn-000DkG-Am; Sun, 13 Sep 2015 19:40:57 +0300 Date: Sun, 13 Sep 2015 19:40:57 +0300 From: Slawa Olhovchenkov To: Dmitry Sivachenko Cc: FreeBSD Stable ML Subject: Re: process scheduling and cpuset Message-ID: <20150913164057.GT21849@zxy.spb.ru> References: <623FA99E-04E7-4D29-953A-61EE7B35CBF6@gmail.com> <20150913130920.GR3158@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Sep 2015 16:41:02 -0000 On Sun, Sep 13, 2015 at 04:44:40PM +0300, Dmitry Sivachenko wrote: > > > On 13 сент. 2015 г., at 16:09, Slawa Olhovchenkov wrote: > > > > On Sun, Sep 13, 2015 at 02:52:08PM +0300, Dmitry Sivachenko wrote: > > > >> Hello, > >> > >> I have 32 processor machine (2x CPU E5-2650) running several CPU-bound processes (ULE scheduler). > >> 3 processes are 32-threaded, and 8 are single threaded. > >> > >> I bind all 3 32-threaded processes to CPUs 0-24 (cpuset -C -l 0-24 -p XXX). > >> > >> I expect that the remaining 8 single-threaded processes will (mostly) run on the remaining 25-31 CPU cores and use (almost) 100% cpu each. > >> > >> But this is not the case (according to top(1)): they spend a lot of time on 0-24 CPUs and CPU Idle time is about 10%. > >> > >> These are all purely computational programs, in idle system single-threaded programs steadily consume 100% of a core, and 32-threaded programs consume all 32 cores and idle time is zero. > >> > >> Is it an ULE scheduler feature or am I doing something wrong? > >> > >> The goal is to give a single-threaded program a chance to run when somebody started several 32-threaded processes. > > > > You don't have 32 processor machine, you have only 16 processor > > machine. > > SMT/hyperthreading don't give real processor, SMT "CPU" have > > unpredicable power and his load depend on load parent CPU. > > > > For example, for my case I see such condition (simpliy) on CPU 0 and 1 > > (SMT of one real core) with rise load: > > > > load 0.1 0.1 > > load 0.2 0.2 > > load 0.3 0.3 > > load 0.4 0.4 > > load 0.45 0.45 > > load 0.48 0.48 > > load 1.00 1.00\ > > > Yes I know about HT. But how does this explain why I have 10% of CPU idle? > > If I explicitly bind my single-threaded processes to the remaining CPU cores (25-32), they start to receive expected 100% of CPU and overall Idle decreases. > > I just expect scheduler to do the same for me. > Idle is not goal, goal is lessing task executing time.