Date: Mon, 14 Sep 2015 15:18:29 +0300 From: Dmitry Sivachenko <trtrmitya@gmail.com> To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: FreeBSD Stable ML <stable@freebsd.org> Subject: Re: process scheduling and cpuset Message-ID: <6FCB48B4-02BD-4048-A3A5-3F86A25F5386@gmail.com> In-Reply-To: <20150913164057.GT21849@zxy.spb.ru> References: <623FA99E-04E7-4D29-953A-61EE7B35CBF6@gmail.com> <20150913130920.GR3158@zxy.spb.ru> <C6ADA4B7-1126-44A5-94B3-97FA79C8582A@gmail.com> <20150913164057.GT21849@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 13 =D3=C5=CE=D4. 2015 =C7., at 19:40, Slawa Olhovchenkov = <slw@zxy.spb.ru> wrote: >=20 > On Sun, Sep 13, 2015 at 04:44:40PM +0300, Dmitry Sivachenko wrote: >=20 >>=20 >>> On 13 =D3=C5=CE=D4. 2015 =C7., at 16:09, Slawa Olhovchenkov = <slw@zxy.spb.ru> wrote: >>>=20 >>> On Sun, Sep 13, 2015 at 02:52:08PM +0300, Dmitry Sivachenko wrote: >>>=20 >>>> Hello, >>>>=20 >>>> I have 32 processor machine (2x CPU E5-2650) running several = CPU-bound processes (ULE scheduler). >>>> 3 processes are 32-threaded, and 8 are single threaded. >>>>=20 >>>> I bind all 3 32-threaded processes to CPUs 0-24 (cpuset -C -l 0-24 = -p XXX). >>>>=20 >>>> I expect that the remaining 8 single-threaded processes will = (mostly) run on the remaining 25-31 CPU cores and use (almost) 100% cpu = each. >>>>=20 >>>> But this is not the case (according to top(1)): they spend a lot = of time on 0-24 CPUs and CPU Idle time is about 10%. >>>>=20 >>>> These are all purely computational programs, in idle system = single-threaded programs steadily consume 100% of a core, and = 32-threaded programs consume all 32 cores and idle time is zero. >>>>=20 >>>> Is it an ULE scheduler feature or am I doing something wrong? >>>>=20 >>>> The goal is to give a single-threaded program a chance to run when = somebody started several 32-threaded processes. >>>=20 >>> You don't have 32 processor machine, you have only 16 processor >>> machine. >>> SMT/hyperthreading don't give real processor, SMT "CPU" have >>> unpredicable power and his load depend on load parent CPU. >>>=20 >>> For example, for my case I see such condition (simpliy) on CPU 0 and = 1 >>> (SMT of one real core) with rise load: >>>=20 >>> load 0.1 0.1 >>> load 0.2 0.2 >>> load 0.3 0.3 >>> load 0.4 0.4 >>> load 0.45 0.45 >>> load 0.48 0.48 >>> load 1.00 1.00\ >>=20 >>=20 >> Yes I know about HT. But how does this explain why I have 10% of CPU = idle? >>=20 >> If I explicitly bind my single-threaded processes to the remaining = CPU cores (25-32), they start to receive expected 100% of CPU and = overall Idle decreases. >>=20 >> I just expect scheduler to do the same for me. >>=20 >=20 > Idle is not goal, goal is lessing task executing time. Thanks for the explanation. In my example SMT pairs are numbered with sequential numbers, so 0+1 is = one SMT group, 2+3 is second SMT group, and so on. So in 25-32 range there are several real CPU cores which remain idle = while processes are fighting for overloaded 0-24. When I explicitly pin my single-threaded processes to 25-32 range, they = start to receive 100% of CPU (and finish faster to be clear).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6FCB48B4-02BD-4048-A3A5-3F86A25F5386>