From owner-freebsd-stable@freebsd.org Sun Sep 13 13:44:46 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B986A0218C for ; Sun, 13 Sep 2015 13:44:46 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2AC951646 for ; Sun, 13 Sep 2015 13:44:46 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 27E0EA02187; Sun, 13 Sep 2015 13:44:46 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D7A2A02186 for ; Sun, 13 Sep 2015 13:44:46 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 893AA1645 for ; Sun, 13 Sep 2015 13:44:45 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: by lamp12 with SMTP id p12so71727780lam.0 for ; Sun, 13 Sep 2015 06:44:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=CkuPP9Ekj9/tHeV12jqYgYTrziQjjpCPOz6u1tjFrw0=; b=ljklRq/3Jbrn0Tfr68Yl4Mwy1gA1g2V8xoXrRo7MdiBDbdD6KoI5dmGe4Bjd8sxRPF VpkC2aYdq7G88NiFFADgTb8181/H3uMQr7FtFCODQEYlEuCFLtMrHa2qWEjQH1ePfQ/y m1Usni9kHMIf62TNzr+QMTpyPkUEY8fQqjXS4YTboRcUUmtxKoygdzIGdowzaaTlfF5H 7uROWhMeF6ruupUzpROS8xl3d5/oXg9F3/orfqgFH1D2pymQ3r0CXJORmUYBvJSTuHxA ceuae3fRhkdysNpxRRQ1jOOIzhFRoqoMWjGb1T+2EGwIP+DGqZJ3QVSPIUkG4jaGcZ0W vRPQ== X-Received: by 10.152.170.196 with SMTP id ao4mr8669360lac.101.1442151883519; Sun, 13 Sep 2015 06:44:43 -0700 (PDT) Received: from [172.20.10.4] ([213.87.146.160]) by smtp.gmail.com with ESMTPSA id j6sm1527266laf.39.2015.09.13.06.44.42 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 13 Sep 2015 06:44:42 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: process scheduling and cpuset From: Dmitry Sivachenko In-Reply-To: <20150913130920.GR3158@zxy.spb.ru> Date: Sun, 13 Sep 2015 16:44:40 +0300 Cc: FreeBSD Stable ML Content-Transfer-Encoding: quoted-printable Message-Id: References: <623FA99E-04E7-4D29-953A-61EE7B35CBF6@gmail.com> <20150913130920.GR3158@zxy.spb.ru> To: Slawa Olhovchenkov X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Sep 2015 13:44:46 -0000 > On 13 =D1=81=D0=B5=D0=BD=D1=82. 2015 =D0=B3., at 16:09, Slawa = Olhovchenkov wrote: >=20 > On Sun, Sep 13, 2015 at 02:52:08PM +0300, Dmitry Sivachenko wrote: >=20 >> Hello, >>=20 >> I have 32 processor machine (2x CPU E5-2650) running several = CPU-bound processes (ULE scheduler). >> 3 processes are 32-threaded, and 8 are single threaded. >>=20 >> I bind all 3 32-threaded processes to CPUs 0-24 (cpuset -C -l 0-24 -p = XXX). >>=20 >> I expect that the remaining 8 single-threaded processes will (mostly) = run on the remaining 25-31 CPU cores and use (almost) 100% cpu each. >>=20 >> But this is not the case (according to top(1)): they spend a lot of = time on 0-24 CPUs and CPU Idle time is about 10%. >>=20 >> These are all purely computational programs, in idle system = single-threaded programs steadily consume 100% of a core, and = 32-threaded programs consume all 32 cores and idle time is zero. >>=20 >> Is it an ULE scheduler feature or am I doing something wrong? >>=20 >> The goal is to give a single-threaded program a chance to run when = somebody started several 32-threaded processes. >=20 > You don't have 32 processor machine, you have only 16 processor > machine. > SMT/hyperthreading don't give real processor, SMT "CPU" have > unpredicable power and his load depend on load parent CPU. >=20 > For example, for my case I see such condition (simpliy) on CPU 0 and 1 > (SMT of one real core) with rise load: >=20 > load 0.1 0.1 > load 0.2 0.2 > load 0.3 0.3 > load 0.4 0.4 > load 0.45 0.45 > load 0.48 0.48 > load 1.00 1.00\ Yes I know about HT. But how does this explain why I have 10% of CPU = idle? If I explicitly bind my single-threaded processes to the remaining CPU = cores (25-32), they start to receive expected 100% of CPU and overall = Idle decreases. I just expect scheduler to do the same for me.