From owner-freebsd-hackers@freebsd.org Thu Jan 24 18:11:12 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 368B514B86A0 for ; Thu, 24 Jan 2019 18:11:12 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-it1-x132.google.com (mail-it1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C05476C3D for ; Thu, 24 Jan 2019 18:11:11 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-it1-x132.google.com with SMTP id p197so5868464itp.0 for ; Thu, 24 Jan 2019 10:11:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wXUvV+8GT3e/ClD9IlLfUFgJ55fyudjqugOqJU0qpbE=; b=uFIByv+dlR0iUigwbZ81bXj70I9Kpl3hMnddHI97exfVcjekzrIuH+3Dn7x3harJ+0 Ii+LMQjqwtwpto4mAgoOO1M1jnz+tZD/OTI7PY742qeR3XP48sazFcFdQ0FsOiMwgTkt sWKmo6nQfG6mv9Ay4RuBLpr478Ni8kycu2BqadvVIxxWpRrymlrpS9qw7z1BZU7LrdU1 98kHvzeaLfTzIDN0II12tNbGgWJfo8X3Sncl4QYHq5acVKGO6aoxFE1FRA8bgHxYkTUN yFX62avznxnScXtTsQkL/Qt1rze4rCoZ9mfeUJ15DL9yPzdE6UKuJAK/IxaolAuTVhL5 b1zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wXUvV+8GT3e/ClD9IlLfUFgJ55fyudjqugOqJU0qpbE=; b=nvVEN5Xcztc6wSfCLGIwkLoL8Rh5JyYEPus3Gb/gbaU2OsNUEVDGRW4Twf9tHLg8x9 Tp4gIGV3FONCvxahz7p1iQTG9ihgF8wdnfs9F7GTKaRHOlczbWFZNAwpYPpJeo96zejM y6ohYOQCziu8FCaItCSD97wsW6hxbCBIrw7NHIUatUm3mXkwYrs/ZPmv5UJewCZjOXrm CnqLf8t5gLoUYUfGA18aUEHhfpPMiZc4u+MgKp9yhXkIw/CVjli7q8RzZFTrlF1ghLkl eNDSAhW/moz5faqxmJdqP9C+DQfCpHLXUQdsWnMHAb4Z/8H5U1SVkKXy7cbi9qiRNy6L dgZQ== X-Gm-Message-State: AJcUukfcY7knf2Zor/C0P4qg0btTQGT6RaxOzI/0GwlgQJAYiIkblbIA YzlVCsr3h83HHX6kTl4PctyXborX19MIDYBbcZj0YtY= X-Google-Smtp-Source: ALg8bN4glLt5uEj3BBq7Z6puJvuiBhkiD3B3v8jlXyqr0DUQpyxEWM49IJ/2gImph3VM/wNRes/F+VupZcXx3D9uhO0= X-Received: by 2002:a24:214a:: with SMTP id e71mr2200856ita.60.1548353470379; Thu, 24 Jan 2019 10:11:10 -0800 (PST) MIME-Version: 1.0 References: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> In-Reply-To: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> From: Zaphod Beeblebrox Date: Thu, 24 Jan 2019 13:10:59 -0500 Message-ID: Subject: Re: Scheduled ideas: Two threads with one "stone" To: Jan Bramkamp Cc: FreeBSD Hackers X-Rspamd-Queue-Id: 5C05476C3D X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uFIByv+d; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zbeeble@gmail.com designates 2607:f8b0:4864:20::132 as permitted sender) smtp.mailfrom=zbeeble@gmail.com X-Spamd-Result: default: False [-6.48 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.90)[-0.900,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.57)[ip: (-8.37), ipnet: 2607:f8b0::/32(-2.48), asn: 15169(-1.91), country: US(-0.08)]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 18:11:12 -0000 On Thu, Jan 24, 2019 at 12:36 PM Jan Bramkamp wrote: > On 24.01.19 17:32, Zaphod Beeblebrox wrote: > > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had > some > > recommendations for awhile --- namely hyperthreading is bad for secure > > systems. That-is-to-say this is not about recent vulnerabilities of > recent > > Intel flavoured CPUs, but rather some thoughts on a different solution > to a > > different problem. > > > > I was building world on my laptop when I noticed how sucky it had become. > > Now... I well know this is due to the fact that even a "nice -19 make > -j32 > > buildworld" will slow the normal priority userland down by about 1/2 > > because the thread that is "me" on the CPU is competing with one of the > > threads that is saturating the CPU. My normal priority thread will get > > scheduled, but it competes equally for resources with threads at much > lower > > priority. > > You can use `idprio 10 ...` to minimize the impact of CPU bound tasks. > If you want to go even further you could apply hierarchical resource > limits on the make process. The idprio chainloader puts make and all its > descendants into the idle scheduling class which allows normal time > sharing threads to preempt them. > No. This is insufficient to the task. If your goal is that the "idle" or "low priority" task does not effect the high priority tasks, this is insufficient w.r.t. hyperthreading. In hyperthreading (to discuss just one variant of many), two CPU cores share all of their functional bits. This is useful because, in general, CPU functional bits are fairly idle. Even if both threads on hyperthreaded pair are "runable" right now, they can be fetching from memory or, in the exreme, one could be issuing floating point and one could be issuing integer, IIRC. The problem I'm proposing to solve is when the two processes are of sufficiently differing effective priorities (however they get to that point) that they not be scheduled on the same hyperthreaded pair. The reason for this is that beyond scheduling the processes to the cores, we have no control over the resource contention of the cores --- and by that I claim that the lower priority process can force the higher priority process to wait as it consumes resources shared by the two cores. In an ideal world, the hyperthread interface on the cpu would allow some combination of prioritizing cores and their resource use, but that is not how Intel's chips currently work. To be clear, and this is easy to experience, find a FreeBSD desktop with a i7 (or similar) running X and (say) kde. Not something lightweight. Make sure it has enough RAM (16 or 32 big is probably good for this experiment). When idle, take a "feel" on the interface. No lag, things are spiffy. Now, go to /usr/src. Make sure it's there. Then run "nice -19 make -j32 buildworld" there. For extra credit, you can even use idprio if you like. Now... duplicate your feel test. The interface will be laggy. Things will be slower. If you look at the process table, it's not that your effective priorities are coming anywhere close to those from -19 ... or even if you use idprio... still laggy. What is happening here? Think about your processor. FreeBSD will ensure that your normal priority processes are scheduled first on a few of the 8 cores that show with an i7. But chances are very high that your processes are landing opposite low priority processes on the hyperthreaded pairs. This means that your effective processsor use is being cut in half (-ish) by the hyperthreaded processes grabbing the shared resources roughly half the time. You can game out having two processes ready and how often they fall on the same pair vs. falling on non-paired. You can even do the thought experiment of higher priority processes being placed on cores with pairs deliberately not used until all non-pared cores are in use --- urm... this is a big diversion... then we have the opportunity cost of the then partially unused resources ... etc. I should stop there. There are simple things that make a big difference before that digression produces many small things with even smaller effects.